Unsupervised feature selection based on the two-dimensional principal component analysis and bipartite graph for face image classification

Articles in Press

Document Type : Research Paper

Authors

1 Department of Electrical Engineering, Lorestan University, Khorramabad, Iran.

2 Department of Computer Engineering, Lorestan University, Khorramabad, Iran.

Abstract

In this paper, we propose a new matrix-based feature selection method, called UFS2DPCA, which leverages the hidden knowledge in orthogonal features obtained from two-dimensional principal component analysis (2DPCA) to perform accurate unsupervised feature selection. The UFS2DPCA algorithm first uses 2DPCA to directly extract uncorrelated and orthogonal features from the two-dimensional image datasets. We then compute the correlation similarity between the main and extracted features. Finally, a weighted bipartite graph is constructed using two sets of features, and the best features are selected using the fast LAPJV algorithm. The selected features are classified using the K-Nearest Neighbor (KNN) classifier. To ensure statistical significance, the Friedman test is applied to compare the performance of UFS2DPCA with other methods. The algorithm is evaluated on four well-known image datasets: Jaffe, Yale, ORL, and pixraw10P. Key performance metrics such as accuracy, normalized mutual information (NMI), precision, recall, and F-measure are used for evaluation. The experimental results show that UFS2DPCA consistently outperforms other state-of-the-art unsupervised feature selection methods. For example, UFS2DPCA achieves an average NMI of 0.9244 and average accuracy of 0.9033 on the pixraw10P face image dataset that has 10000 features. Similarly, it demonstrates superior performance in accuracy, recall, Precision, F-measure, and NMI across all datasets.

Keywords

Main Subjects

References

[1] Ahamad, M., & Ahmad, N. (2021). Machine Learning Approaches to Digital Learning Performance Analysis.  International Journal of Computing and Digital Systems, 10(1), 963{971. https://doi.org/10.12785/IJCDS/100187
[2] Beiranvand, F., Mehrdad, V., & Dowlatshahi, M. B. (2022). Unsupervised feature selection for image classi cation: A bipartite matching-based principal component analysis approach. Knowledge-Based Systems, 109085.
https://doi.org/10.1016/j.knosys.2022.109085
[3] Derigs, U. (1981). A shortest augmenting path method for solving minimal perfect matching problems. Networks, 11(4), 379{390. https://doi.org/10.1002/net.3230110407
[4] Derrac, J., Garca, S., Molina, D., & Herrera, F. (2011). A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm and Evolutionary Computation, 1(1), 3{18. https://doi.org/10.1016/j.swevo.2011.02.002
[5] Fankhauser, S., Riesen, K., Bunke, H., & Dickinson, P. (2012). Suboptimal graph isomorphism using bipartite matching. International Journal of Pattern Recognition and Arti cial Intelligence, 26(6), 1{26. https://doi.org/10.1142/S0218001412500139
[6] Guo, J., & Zhu, W. (2018). Dependence guided unsupervised feature selection. 32nd AAAI Conference on Arti cial Intelligence, 2232{2239.
[7] Guyon, I. (2008). Practical feature selection: From correlation to causality. PASCAL Network. http://eprints.pascal-network.org/archive/00004038
[8] Han, K., Wang, Y., Zhang, C., Li, C., & Xu, C. (2018). Autoencoder inspired unsupervised feature selection. ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, 2941{2945. https://doi.org/10.1109/ICASSP.2018.8462261
[9] Hashemi, A., Bagher, M., & Nezamabadi-pour, H. (2020). MFS-MCDM: Multi-label feature selection using multi-criteria decision making. Knowledge-Based Systems, 206, 106365. https://doi.org/10.1016/j.knosys.2020.106365
[10] Hashemi, A., Bagher, M., & Nezamabadi-pour, H. (2021a). VMFS: A VIKORbased multi-target feature selection. Expert Systems with Applications, 182, 115224. https://doi.org/10.1016/j.eswa.2021.115224
[11] Hashemi, A., Bagher, M., & Nezamabadi-pour, H. (2021b). An ecient Paretobased feature selection algorithm for multi-label classi cation. Information Sciences, 581, 428{447. https://doi.org/10.1016/j.ins.2021.09.052
[12] Hashemi, A., Dowlatshahi, M. B., & Nezamabadi-Pour, H. (2021). A bipartite matching-based feature selection for multi-label learning. International Journal of Machine Learning and Cybernetics, 12(2), 459{475. https://doi.org/10.1007/s13042-020-01180-w
[13] Hashemi, A., Dowlatshahi, M. B., & Nezamabadi-pour, H. (2021). Ensemble of feature selection algorithms: A multi-criteria decision-making approach. International Journal of Machine Learning and Cybernetics. https://doi.org/10.1007/s13042-021-01347-z
[14] Huang, D., Cai, X., & Wang, C. D. (2019). Unsupervised feature selection with multi-subspace randomization and collaboration. Knowledge-Based Systems, 182. https://doi.org/10.1016/j.knosys.2019.07.027
[15] Huang, P., & Yang, X. (2022). Unsupervised feature selection via adaptive graph and dependency score. Pattern Recognition, 127, 108622. https://doi.org/10.1016/j.patcog.2022.108622
[16] https://www.kaggle.com/datasets/tavarez/the-orl-database-for-training-andtesting
[17] https://jundongl.github.io/scikit-feature/datasets.html
[18] Jiang, L., Cai, Z., Wang, D., & Jiang, S. (2007). Survey of improving K-nearest-neighbor for classi cation. Proceedings - Fourth International Conference on Fuzzy Systems and Knowledge Discovery, FSKD 2007, 1, 679{683. https://doi.org/10.1109/FSKD.2007.552
[19] Jones, W., Chawdhary, A., & King, A. (2017). Optimising the Volgenant{Jonker algorithm for approximating graph edit distance. Pattern Recognition Letters, 87, 47{54. https://doi.org/10.1016/j.patrec.2016.07.024
[20] Jonker, R., & Volgenant, T. (1988). A shortest augmenting path algorithm for dense and sparse linear assignment problems. In DGOR/NSOR: Papers of the 16th Annual Meeting of DGOR in Cooperation with NSOR/Vortrage der 16. Jahrestagung der DGOR zusammen mit der NSOR 1988 (pp. 622-622). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-73778-7-164
[21] Karami, S., Saberi-Movahed, F., Tiwari, P., & Marttinen, P. (2023). Unsupervised feature selection based on variance{covariance subspace distance. Neural Networks, 166, 188{203. https://doi.org/10.1016/j.neunet.2023.06.018
[22] Khalid, S., Khalil, T., & Nasreen, S. (2014). A survey of feature selection and feature extraction techniques in machine learning. In Proceedings of 2014 Science and Information Conference (SAI 2014) (pp. 372{378). https://doi.org/10.1109/SAI.2014.6918213
[23] Kong, H., Wang, L., Teoh, E. K., Li, X., Wang, J. G., & Venkateswarlu, R. (2005). Generalized 2D principal component analysis for face image representation and recognition. Neural Networks, 18(5{6), 585{594. https://doi.org/10.1016/j.neunet.2005.06.041
[24] Li, J., Chen, J., Qi, F., Dan, T., Weng, W., Zhang, B., Yuan, H., Cai, H., & Zhong, C. (2022). Two-dimensional unsupervised feature selection via sparse feature  lter. IEEE Transactions on Cybernetics, 1{13. https://doi.org/10.1109/TCYB.2022.3162908
[25] Li, J., Liang, X., & Li, P. (2020). Two-dimensional semi-supervised feature selection. Proceedings of the Conference, 280{287.
[26] Li, X., Zhang, H., Zhang, R., Liu, Y., & Nie, F. (2019). Generalized uncorrelated regression with adaptive graph for unsupervised feature selection. IEEE Transactions on Neural Networks and Learning Systems, 30(5), 1587{1595. https://doi.org/10.1109/TNNLS.2018.2868847
[27] Lu, H., Plataniotis, K. N., & Venetsanopoulos, A. N. (2011). A survey of multilinear subspace learning for tensor data. Pattern Recognition, 44(7), 1540{1551. https://doi.org/10.1016/j.patcog.2011.01.004
[28] Hashemi, A., Dowlatshahi, M. B., & Nezamabadi-Pour, H. (2021). A bipartite matching-based feature selection for multi-label learning. International Journal of Machine Learning and Cybernetics, 12(2), 459{475. https://doi.org/10.1007/s13042-020-01180-w
[29] Yang, J., Zhang, D., Frangi, A. F., & Yang, J. Y. (2004). Two-Dimensional PCA: A New Approach to Appearance-Based Face Representation and Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(1), 131{137.
https://doi.org/10.1109/TPAMI.2004.161097
[30] Miao, J., & Niu, L. (2016). A survey on feature selection. Procedia Computer Science, 91, 919{926. https://doi.org/10.1016/j.procs.2016.07.111
[31] Nixon, M. (2007). Feature extraction & image processing (2nd ed.).
[32] Paniri, M., Dowlatshahi, M. B., & Nezamabadi-pour, H. (2021). Ant-TD: Ant colony optimization plus temporal di erence reinforcement learning for multi-label feature selection. Swarm and Evolutionary Computation, 64, 100892. https://doi.org/10.1016/j.swevo.2021.100892
[33] Rahimi, Z., Taghipour, K., Khadivi, S., & Afhami, N. (2012). Document and sentence alignment in comparable corpora using bipartite graph matching. 2012 6th International Symposium on Telecommunications, IST 2012, 817{821. https://doi.org/10.1109/ISTEL.2012.6483098
[34] Sanguansat, P. (Ed.). (2012). Principal component analysis. BoD{Books on Demand.
[35] Schrijver, A. (2012). A course in combinatorial optimization.
[36] Tabakhi, S., Moradi, P., & Akhlaghian, F. (2014). An unsupervised feature selection algorithm based on ant colony optimization. Engineering Applications of Arti cial Intelligence, 32, 112{123. https://doi.org/10.1016/j.engappai.2014.03.007
[37] Tan, H., Zhang, X., Guan, N., Tao, D., & Huang, X. (2015). Two-Dimensional Euler PCA, 548{559.
[38] Tharwat, A. (2018). Classi cation assessment methods. Applied Computing and Informatics, 17(1), 168{192. https://doi.org/10.1016/j.aci.2018.08.003
[39] Vasudev, C. (2006). Graph theory with applications. New Age International.
[40] Wahid, A., Khan, D. M., Hussain, I., Khan, S. A., & Khan, Z. (2022). Unsupervised feature selection with robust data reconstruction (UFSRDR) and outlier detection. Expert Systems with Applications, 201, 117008. https://doi.org/10.1016/j.eswa.2022.117008
[41] Wang, C., Wang, J., Gu, Z., Wei, J. M., & Liu, J. (2024). Unsupervised feature selection by learning exponential weights. Pattern Recognition, 148, 110183. https://doi.org/10.1016/j.patcog.2023.110183
[42] Wang, C., Yin, B., Bai, X., & Sun, Y. (2008). Color face recognition based on 2DPCA. Proceedings - International Conference on Pattern Recognition, 8{11. https://doi.org/10.1109/ICPR.2008.4761093
[43] Wang, L. (2019). Research and Implementation of Machine Learning Classi er Based on KNN. IOP Conference Series: Materials Science and Engineering, 677(5), 0{5. https://doi.org/10.1088/1757-899X/677/5/052038
[44] Wang, Q., & Gao, Q. (2016). Robust 2DPCA and its application. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 1152{1158. https://doi.org/10.1109/CVPRW.2016.147
[45] Wang, X., Li, H., Zhou, Y., & Zheng, H. (2021). Lanczos algorithm for 2DPCA. Journal of Physics: Conference Series, 2025(1). https://doi.org/10.1088/1742-6596/2025/1/012035
[46] Xiang, L., Chen, H., Yin, T., Horng, S. J., & Li, T. (2024). Unsupervised feature selection based on bipartite graph and low-redundant regularization. Knowledge-Based Systems, 302, 112379. https://doi.org/10.1016/j.knosys.2024.112379
[47] Xie, J., Wang, M., Xu, S., Huang, Z., & Grant, P. W. (2021). The unsupervised feature selection algorithms based on standard deviation and cosine similarity for genomic data analysis. Frontiers in Genetics, 12, 684100. https://doi.org/10.3389/fgene.2021.684100
[48] Yang, J., & Liu, C. (2007). Horizontal and vertical 2DPCA-based discriminant analysis for face veri cation on a large-scale database. IEEE Transactions on Informa-tion Forensics and Security, 2(4), 781{792. https://doi.org/10.1109/TIFS.2007.910239
[49] Yang, J., Zhang, D., Frangi, A. F., & Yang, J. Y. (2004). Two-dimensional PCA: A new approach to appearance-based face representation and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(1), 131{137. https://doi.org/10.1109/TPAMI.2004.1261097
[50] Yu, L., & Liu, H. (2004). Ecient feature selection via analysis of relevance and redundancy. The Journal of Machine Learning Research, 5, 1205{1224.
[51] Yuan, H., Li, J., Lai, L. L., & Tang, Y. Y. (2019). Joint sparse matrix regression and nonnegative spectral analysis for two-dimensional unsupervised feature selection. Pattern Recognition, 89, 119{133. https://doi.org/10.1016/j.patcog.2019.01.014
[52] Yuan, H., Li, J., Lai, L. L., & Tang, Y. Y. (2020). Low-rank matrix regression for image feature extraction and feature selection. Information Sciences, 522, 214{226. https://doi.org/10.1016/j.ins.2020.02.070
[53] Zhang, F., Yang, J., Qian, J., & Xu, Y. (2015). Nuclear norm-based 2-DPCA for extracting features from images. IEEE Transactions on Neural Networks and Learning Systems, 26(10), 2247{2260.
[54] Zhou, X., & Wang, J. (2015). Feature selection for image classi cation based on a new ranking criterion. Journal of Computer and Communications, 3(3), 74{79. https://doi.org/10.4236/jcc.2015.33013
[55] Zhu, P., Zuo, W., Zhang, L., Hu, Q., & Shiu, S. C. K. (2015). Unsupervised feature selection by regularized self-representation. Pattern Recognition, 48(2), 438{446. https://doi.org/10.1016/j.patcog.2014.08.006
Journal of Mahani Mathematical Research

Articles in Press, Accepted Manuscript
Available Online from 01 February 2025

Files

History

  • Receive Date: 11 June 2024
  • Revise Date: 22 September 2024
  • Accept Date: 17 January 2025

Share

How to cite

Statistics