A New Hybrid Filter-Wrapper Feature Selection using Equilibrium Optimizer and Simulated Annealing

Document Type : Research Paper

Authors

Department of Computer Science, Shahid Bahonar University of Kerman, Kerman, Iran

Abstract

Data dimensions and networks have grown exponentially with the Internet and communications. The challenge of high-dimensional data is increasing for machine learning and data science. This paper presents a hybrid filter-wrapper feature selection method based on Equilibrium Optimization (EO) and Simulated Annealing (SA). The proposed algorithm is named Filter-Wrapper Binary Equilibrium Optimizer Simulated Annealing (FWBEOSA). We used SA to solve the local optimal problem so that EO could be more accurate and better able to select the best subset of features. FWBEOSA utilizes a filtering phase that increases accuracy as well as reduces the number of selected features. The proposed method is evaluated on 17 standard UCI datasets using Support Vector Machine (SVM) and K-Nearest Neighbors (KNN) classifiers and compared with ten state-of-the-art algorithms (i.e., Binary Equilibrium Optimizer (BEO), Binary Gray Wolf Optimization (BGWO), Binary Swarm Slap Algorithm (BSSA), Binary Genetic Algorithm (BGA), Binary Particle Swarm Optimization (BPSO), Binary Social Mimic Optimization (BSMO), Binary Atom Search Optimization (BASO), Modified Flower Pollination Algorithm (MFPA), Bar Bones Particle Swarm Optimization (BBPSO) and Two-phase Mutation Gray Wolf Optimization (TMGWO)). Based on the results of the SVM classification, the highest level of accuracy was achieved in 13 out of 17 data sets (76%), and the lowest number of selected features was achieved in 15 out of 17 data sets (88%). Furthermore, the proposed algorithm using class KNN achieved the highest accuracy rate in 14 datasets (82%) and the lowest selective feature rate in 13 datasets (76%).

Keywords

Main Subjects

References

[1] Adamu, A., Abdullahi, M., Junaidu, S. B., & Hassan, I. H. (2021). An hybrid particle swarm optimization with crow search algorithm for feature selection. Machine Learning with Applications, 6, 100108. https://doi.org/10.1016/j.mlwa.2021.100108
[2] Abualigah, L., & Dulaimi, A. J. (2021). A novel feature selection method for data mining tasks using hybrid sine cosine algorithm and genetic algorithm. Cluster Computing, 24, 2161-2176. https://doi.org/10.1007/s10586-021-03254-y
[3] Abdel-Basset, M., Ding, W., & El-Shahat, D. (2021). A hybrid Harris Hawks optimization algorithm with simulated annealing for feature selection. Artificial Intelligence Review, 54, 593-637. https://doi.org/10.1007/s10462-020-09860-3
[4] Abdel-Basset, M., El-Shahat, D., El-Henawy, I., De Albuquerque, V. H. C., & Mirjalili, S. (2020). A new fusion of grey wolf optimizer algorithm with a twophase mutation for feature selection. Expert Systems with Applications, 139, 112824. https://doi.org/10.1016/j.eswa.2019.112824
[5] Agrawal, P., Abutarboush, H. F., Ganesh, T., & Mohamed, A. W. (2021). Metaheuristic algorithms on feature selection: A survey of one decade of research (2009-2019). Ieee Access, 9, 26766-26791. https://doi.org/10.1109/ACCESS.2021.3056407
[6] Agrawal, P., Ganesh, T., & Mohamed, A. W. (2021). Chaotic gaining sharing knowledgebased optimization algorithm: an improved metaheuristic algorithm for feature selection. Soft Computing, 25(14), 9505-9528. https://doi.org/10.1007/s00500-021-05874-3
[7] Al-Wajih, R., Abdulkadir, S. J., Aziz, N., Al-Tashi, Q., & Talpur, N. (2021). Hybrid binary grey wolf with Harris hawks optimizer for feature selection. IEEE Access, 9, 31662-31677. https://doi.org/10.1109/ACCESS.2021.3060096
[8] Ahmed, S., Ghosh, K. K., Mirjalili, S., & Sarkar, R. (2021). AIEOU: Automata-based improved equilibrium optimizer with U-shaped transfer function for feature selection. Knowledge-Based Systems, 228, 107283. https://doi.org/10.1016/j.knosys.2021.107283
[9] Bacanin, N., Venkatachalam, K., Bezdan, T., Zivkovic, M., & Abouhawwash, M. (2023). A novel firefly algorithm approach for efficient feature selection with COVID-19 dataset. Microprocessors and Microsystems, 98, 104778. https://doi.org/10.1016/j.micpro.2023.104778
[10] Balochian, S., & Baloochian, H. (2019). Social mimic optimization algorithm and engineering applications. Expert Systems with Applications, 134, 178-191. https://doi.org/10.1016/j.eswa.2019.05.035 
[11] Chandrashekar, G., & Sahin, F. (2014). A survey on feature selection methods. Computers & Electrical Engineering, 40(1), 16-28. https://doi.org/10.1016/j.compeleceng.2013.11.024
[12] Chantar, H., Tubishat, M., Essgaer, M., & Mirjalili, S. (2021). Hybrid binary dragonfly algorithm with simulated annealing for feature selection. SN computer science, 2(4), 295. https://doi.org/10.1007/s42979-021-00687-5
[13] Ding, Y., Zhou, K., & Bi, W. (2020). Feature selection based on hybridization of genetic algorithm and competitive swarm optimizer. Soft Computing, 24, 11663-11672. https://doi.org/10.1007/s00500-019-04628-6
[14] Elgamal, Z. M., Yasin, N. B. M., Tubishat, M., Alswaitti, M., & Mirjalili, S. (2020). An improved harris hawks optimization algorithm with simulated annealing for feature selection in the medical field. IEEE access, 8, 186638-186652. https://doi.org/10.1109/ACCESS.2020.3029728
[15] Faramarzi, A., Heidarinejad, M., Stephens, B., & Mirjalili, S. (2020). Equilibrium optimizer: A novel optimization algorithm. Knowledge-Based Systems, 191, 105190. https://doi.org/10.1016/j.knosys.2019.105190
[16] Ghosh, K. K., Guha, R., Bera, S. K., Sarkar, R., & Mirjalili, S. (2020). BEO: Binary equilibrium optimizer combined with simulated annealing for feature selection. Research Square. https://doi.org/10.21203/rs.3.rs-28683/v1
[17] Gao, Y., Zhou, Y., & Luo, Q. (2020). An efficient binary equilibrium optimizer algorithm for feature selection. IEEE Access, 8, 140936-140963. https://doi.org/10.1109/ACCESS.2020.3013617
[18] Hussain, K., Neggaz, N., Zhu, W., & Houssein, E. H. (2021). An efficient hybrid sinecosine Harris hawks optimization for low and high-dimensional feature selection. Expert Systems with Applications, 176, 114778. https://doi.org/10.1016/j.eswa.2021.114778
[19] Igiri, C. P., Singh, Y., & Poonia, R. C. (2020). A review study of modified swarm intelligence: particle swarm optimization, firefly, bat and gray wolf optimizer algorithms. Recent Advances in Computer Science and Communications (Formerly: Recent Patents on Computer Science), 13(1), 5-12. https://doi.org/10.2174/2213275912666190101120202
[20] Ibrahim, R. A., Ewees, A. A., Oliva, D., Abd Elaziz, M., & Lu, S. (2019). Improved salp swarm algorithm based on particle swarm optimization for feature selection. Journal of Ambient Intelligence and Humanized Computing, 10, 3155-3169. https://doi.org/10.1007/s12652-018-1031-9
[21] Ibrahim, R. A., Abd Elaziz, M., Ewees, A. A., El-Abd, M., & Lu, S. (2021). New feature selection paradigm based on hyper-heuristic technique. Applied Mathematical Modelling, 98, 14-37. https://doi.org/10.1016/j.apm.2021.04.018
[22] Johnson, J. M., & Rahmat-Samii, Y. (1994, June). Genetic algorithm optimization and its application to antenna design. In Proceedings of IEEE Antennas and Propagation Society International Symposium and URSI National Radio Science Meeting (Vol. 1, pp. 326-329). IEEE. https://doi.org/10.1109/APS.1994.407746
[23] Kennedy, J. (2003, April). Bare bones particle swarms. In Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS’03 (Cat. No. 03EX706) (pp. 80-87). IEEE. https://doi.org/10.1109/SIS.2003.1202251
[24] Kirkpatrick, S., Gelatt Jr, C. D., & Vecchi, M. P. (1983). Optimization by simulated annealing. science, 220(4598), 671-680. https://doi.org/10.1126/science.220.4598.671
[25] Liu, Y., Zou, X., Ma, S., Avdeev, M., & Shi, S. (2022). Feature selection method reducing correlations among features by embedding domain knowledge. Acta Materialia, 238, 118195. https://doi.org/10.1016/j.actamat.2022.118195
[26] Moslehi, F., & Haeri, A. (2020). A novel hybrid wrapper–filter approach based on genetic algorithm, particle swarm optimization for feature subset selection. Journal of Ambient Intelligence and Humanized Computing, 11, 1105-1127. https://doi.org/10.1007/s12652-019-01364-5
[27] Mafarja, M., Qasem, A., Heidari, A. A., Aljarah, I., Faris, H., & Mirjalili, S. (2020). Efficient hybrid nature-inspired binary optimizers for feature selection. Cognitive Computation, 12, 150-175. https://doi.org/10.1007/s12559-019-09668-6
[28] Nssibi, M., Manita, G., & Korbaa, O. (2023). Advances in nature-inspired metaheuristic optimization for feature selection problem: A comprehensive survey. Computer Science Review, 49, 100559. https://doi.org/10.1016/j.cosrev.2023.100559
[29] Piri, J., Mohapatra, P., Singh, H. K. R., Acharya, B., & Patra, T. K. (2022). An Enhanced Binary Multiobjective Hybrid Filter-Wrapper Chimp Optimization Based Feature Selection Method for COVID-19 Patient Health Prediction. IEEE Access, 10,
100376-100396. https://doi.org/10.1109/ACCESS.2022.3203400
[30] Sharifai, A. G., & Zainol, Z. B. (2021). Multiple filter-based rankers to guide hybrid grasshopper optimization algorithm and simulated annealing for feature selection with high dimensional multi-class imbalanced datasets. IEEE Access, 9, 74127-74142. https://doi.org/10.1109/ACCESS.2021.3081366
[31] Sayed, G. I., Khoriba, G., & Haggag, M. H. (2022). A novel chaotic equilibrium optimizer algorithm with S-shaped and V-shaped transfer functions for feature selection. Journal of Ambient Intelligence and Humanized Computing, 1-26. https://doi.org/10.1007/s12652-021-03151-7
[32] Shambour, M. D. K. Y., Abusnaina, A. A., & Alsalibi, A. I. (2019). Modified global flower pollination algorithm and its application for optimization problems. Interdisciplinary Sciences: Computational Life Sciences, 11, 496-507. https://doi.org/10.1007/s12539-018-0295-2
[33] Tiwari, A., & Chaturvedi, A. (2022). A hybrid feature selection approach based on information theory and dynamic butterfly optimization algorithm for data classification. Expert Systems with Applications, 196, 116621. https://doi.org/10.1016/j.eswa.2022.116621
[34] Thakkar, A., & Lohiya, R. (2022). A survey on intrusion detection system: feature selection, model, performance measures, application perspective, challenges, and future research directions. Artificial Intelligence Review, 55(1), 453-563.
https://doi.org/10.1007/s10462-021-10037-9
[35] Too, J., & Mirjalili, S. (2021). General learning equilibrium optimizer: a new feature selection method for biological data classification. Applied Artificial Intelligence, 35(3), 247-263. https://doi.org/10.1080/08839514.2020.1861407
[36] Too, J., & Abdullah, A. R. (2020). Chaotic atom search optimization for feature selection. Arabian Journal for Science and Engineering, 45(8), 6063-6079. https://doi.org/10.1007/s13369-020-04486-7
[37] Tanveer, M., Rajani, T., Rastogi, R., Shao, Y. H., & Ganaie, M. A. (2022). Comprehensive review on twin support vector machines. Annals of Operations Research, 1-46. https://doi.org/10.1007/s10479-022-04575-w
[38] Thaher, T., Chantar, H., Too, J., Mafarja, M., Turabieh, H., & Houssein, E. H. (2022). Boolean Particle Swarm Optimization with various Evolutionary Population Dynamics approaches for feature selection problems. Expert Systems with Applications, 195, 116550. https://doi.org/10.1016/j.eswa.2022.116550
[39] Unler, A., Murat, A., & Chinnam, R. B. (2011). mr2PSO: A maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification. Information Sciences, 181(20), 4625-4641.
https://doi.org/10.1016/j.ins.2010.05.037
[40] Vergara, J. R., & Estévez, P. A. (2014). A review of feature selection methods based on mutual information. Neural computing and applications, 24, 175-186. https://doi.org/10.1007/s00521-013-1368-0
[41] Wang, L., Jiang, S., & Jiang, S. (2021). A feature selection method via analysis of relevance, redundancy, and interaction. Expert Systems with Applications, 183, 115365. https://doi.org/10.1016/j.eswa.2021.115365
[42] Zivkovic, M., Stoean, C., Chhabra, A., Budimirovic, N., Petrovic, A., & Bacanin, N. (2022). Novel improved salp swarm algorithm: An application for feature selection. Sensors, 22(5), 1711. https://doi.org/10.3390/s22051711
Journal of Mahani Mathematical Research
Volume 13, Issue 1 - Serial Number 26
November 2023
Pages 293-332

Files

History

  • Receive Date: 02 March 2023
  • Revise Date: 24 July 2023
  • Accept Date: 14 September 2023

Share

How to cite

Statistics