SavedAdd your journal

Journals / Black Sea Journal of Engineering and Science / 2026 / Cilt: 9 Sayı: 1

Simultaneous Feature Selection and Hyperparameter Tuning of K-Nearest Neighbors for Glass Classification: A Comparative Study of Artificial Bee Colony, Simulated Annealing, and Hill Climbing

Sertaç Savaş(ERCİYES ÜNİVERSİTESİ, MÜHENDİSLİK FAKÜLTESİ, MEKATRONİK MÜHENDİSLİĞİ BÖLÜMÜ, BİLGİSAYAR DESTEKLİ TASARIM ANABİLİM DALI)0000-0001-8096-1140
Pages
433–443
Publication Date
2026-01-15
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

The results show that the population-based ABC algorithm, as a meta-heuristic approach, outperforms local search methods by avoiding local optima and effectively reducing the dimensionality of the data.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:Glass identification is essential in forensic science and industrial uses.
📊
Key Findings & Data:By reframing the classification problem as a multi-dimensional optimization task, the algorithms are assessed based on accuracy, precision, recall, and F1 score.
💡
Conclusion & Impact:The results show that the population-based ABC algorithm, as a meta-heuristic approach, outperforms local search methods by avoiding local optima and effectively reducing the dimensionality of the data.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:Glass classification · K-Nearest neighbors · Artificial bee colony

Abstract

Glass identification is essential in forensic science and industrial uses. However, the effectiveness of classification algorithms heavily relies on selecting the correct hyperparameters and relevant features. This study investigates the efficacy of three optimization algorithms—Artificial Bee Colony (ABC), Simulated Annealing (SA), and Hill Climbing (HC)—for optimizing the k value, which is the neighbor value of the k-Nearest Neighbors (k-NN) classifier, the distance measure expressing the proximity between two samples, and the feature subset of the UCI Glass Identification dataset. By reframing the classification problem as a multi-dimensional optimization task, the algorithms are assessed based on accuracy, precision, recall, and F1 score. The results show that the population-based ABC algorithm, as a meta-heuristic approach, outperforms local search methods by avoiding local optima and effectively reducing the dimensionality of the data. RI (refractive index) and Mg (magnesium oxide content) emerge as particularly discriminative features. The study demonstrates that combining automatic parameter tuning with feature selection can substantially enhance the predictive performance of k-NN in complex classification tasks.

Özet

Glass identification is essential in forensic science and industrial uses. However, the effectiveness of classification algorithms heavily relies on selecting the correct hyperparameters and relevant features. This study investigates the efficacy of three optimization algorithms—Artificial Bee Colony (ABC), Simulated Annealing (SA), and Hill Climbing (HC)—for optimizing the k value, which is the neighbor value of the k-Nearest Neighbors (k-NN) classifier, the distance measure expressing the proximity between two samples, and the feature subset of the UCI Glass Identification dataset. By reframing the classification problem as a multi-dimensional optimization task, the algorithms are assessed based on accuracy, precision, recall, and F1 score. The results show that the population-based ABC algorithm, as a meta-heuristic approach, outperforms local search methods by avoiding local optima and effectively reducing the dimensionality of the data. RI (refractive index) and Mg (magnesium oxide content) emerge as particularly discriminative features. The study demonstrates that combining automatic parameter tuning with feature selection can substantially enhance the predictive performance of k-NN in complex classification tasks.

🔖

Cite This Article

Savaş, S. (2026). Simultaneous Feature Selection and Hyperparameter Tuning of K-Nearest Neighbors for Glass Classification: A Comparative Study of Artificial Bee Colony, Simulated Annealing, and Hill Climbing. Black Sea Journal of Engineering and Science, Cilt: 9 Sayı: 1, 433–443. https://doi.org/10.34248/bsengineering.1841886

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (22)

  1. Al-Betar, M. A., Hammouri, A. I., Awadallah, M. A., & Doush, I. A. (2021). Binary β-hill climbing optimizer with S-shape transfer function for feature selection. Journal of Ambient Intelligence and Humanized Computing, 12(7), 7637–7665. https://doi.org/10.1007/s12652-020-02484-z
  2. Ay, Ş., Ekinci, E., & Garip, Z. (2023). A comparative analysis of meta-heuristic optimization algorithms for feature selection on ML-based classification of heart-related diseases. The Journal of Supercomputing, 79(11), 11797–11826. https://doi.org/10.1007/s11227-023-05132-3
  3. Bai, X., Zheng, Y., Lu, Y., & Shi, Y. (2024). Chain hybrid feature selection algorithm based on improved Grey Wolf Optimization algorithm. PLOS ONE, 19(10), e0311602. https://doi.org/10.1371/journal.pone.0311602
  4. Bhowmick, S., & Saha, A. (2023). Enhancing the performance of kNN for glass identification dataset using inverse distance weight, ReliefF ranking, and SMOTE. AIP Conference Proceedings, 2754(1), 020021. https://doi.org/10.1063/5.0161083
  5. Bouziane, M., Bouziane, A., Larguech, S., Naima, K., Haque, M. S., & Menni, Y. (2025). High-performance glass classification using advanced machine learning and deep learning algorithms with a comprehensive feature analysis. AIP Advances, 15(5), 055013. https://doi.org/10.1063/5.0260868
  6. Chantar, H., Tubishat, M., Essgaer, M., & Mirjalili, S. (2021). Hybrid binary dragonfly algorithm with simulated annealing for feature selection. SN Computer Science, 2(4), 295. https://doi.org/10.1007/s42979-021-00687-5
  7. Chen, Z., Xu, Y., Zhang, C., & Tang, M. (2024). Prediction of glass chemical composition and type identification based on machine learning algorithms. Applied Sciences, 14(10), 4017. https://doi.org/10.3390/app14104017
  8. Cover, T., & Hart, P. (1967). Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13(1), 21–27. https://doi.org/10.1109/TIT.1967.1053964
  9. Deza, M. M., & Deza, E. (2016). Encyclopedia of distances (4th ed.). Springer. https://doi.org/10.1007/978-3-662-52844-0
  10. German, B. (1987). Glass identification dataset. UCI Machine Learning Repository. https://doi.org/10.24432/C5WW2P
  11. Hussien, A. G., Oliva, D., Houssein, E. H., Juan, A. A., & Yu, X. (2020). Binary whale optimization algorithm for dimensionality reduction. Mathematics, 8(10), 1821. https://doi.org/10.3390/math8101821
  12. Karaboga, D. (2005). An idea based on honey bee swarm for numerical optimization (Technical Report TR06). Erciyes University, Engineering Faculty, Computer Engineering Department. https://abc.erciyes.edu.tr/pub/tr06_2005.pdf
  13. Kaspi, O., Israelsohn-Azulay, O., Yigal, Z., Rosengarten, H., Krmpotić, M., Gouasmia, S., Radović, I. B., Jalkanen, P., Liski, A., Mizohata, K., Räisänen, J., Kasztovszky, Z., Harsányi, I., Acharya, R., Pujari, P. K., Mihály, M., Braun, M., Shabi, N., Girshevitz, O., & Senderowitz, H. (2023). Toward developing techniques—Agnostic machine learning classification models for forensically relevant glass fragments. Journal of Chemical Information and Modeling, 63(1), 87–100. https://doi.org/10.1021/acs.jcim.2c01362
  14. Khan, M. A., Mazhar, T., Yaqoob, M. M., Khan, M. B., Saudagar, A. K. J., Ghadi, Y. Y., Khattak, U. F., & Shahid, M. (2024). Optimal feature selection for heart disease prediction using modified artificial bee colony (M-ABC) and k-nearest neighbors (KNN). Scientific Reports, 14, 26241. https://doi.org/10.1038/s41598-024-78021-1
  15. Kirkpatrick, S., Gelatt, C. D., Jr., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220(4598), 671–680. https://doi.org/10.1126/science.220.4598.671
  16. Li, G., Shi, M., Hao, T., Wang, P., Ma, Y., & Han, Y. (2024). A hybrid framework for ancient glass classification and recognition: Combining binary nutcracker optimization algorithm with KNN method. In 2024 9th International Conference on Control and Robotics Engineering (ICCRE) (pp. 345–351). IEEE. https://doi.org/10.1109/ICCRE61448.2024.10589876
  17. Rachmatullah, M. I. C. (2022). The application of repeated SMOTE for multi-class classification on imbalanced data. MATRIK: Jurnal Manajemen, Teknik Informatika ve Rekayasa Komputer, 22(1), 13–24. https://doi.org/10.30812/matrik.v22i1.1803
  18. Russell, S. J., & Norvig, P. (2020). Artificial intelligence: A modern approach (4th ed.). Pearson.
  19. Schiezaro, M., & Pedrini, H. (2013). Data feature selection based on artificial bee colony algorithm. EURASIP Journal on Image and Video Processing, 2013, 47. https://doi.org/10.1186/1687-5281-2013-47
  20. Uzer, M. S., Yilmaz, N., & Inan, O. (2013). Feature selection method based on artificial bee colony algorithm and support vector machines for medical datasets classification. The Scientific World Journal, 2013, 419187. https://doi.org/10.1155/2013/419187
  21. Visweshwaran, S., Anbazhagan, M., & Ganesh, K. (2024). An empirical study on ML models with glass classification dataset. In H. Sharma, V. Shrivastava, A. K. Tripathi, & L. Wang (Eds.), Communication and Intelligent Systems. ICCIS 2023. Lecture Notes in Networks and Systems (Vol. 968). Springer. https://doi.org/10.1007/978-981-97-2079-8_30
  22. Wang, Z., Lu, Y., & Li, T. (2023). Ancient glass classification based on random forest algorithm and decision tree model. Highlights in Science, Engineering and Technology, 34, 344–351. https://doi.org/10.54097/hset.v34i.5492