SavedAdd your journal

Journals / International Journal of Assessment Tools in Education / 2021 / Cilt: 8 - Sayı: 4

Classification of Scale Items with Exploratory Graph Analysis and Machine Learning Methods

İlhan KOYUNCU
Abdullah Faruk KILIÇ
Pages
928–947
Publication Date
2022-10-18
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

The results revealed that machine learning methods can be used for similar conditions to examine whether the distribution of items across factors is done accurately or not.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:In exploratory factor analysis, although the researchers decide which items belong to which factors by considering statistical results, the decisions taken sometimes can be subjective in case of having items with similar factor loadings and complex factor structures.
📊
Key Findings & Data:A Monte Carlo simulation was performed with a total number of 96 simulation conditions including average factor loadings, sample size, number of items per dimension, number of dimensions, and distribution of data.
💡
Conclusion & Impact:The results revealed that machine learning methods can be used for similar conditions to examine whether the distribution of items across factors is done accurately or not.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:PISA · Machine learning · Exploratory factor analysis

Abstract

In exploratory factor analysis, although the researchers decide which items belong to which factors by considering statistical results, the decisions taken sometimes can be subjective in case of having items with similar factor loadings and complex factor structures. The aim of this study was to examine the validity of classifying items into dimensions with exploratory graph analysis (EGA), which has been used in determining the number of dimensions in recent years and machine learning methods. A Monte Carlo simulation was performed with a total number of 96 simulation conditions including average factor loadings, sample size, number of items per dimension, number of dimensions, and distribution of data. Percent correct and Kappa concordance values were used in the evaluation of the methods. When the findings obtained for different conditions were evaluated together, it was seen that the machine learning methods gave results comparable to those of EGA. Machine learning methods showed high performance in terms of percent correct values, especially in small and medium-sized samples. In all conditions where the average factor loading was .70, BayesNet, Naive Bayes, RandomForest, and RseslibKnn methods showed accurate classification performances above 80% like EGA method. BayesNet, Simple Logistic and RBFNetwork methods also demonstrated acceptable or high performance under many conditions. In general, Kappa concordance values also supported these results. The results revealed that machine learning methods can be used for similar conditions to examine whether the distribution of items across factors is done accurately or not.

Özet

In exploratory factor analysis, although the researchers decide which items belong to which factors by considering statistical results, the decisions taken sometimes can be subjective in case of having items with similar factor loadings and complex factor structures. The aim of this study was to examine the validity of classifying items into dimensions with exploratory graph analysis (EGA), which has been used in determining the number of dimensions in recent years and machine learning methods. A Monte Carlo simulation was performed with a total number of 96 simulation conditions including average factor loadings, sample size, number of items per dimension, number of dimensions, and distribution of data. Percent correct and Kappa concordance values were used in the evaluation of the methods. When the findings obtained for different conditions were evaluated together, it was seen that the machine learning methods gave results comparable to those of EGA. Machine learning methods showed high performance in terms of percent correct values, especially in small and medium-sized samples. In all conditions where the average factor loading was .70, BayesNet, Naive Bayes, RandomForest, and RseslibKnn methods showed accurate classification performances above 80% like EGA method. BayesNet, Simple Logistic and RBFNetwork methods also demonstrated acceptable or high performance under many conditions. In general, Kappa concordance values also supported these results. The results revealed that machine learning methods can be used for similar conditions to examine whether the distribution of items across factors is done accurately or not.

🔖

Cite This Article

KOYUNCU, İ., KILIÇ, A. F. (2022). Classification of Scale Items with Exploratory Graph Analysis and Machine Learning Methods. International Journal of Assessment Tools in Education, Cilt: 8 - Sayı: 4, 928–947. https://ajindex.com/articles/540099

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (11)

  1. Aha, D. W., Kibler, D., & Albert, M. K. (1991). Instance-based learning algorithms. Machine Learning 6, 37-66.
  2. Akpınar, H. (2014). Veri madenciliği veri analizi [Data mining data analysis]. Papatya Yayınları.
  3. Alpaydin, E. (2010). Introduction to machine learning: Adaptive computation and machine learning series. MIT Press.
  4. Auerswald, M., & Moshagen, M. (2019). How to determine the number of factors to retain in exploratory factor analysis: A comparison of extraction methods under realistic conditions. Psychological Methods, 24(4), 468–491. https://doi:10.1037/met0000200 Azqueta-Gavaldón, A. (2017). Developing news-based economic policy uncertainty index with unsupervised machine learning. Economics Letters, 158, 47-50.
  5. Baker, R. S. J. (2010). Machine learning for education. International Encyclopedia of Education, 7(3), 112-118.
  6. Baldi, P., & Hornik, K. (1989). Neural networks and principal component analysis: Learning from examples without local minima. Neural Networks, 2, 53-58.
  7. Bandalos, D. L., & Leite, W. (2013). Use of Monte Carlo studies in structural equation modeling research. In G. R. Hancock & R. O. Mueller (Eds.), Structural equation modeling: A second course (2nd ed.). Information Age. Barker, K., Trafalis, T., & Rhoads, T. R. (2004). Learning from student data. In Proceedings of the 2004 Systems and Information Engineering Design Symposium (pp. 79-86).
  8. IEEE. Beauducel, A., & Herzberg, P. Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling: A Multidisciplinary Journal, 13(2), 186 203. https://doi.org/10.1207/s15328007sem1302_2
  9. Beleites, C., Neugebauer, U., Bocklitz, T., Krafft, C., & Popp, J. (2013). Sample size planning for classification models. Analytica Chimica Acta, 760, 25-33.
  10. Belvederi Murri, M., Caruso, R., Ounalli, H., Zerbinati, L., Berretti, E., Costa, S., … Grassi, L. (2020). The relationship between demoralization and depressive symptoms among patients from the general hospital: network and exploratory graph analysis: Demoralization and depression symptom network. Journal of Affective Disorders, 276(June), 137–146. https://doi.org/10.1016/j.jad.2020.06.074
  11. Berens, J., Schneider, K., Gortz, S., Oster, S., & Burghoff, J. (2019). Early detection of students at risk - predicting student dropouts using administrative student data from German universities and machine learning methods. Journal of Educational Machine learning, 11(3), 1-