Dergiler / International Journal of Assessment Tools in Education / 2026 / Volume: 12 Issue: 2

Exploring number of response categories in factor analysis: Implications for sample size

Fatih Orçan(KAHRAMANMARAŞ SÜTÇÜ İMAM ÜNİVERSİTESİ)0000-0003-1727-0456
Sayfa
—
Yayın Tarihi
2025-06-01
🏛️Bu makalenin bibliyografik verileri ve tam metin bağlantısı DergiPark üzerinden sağlanmıştır.
DergiPark Kaydı ↗
✨

Yapay Zeka Hızlı Bakış & Çıkarımlar

AI Insight
⚡ TL;DR(1 Cümlede Hızlı Çıkarım)

The findings suggest that for optimal results, researchers should carefully balance sample size, number of items, and response categories, particularly in studies with categorical data.

📌 3 Maddede Bu Makale

🎯
Amaç & Hipotez:Factor analysis is a statistical method to explore the relationships among observed variables and identify latent structures.
📊
Temel Bulgular & Veri:While some studies suggest that reliability improves with more response categories, others find no significant relationship between the number of response categories and reliability.
💡
Sonuç & Katkı:The findings suggest that for optimal results, researchers should carefully balance sample size, number of items, and response categories, particularly in studies with categorical data.
🔬Tür:Modelleme & Sayısal Simülasyon
📍Kapsam:Genişletilmiş Akademik Örneklem
⚙️Değişkenler:Factor analysis · Sample size · Number of response categories

Özet

Factor analysis is a statistical method to explore the relationships among observed variables and identify latent structures. It is crucial in scale development and validity analysis. Key factors affecting the accuracy of factor analysis results include the type of data, sample size, and the number of response categories. While some studies suggest that reliability improves with more response categories, others find no significant relationship between the number of response categories and reliability. A key consideration is that increasing the number of response categories can introduce measurement errors, especially when there are too many categories for participants to respond accurately. The study examines how different numbers of response categories affect sample size requirements in factor analysis, particularly under misspecified and correctly specified models. MonteCarloSEM package in R was used to simulate data sets based on sample size, number of response categories, model specification, and test length. Results show that a higher number of categories helps reduce bias and improve model fit, especially in smaller samples. However, when sample sizes are small or when fewer categories are used, increasing the number of items or the number of categories can improve parameter estimation. The findings suggest that for optimal results, researchers should carefully balance sample size, number of items, and response categories, particularly in studies with categorical data.

Abstract

Factor analysis is a statistical method to explore the relationships among observed variables and identify latent structures. It is crucial in scale development and validity analysis. Key factors affecting the accuracy of factor analysis results include the type of data, sample size, and the number of response categories. While some studies suggest that reliability improves with more response categories, others find no significant relationship between the number of response categories and reliability. A key consideration is that increasing the number of response categories can introduce measurement errors, especially when there are too many categories for participants to respond accurately. The study examines how different numbers of response categories affect sample size requirements in factor analysis, particularly under misspecified and correctly specified models. MonteCarloSEM package in R was used to simulate data sets based on sample size, number of response categories, model specification, and test length. Results show that a higher number of categories helps reduce bias and improve model fit, especially in smaller samples. However, when sample sizes are small or when fewer categories are used, increasing the number of items or the number of categories can improve parameter estimation. The findings suggest that for optimal results, researchers should carefully balance sample size, number of items, and response categories, particularly in studies with categorical data.

🔖

Bu Makaleye Atıf Yap

Orçan, F. (2025). Exploring number of response categories in factor analysis: Implications for sample size. International Journal of Assessment Tools in Education, Volume: 12 Issue: 2. https://doi.org/10.21449/ijate.1581217

📄 Tam Metin PDF Okuyucu

📄

PDF görüntüleyici hazırlanıyor...

Kaynakça (23)

  1. Abulela, M.A.A., & Khalaf, M.A. (2024). Does the number of response categories impact validity evidence in self report measures? A scoping review. Sage Open, 14(1), 1 16. https://doi.org/10.1177/21582440241230363
  2. Abdelsamea, M. (2020). The effect of the number of response categories on the assumptions and outputs of item exploratory and confirmatory factor analyses of measurement instruments in psychological research. Journal of Education Sohag UNV, 76, 1153-1222. https://doi.org/10.21608/edusohag.2020.103373
  3. Bandalos, D.L., & Enders, C.K. (1996). The effects of nonnormality and number of response categories on reliability. Applied Measurement in Education, 9(2), 151 160. https://doi.org/10.1207/s15324818ame0902_4
  4. Flora, D.B., & Curran, P.J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9(4), 466-491. https://doi.org/10.1037/1082-989X.9.4.466
  5. Hu, L., & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
  6. Kline, R.B. (2011). Principles and Practice of Structural Equation Modeling. Guilford Press.
  7. Komorita, S.S., & Graham, W.K. (1965). Number of scale points and the reliability of scales. Educational and Psychological Measurement, 25(4), 987 995. https://doi.org/10.1177/001316446502500404
  8. Kılıç, A.F. (2022). The effect of categories and distribution of variables on correlation coefficients. Ege Eğitim Dergisi, 23(1), 50-80. https://doi.org/10.12984/egeefd.890104
  9. Kyriazos, T.A. (2018). Applied psychometrics: Sample size and sample power considerations in factor analysis (EFA, CFA) and SEM in general. Psychology, 9(8), 2207 2230. https://doi.org/10.4236/psych.2018.98126
  10. Lozano, L., García-Cueto, E., & Muñiz, J. (2008). Effect of the number of response categories on the reliability and validity of rating scales. Methodology 4(2). 73 79. https://doi.org/10.1027/1614-2241.4.2.73
  11. MacCallum, R.C., Widaman, K.F., Zhang, S., & Hong, S. (1999). Sample size in factor analysis. Psychological Methods, 4(1), 84–99. https://doi.org/10.1037/1082-989X.4.1.84
  12. Matell, M.S., & Jacoby, J. (1971). Is there an optimal number of alternatives for Likert scale items? I. Reliability and validity. Educational and Psychological Measurement, 31, 657-674. https://doi.org/10.1177/001316447103100307
  13. Maydeu-Olivares, A., Fairchild, A.J., & Hall, A.G. (2017). Goodness of fit in item factor analysis: Effect of the number of response alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 24(4), 495 505. https://doi.org/10.1080/10705511.2017.1289816
  14. Orçan, F. (2021). MonteCarloSEM: An R package to simulate data for SEM. International Journal of Assessment Tools in Education, 8(3), 704 713. https://doi.org/10.21449/ijate.804203
  15. Orçan, F., & Yanyun, Y. (2016). A note on the use of item parceling in structural equation modeling with missing data. Journal of Measurement and Evaluation in Education and Psychology, 7(1), 59-72. https://doi.org/10.21031/epod.88204
  16. Preston, C.C., & Colman, A.M. (2000). Optimal number of response categories in rating scales: Reliability, validity, discriminating power, and respondent preferences. Acta Psychologica, 104(1), 1–15. https://doi.org/10.1016/S0001-6918(99)00050-5
  17. R Core Team. (2020). R: A language and environment for statistical computing [Computer software]. R Foundation for Statistical Computing. https://www.R-project.org/
  18. Rhemtulla, M., Brosseau-Liard, P.É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354 373. https://doi.org/10.1037/a0029315
  19. Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02
  20. Shi, D., Siceloff, E.R., Castellanos, R.E., Bridges, R.M., Jiang, Z., Flory, K., & Benson, K. (2021). Revisiting the effect of varying the number of response alternatives in clinical assessment: Evidence from measuring ADHD symptoms. Assessment, 28(5), 1287-1300. https://doi.org/10.1177/1073191120952885
  21. Simms, L.J., Zelazny, K., Williams, T.F., & Bernstein, L. (2019). Does the number of response options matter? Psychometric perspectives using personality questionnaire data. Psychological Assessment, 31(4), 557-566. http://dx.doi.org/10.1037/pas0000648
  22. Yoon, G. (2024) No one optimal way to measure people’s attitudes? Preferred length of scales in advertising research. Journal of Current Issues & Research in Advertising, 45(1), 43-70, https://doi.org/10.1080/10641734.2023.2246049
  23. Wakita, T., Ueshima, N., & Noguchi, H. (2012). Psychological distance between categories in the Likert scale: Comparing different numbers of options. Educational and Psychological Measurement, 72(4), 533-546. https://doi.org/10.1177/0013164411431162