SavedAdd your journal

Journals / Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi / 2022 / Cilt: 13 - Sayı: 1

Comparison of Item Response Theory Scaling Methods withROC Analysis

Cem Oktay GÜZELLER
Meltem YURÇU
Pages
15–22
Publication Date
2022-10-12
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

It is concluded that ROC analysis can be used to compare different conditions, methods and models.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:In this study, one-dimensional item response theory models were evaluated using different scaling methods.
📊
Key Findings & Data:In this context, the equating errors and the area under the curve of four scaling methods (Stocking-Lord, Heabara, Mean-Sigma, Mean-Mean), and one, two, and three parameters logistic models (1PL, 2PL, and 3PL) in non-equivalent groups with anchor test (NEAT) design were examined.
💡
Conclusion & Impact:It is concluded that ROC analysis can be used to compare different conditions, methods and models.
🔬Design:Modeling & Simulation
📍Scope:2012 Verileri
⚙️Variables:Domain Parameters

Abstract

In this study, one-dimensional item response theory models were evaluated using different scaling methods. In this context, the equating errors and the area under the curve of four scaling methods (Stocking-Lord, Heabara, Mean-Sigma, Mean-Mean), and one, two, and three parameters logistic models (1PL, 2PL, and 3PL) in non-equivalent groups with anchor test (NEAT) design were examined. Additionally, the equating errors of the scaling methods and the results obtained from ROC analysis were compared. Qatar's and Australia's PISA 2012 mathematical literacy test data were used in the study. The minimum error was obtained from the Mean-Mean method with the 1PL model, and the maximum error was obtained from the Mean-Mean method with the 3PL model. Similar results were observed in all comparisons and supported each other. It is concluded that ROC analysis can be used to compare different conditions, methods and models.

🔖

Cite This Article

GÜZELLER, C. O., YURÇU, M. (2022). Comparison of Item Response Theory Scaling Methods withROC Analysis. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, Cilt: 13 - Sayı: 1, 15–22. https://ajindex.com/articles/497137

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (11)

  1. Aşiret, S., & Sünbül, S. Ö. (2016). Investigating test equating methods in small samples through various factors. Educational Sciences: Theory & Practice, 16(2), 647-668. https://doi.org/10.12738/estp.2016.2.2762
  2. Baker, F. B., & Al-Karni, A. (1991). A comparison of two procedures for computing IRT equating coefficients. Journal of Educational Measurement, 28, 147-162. https://www.jstor.org/stable/1434796
  3. Boduroğlu, E. (2017). The study of classification consistency of transition to higher education examination according to the cut-off scores obtained from different [Master’s Thesis, Mersin University]. https://tez.yok.gov.tr/UlusalTezMerkezi/
  4. Branberg, K., & Wiberg, M. (2011). Observed score linear equating with covariates. Journal of Educational Measurement, 48(4), 419-440. https://www.jstor.org/stable/41427533
  5. Carrington, A. M., Manuel, D. G., Fieguth, P. W., Ramsay, T., Osmani, V., Wernly, B.,. & Holzinger, A. (2021). Deep ROC analysis and AUC as balanced average accuracy to improve model selection, understanding and interpretation. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/TPAMI.2022.3145392
  6. Çokluk, Ö., Şekercioğlu, G., & Büyüköztürk, Ş. (2012). Sosyal bilimler için çok değişkenli istatistik: SPSS ve Lisrel uygulamaları. Pegem Akademi. Embretson, S. (1996) The new rules of measurement. Psychological Assessment, 8(4), 341-349. https://doi.org/10.1037/1040-3590.8.4.341
  7. Embretson, S. (1997). Multicomponent response models. In W. van der Linden & R. Hambleton (Eds.), Handbook of modern Item Response Theory (pp. 305-321).
  8. Springer-Verlag. Embretson, S. E., & Reise, S. P. (2000). Item Response Theory for psychologists. Lawrence Elbaum Associates. Faraggi, D., & Reiser, B. (2002). Estimation of the area under the ROC curve. Statistics in Medicine, 21, 3093-3106. https://doi.org/10.1002/sim.1228
  9. Flach, P., Blockeel, H., Ferri, C., Hernandez-Orallo, J., & Struyf, J. (2003). Decision support for data mining: An introduction to ROC analysis and its applications. In D. Mladenić, N. Lavrač, M. Bohanec & S. Moyle (Eds.), Data mining and decision support: Integration and collaboration (vol. 745, pp. 81-90). Springer. https://doi.org/10.1007/978-1-4615-0286-9_7
  10. Flach, P. (2019). Performance evaluation in machine learning: the good, the bad, the ugly, and the way forward. Proceedings of the AAAI Conference on Artificial Intelligence, 33, 9808–9814. https://doi.org/10.1609/aaai.v33i01.33019808
  11. Gao, X., Zhu, R., Chen, H.,