KaydedilenlerDerginizi ekleyin

Dergiler / Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi / 2022 / Cilt: 13 - Sayı: 1

Comparison of Item Response Theory Scaling Methods withROC Analysis

Cem Oktay GÜZELLER
Meltem YURÇU
Sayfa
15–22
Yayın Tarihi
2022-10-12
DOI
—
🏛️Bu makalenin bibliyografik verileri ve tam metin bağlantısı DergiPark üzerinden sağlanmıştır.
DergiPark Kaydı ↗
✨

Yapay Zeka Hızlı Bakış & Çıkarımlar

AI Insight
⚡ TL;DR(1 Cümlede Hızlı Çıkarım)

It is concluded that ROC analysis can be used to compare different conditions, methods and models.

📌 3 Maddede Bu Makale

🎯
Amaç & Hipotez:In this study, one-dimensional item response theory models were evaluated using different scaling methods.
📊
Temel Bulgular & Veri:In this context, the equating errors and the area under the curve of four scaling methods (Stocking-Lord, Heabara, Mean-Sigma, Mean-Mean), and one, two, and three parameters logistic models (1PL, 2PL, and 3PL) in non-equivalent groups with anchor test (NEAT) design were examined.
💡
Sonuç & Katkı:It is concluded that ROC analysis can be used to compare different conditions, methods and models.
🔬Tür:Modelleme & Sayısal Simülasyon
📍Kapsam:2012 Verileri
⚙️Değişkenler:Alan Parametreleri

Abstract

In this study, one-dimensional item response theory models were evaluated using different scaling methods. In this context, the equating errors and the area under the curve of four scaling methods (Stocking-Lord, Heabara, Mean-Sigma, Mean-Mean), and one, two, and three parameters logistic models (1PL, 2PL, and 3PL) in non-equivalent groups with anchor test (NEAT) design were examined. Additionally, the equating errors of the scaling methods and the results obtained from ROC analysis were compared. Qatar's and Australia's PISA 2012 mathematical literacy test data were used in the study. The minimum error was obtained from the Mean-Mean method with the 1PL model, and the maximum error was obtained from the Mean-Mean method with the 3PL model. Similar results were observed in all comparisons and supported each other. It is concluded that ROC analysis can be used to compare different conditions, methods and models.

🔖

Bu Makaleye Atıf Yap

GÜZELLER, C. O., YURÇU, M. (2022). Comparison of Item Response Theory Scaling Methods withROC Analysis. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, Cilt: 13 - Sayı: 1, 15–22. https://ajindex.com/articles/497137

📄 Tam Metin PDF Okuyucu

📄

PDF görüntüleyici hazırlanıyor...

Kaynakça (11)

  1. Aşiret, S., & Sünbül, S. Ö. (2016). Investigating test equating methods in small samples through various factors. Educational Sciences: Theory & Practice, 16(2), 647-668. https://doi.org/10.12738/estp.2016.2.2762
  2. Baker, F. B., & Al-Karni, A. (1991). A comparison of two procedures for computing IRT equating coefficients. Journal of Educational Measurement, 28, 147-162. https://www.jstor.org/stable/1434796
  3. Boduroğlu, E. (2017). The study of classification consistency of transition to higher education examination according to the cut-off scores obtained from different [Master’s Thesis, Mersin University]. https://tez.yok.gov.tr/UlusalTezMerkezi/
  4. Branberg, K., & Wiberg, M. (2011). Observed score linear equating with covariates. Journal of Educational Measurement, 48(4), 419-440. https://www.jstor.org/stable/41427533
  5. Carrington, A. M., Manuel, D. G., Fieguth, P. W., Ramsay, T., Osmani, V., Wernly, B.,. & Holzinger, A. (2021). Deep ROC analysis and AUC as balanced average accuracy to improve model selection, understanding and interpretation. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/TPAMI.2022.3145392
  6. Çokluk, Ö., Şekercioğlu, G., & Büyüköztürk, Ş. (2012). Sosyal bilimler için çok değişkenli istatistik: SPSS ve Lisrel uygulamaları. Pegem Akademi. Embretson, S. (1996) The new rules of measurement. Psychological Assessment, 8(4), 341-349. https://doi.org/10.1037/1040-3590.8.4.341
  7. Embretson, S. (1997). Multicomponent response models. In W. van der Linden & R. Hambleton (Eds.), Handbook of modern Item Response Theory (pp. 305-321).
  8. Springer-Verlag. Embretson, S. E., & Reise, S. P. (2000). Item Response Theory for psychologists. Lawrence Elbaum Associates. Faraggi, D., & Reiser, B. (2002). Estimation of the area under the ROC curve. Statistics in Medicine, 21, 3093-3106. https://doi.org/10.1002/sim.1228
  9. Flach, P., Blockeel, H., Ferri, C., Hernandez-Orallo, J., & Struyf, J. (2003). Decision support for data mining: An introduction to ROC analysis and its applications. In D. Mladenić, N. Lavrač, M. Bohanec & S. Moyle (Eds.), Data mining and decision support: Integration and collaboration (vol. 745, pp. 81-90). Springer. https://doi.org/10.1007/978-1-4615-0286-9_7
  10. Flach, P. (2019). Performance evaluation in machine learning: the good, the bad, the ugly, and the way forward. Proceedings of the AAAI Conference on Artificial Intelligence, 33, 9808–9814. https://doi.org/10.1609/aaai.v33i01.33019808
  11. Gao, X., Zhu, R., Chen, H.,