KaydedilenlerDerginizi ekleyin

Dergiler / International Journal of Assessment Tools in Education / 2021 / Cilt: 8 - Sayı: 1

The Effect of Item Pools of Different Strengths on the Test Results of Computerized-Adaptive Testing

Fatih Kezer
Sayfa
145–155
Yayın Tarihi
2022-10-18
DOI
—
🏛️Bu makalenin bibliyografik verileri ve tam metin bağlantısı DergiPark üzerinden sağlanmıştır.
DergiPark Kaydı ↗
✨

Yapay Zeka Hızlı Bakış & Çıkarımlar

AI Insight
⚡ TL;DR(1 Cümlede Hızlı Çıkarım)

It was difficult to make accurate estimations for individuals with high ability levels especially in test applications conducted with an item pool that consists of easy items, and for individuals with low ability levels in test applications conducted with an item pool consisting of difficult items.

📌 3 Maddede Bu Makale

🎯
Amaç & Hipotez:Item response theory provides various important advantages for exams carried out or to be carried out digitally.
📊
Temel Bulgular & Veri:To this end, eight different 2000-people item pools were designed in simulation which consist of 500 items with ability scores and varying difficulty levels.
💡
Sonuç & Katkı:It was difficult to make accurate estimations for individuals with high ability levels especially in test applications conducted with an item pool that consists of easy items, and for individuals with low ability levels in test applications conducted with an item pool consisting of difficult items.
🔬Tür:Laboratuvar & Saha Deneyi
📍Kapsam:2000 Verileri
⚙️Değişkenler:Adaptive test · Item pool · Item difficulty

Özet

Item response theory provides various important advantages for exams carried out or to be carried out digitally. For computerized adaptive tests to be able to make valid and reliable predictions supported by IRT, good quality item pools should be used. This study examines how adaptive test applications vary in item pools which consist of items with varying difficulty levels. Within the scope of the study, the impact of items was examined where the parameter b differentiates while the parameters a and c are kept in fixed range. To this end, eight different 2000-people item pools were designed in simulation which consist of 500 items with ability scores and varying difficulty levels. As a result of CAT simulations, RMSD, BIAS and test lengths were examined. At the end of the study, it was found that tests run by item pools with parameter b in the range that matches the ability level end up with fewer items and have a more accurate stimation. When parameter b takes value in a narrower range, estimation of ability for extreme ability values that are not consistent with parameter b required more items. It was difficult to make accurate estimations for individuals with high ability levels especially in test applications conducted with an item pool that consists of easy items, and for individuals with low ability levels in test applications conducted with an item pool consisting of difficult items.

Abstract

Item response theory provides various important advantages for exams carried out or to be carried out digitally. For computerized adaptive tests to be able to make valid and reliable predictions supported by IRT, good quality item pools should be used. This study examines how adaptive test applications vary in item pools which consist of items with varying difficulty levels. Within the scope of the study, the impact of items was examined where the parameter b differentiates while the parameters a and c are kept in fixed range. To this end, eight different 2000-people item pools were designed in simulation which consist of 500 items with ability scores and varying difficulty levels. As a result of CAT simulations, RMSD, BIAS and test lengths were examined. At the end of the study, it was found that tests run by item pools with parameter b in the range that matches the ability level end up with fewer items and have a more accurate stimation. When parameter b takes value in a narrower range, estimation of ability for extreme ability values that are not consistent with parameter b required more items. It was difficult to make accurate estimations for individuals with high ability levels especially in test applications conducted with an item pool that consists of easy items, and for individuals with low ability levels in test applications conducted with an item pool consisting of difficult items.

🔖

Bu Makaleye Atıf Yap

Kezer, F. (2022). The Effect of Item Pools of Different Strengths on the Test Results of Computerized-Adaptive Testing. International Journal of Assessment Tools in Education, Cilt: 8 - Sayı: 1, 145–155. https://ajindex.com/articles/540083

📄 Tam Metin PDF Okuyucu

📄

PDF görüntüleyici hazırlanıyor...

Kaynakça (11)

  1. Adams, R. (2005). Reliabilty as a measurement design effect. Studies in Educational Evaluation, 31, 162–172.
  2. Baker, F. B., & Kim, S. H. (2004). Item response theory: Parameter estimation techniques. Marcel Bekker Inc. Boyd, A. M., Dodd, B. & Fitzpatrick, S. (2013). A comparison of exposure control procedures in cat systems based on different measurement models for testlets. Applied Measurement in Education, 26(2), 113-115.
  3. Brown, J. M., & Weiss, D. J. (1977). An adaptive testing strategy for achievement test batteries (Research Rep. No. 77-6).
  4. University of Minnesota, Department of Psychology, Psychometric Methods Program. Bulut, O., & Kan, A. (2012) Application of computerized adaptive testing to entrance examination for graduate studies in Turkey. Egitim Arastirmalari-Eurasian Journal of Educational Research, 49, 61–80.
  5. Chang, H. H. (2014). Psychometrics behind computerized adaptive testing. Psychometrika, 1-20.
  6. Cömert, M. (2008). Bireye uyarlanmış bilgisayar destekli ölçme ve değerlendirme yazılımı geliştirilmesi [Computer-aided assessment and evaluation analysis adapted to the individual] [Unpublished master’s thesis]. Bahçeşehir University. Crocker, L., & Algina, J. (1986). Introduction classical and modern test theory. Harcourt Brace Javonovich College Publishers.
  7. Dodd, B. G., Koch, W. R., & Ayala, R. J. (1993). Computerized adaptive testing using the partial credit model: effects of item pool characteristics and different stopping rules. Educational and Psychological Measurement, 53(1), 61-77.
  8. Eggen, T. J. H. M., & Verschoor, A. J. (2006). Optimal testing with easy or difficult items in computerized adaptive testing. Applied Psychologgical Measurement, 30(5), 379-393.
  9. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates. Georgiadou, E., Triantafillou, E., & Economides, A. A. (2006). Evaluation parameters for computer adaptive testing. British Journal of Educational Techonology, 37(2), 261–278.
  10. Glas, C. A., & Linden, W. (2003). Computerized adaptive testing with item cloning. Applied Psychological Measurement, 27(4), 247–261.
  11. Hambleton, R. K., & Swaminathan, H. (1989). Item response teory: Principles and applications. Kluwer Nijhoff Publishing. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Sage Publications Inc. Iseri, A. I. (2002). Assessment of students' mathematics achievement through computer adaptive testing procedures [Unpublished doctoral