Journals / Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi / 2016 / Cilt: 7 - Sayı: 2

Computer Adaptive Multistage Testing: Practical Issues, Challenges and Principles

Pages
388–406
Publication Date
2016-12-25
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

The books, software and future work for ca-MST are also discussed.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:The purpose of many test in the educational and psychological measurement is to measure test takers’ latent trait scores from responses given to a set of items.
📊
Key Findings & Data:Over the years, this has been done by traditional methods (paper and pencil tests).
💡
Conclusion & Impact:The books, software and future work for ca-MST are also discussed.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:Domain Parameters

Abstract

The purpose of many test in the educational and psychological measurement is to measure test takers’ latent trait scores from responses given to a set of items. Over the years, this has been done by traditional methods (paper and pencil tests). However, compared to other test administration models (e.g., adaptive testing), traditional methods are extensively criticized in terms of producing low measurement accuracy and long test length. Adaptive testing has been proposed to overcome these problems. There are two popular adaptive testing approaches. These are computerized adaptive testing (CAT) and computer adaptive multistage testing (ca-MST). The former is a well-known approach that has been predominantly used in this field. We believe that researchers and practitioners are fairly familiar with many aspects of CAT because it has more than a hundred years of history. However, the same thing is not true for the latter one. Since ca-MST is relatively new, many researchers are not familiar with features of it. The purpose of this study is to closely examine the characteristics of ca-MST, including its working principle, the adaptation procedure called the routing method, test assembly, and scoring, and provide an overview to researchers, with the aim of drawing researchers’ attention to ca-MST and encouraging them to contribute to the research in this area. The books, software and future work for ca-MST are also discussed.

🔖

Cite This Article

SARI, H. İ., YAHSI-SARI, H., HUGGINS-MANLEY, A. C. (2016). Computer Adaptive Multistage Testing: Practical Issues, Challenges and Principles. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, Cilt: 7 - Sayı: 2, 388–406. https://ajindex.com/articles/137339

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (10)

  1. Angoff, W. H., & Huddleston, E. M. (1958). The multi-level experiment: a study of a two-level test system for the College Board Scholastic Aptitude Test (SR-58-21).
  2. Princeton, New Jersey: Educational Testing Service. Armstrong, R. D. & Roussos, L. (2005). A method to determine targets for multi-stage adaptive tests. (Research Report 02-07).
  3. Newtown, PA: Law School Admissions Council. Armstrong, R. D., Jones, D. H., Koppel, N. B., & Pashley, P. J. (2004). Computerized adaptive testing with multiple-form structures. Applied Psychological Measurement, 28, 147- 164. Barrada, J. R., Olea, J., Ponsoda, V., & Abad, F. J. (2008). Incorporating randomness to the Fisher information for improving item exposure control in CATS. British Journal of Mathematical and Statistical Psychology, 61, 493-513.
  4. Becker, K. A., & Bergstrom, B. A. (2013). Test administration models. Practical Assessment, Research & Evaluation, 18(14), 7. Belov, D. I., & Armstrong, R. D. (2005). Monte Carlo test assembly for item pool analysis and extension. Applied Psychological Measurement, 29, 239-261.
  5. Berkelaar, M. (2015). Package ‘lpSolve’. Bridgeman, B. (2012). A Simple Answer to a Simple Question on Changing Answers. Journal of Educational Measurement, 49, 467-468.
  6. Chang, H.H., & Ying, Z. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213–229.
  7. Choi, S. W., Grady, M. W., & Dodd, B. G. (2010). A New Stopping Rule for Computerized Adaptive Testing. Educational and Psychological Measurement, 70(6), 1–17.
  8. Cor, K., Alves, C., & Gierl, M. (2009). Three Applications of Automated Test Assembly within a User-Friendly Modeling Environment. Practical Assessment, Research & Evaluation, 14(14). Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart & Winston. Cronbach, L. J., & Glaser, G. C. (1965). Psychological tests and personnel decisions. Urbana, IL: University of Illinois Press.
  9. Crotts, K. M., Zenisky, A. L., & Sireci, S. G. (2012, April). Estimating measurement precision in reduced-length multistage-adaptive testing. Paper presented at the meeting of the National Council on Measurement in Education, Vancouver, BC, Canada. Davey, T., & Y.H. Lee. (2011). Potential impact of context effects on the scoring and equating of the multistage GRE Revised General Test. (GRE Board Research Report 08-01).
  10. Princeton, NJ: Educational Testing Service. Davis, L. L., & Dodd, B. G. (2003). Item Exposure