SavedAdd your journal

Journals / International Journal of Assessment Tools in Education / 2021 / Cilt: 8 - Sayı: 2

Detecting Differential Item Functioning: Item Response Theory Methods Versus the Mantel-Haenszel Procedure

Emily DIAZ
Gordon BROOKS
George JOHANSON
Pages
376–393
Publication Date
2022-10-18
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

The findings suggested the MH procedure without the continuity correction is best based on Type I error rate.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:This Monte Carlo study assessed Type I error in differential item functioning analyses using Lord’s chi-square (LC), Likelihood Ratio Test (LRT), and Mantel-Haenszel (MH) procedure.
📊
Key Findings & Data:There were three fixed factors (number of test items, IRT parameter estimation method, and item parameter equating) and four varied factors (IRT model used to generate data and fit the data, sample size, and impact).
💡
Conclusion & Impact:The findings suggested the MH procedure without the continuity correction is best based on Type I error rate.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:Differential item · functioning · Monte Carlo simulation

Abstract

This Monte Carlo study assessed Type I error in differential item functioning analyses using Lord’s chi-square (LC), Likelihood Ratio Test (LRT), and Mantel-Haenszel (MH) procedure. Two research interests were investigated: item response theory (IRT) model specification in LC and the LRT and continuity correction in the MH procedure. This study enhances the literature by investigating LC and the LRT using correct and incorrect model-data fit and comparing those results to the MH procedure. There were three fixed factors (number of test items, IRT parameter estimation method, and item parameter equating) and four varied factors (IRT model used to generate data and fit the data, sample size, and impact). The findings suggested the MH procedure without the continuity correction is best based on Type I error rate.

Özet

This Monte Carlo study assessed Type I error in differential item functioning analyses using Lord’s chi-square (LC), Likelihood Ratio Test (LRT), and Mantel-Haenszel (MH) procedure. Two research interests were investigated: item response theory (IRT) model specification in LC and the LRT and continuity correction in the MH procedure. This study enhances the literature by investigating LC and the LRT using correct and incorrect model-data fit and comparing those results to the MH procedure. There were three fixed factors (number of test items, IRT parameter estimation method, and item parameter equating) and four varied factors (IRT model used to generate data and fit the data, sample size, and impact). The findings suggested the MH procedure without the continuity correction is best based on Type I error rate.

🔖

Cite This Article

DIAZ, E., BROOKS, G., JOHANSON, G. (2022). Detecting Differential Item Functioning: Item Response Theory Methods Versus the Mantel-Haenszel Procedure. International Journal of Assessment Tools in Education, Cilt: 8 - Sayı: 2, 376–393. https://ajindex.com/articles/540089

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (10)

  1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. American Educational Research Association. Bolt, D. M. (2002). A Monte Carlo comparison of parametric and nonparametric polytomous DIF detection methods. Applied Measurement in Education, 15(2), 113-141. https://doi.org/10.1207/S15324818AME1502_01
  2. Bolt, D. M., Deng, S., & Lee, S. (2014). IRT model misspecification and measurement of growth in vertical scaling. Journal of Educational Measurement, 51(2), 141-162. https://doi.org/10.1111/jedm.12039
  3. Bradley, J. V. (1978). Robustness? British Journal of Mathematical and Statistical Psychology, 31(2), 144-152. https://doi.org/10.1207/S15324818AME1502_01
  4. Camilli, G. (2006). Test fairness. In R. L. Brennan (Ed.), Educational Measurement (4th ed., pp. 220-256). American Council on Education. Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items. Sage. Candell, G. L., & Drasgow, F. (1988). An iterative procedure for linking metrics and assessing item bias in item response theory. Applied Psychological Measurement, 12(3), 253-260. https://doi.org/10.1177/014662168801200304
  5. Cohen, A. S., & Kim, SH. (1993). A comparison of Lord’s χ2 and Raju’s area measures in detection of DIF. Applied Psychological Measurement, 17(1), 39 52. https://doi.org/10.1177/014662169301700109
  6. Cohen, A. S., Kim, S. H., & Wollack, J. A. (1996). An investigation of the likelihood ratio test for detection of differential item functioning. Applied Psychological Measurement, 20(1), 15-26. https://doi.org/10.1177/014662169602000102
  7. Cook, L. L., & Eignor, D. R. (1991). IRT equating methods. Educational Measurement: Issues and Practice, 10(3), 37-45. https://doi.org/10.1111/j.1745-3992.1991.tb00207.x de Ayala, R. J. (2009). The theory and practice of item response theory. Guilford Press.
  8. Creswell, J. W. (2009). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage. DeMars, C. E. (2009). Modification of the Mantel-Haenszel and Logistic Regression DIF procedures to incorporate the SIBTEST regression correction. Journal of Educational and Behavioral Statistics, 34(2), 149-170. https://doi.org/10.3102/1076998607313923
  9. DeMars, C. E. (2010). Type I Error inflation for detecting DIF in the presence of impact. Educational and Psychological Measurement, 70(6), 961 972. https://doi.org/10.1177/0013164410366691
  10. Dorans, N. J., &