SavedAdd your journal

Journals / Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi / 2019 / Cilt: 10 - Sayı: 2

Performances Based on Ability Estimation of the Methods of Detecting Differential Item Functioning: A Simulation Study

İbrahim UYSAL
Levent ERTUNA
F. Güneş ERTAŞ
Hülya KELECİOĞLU
Pages
133–148
Publication Date
2019-06-28
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

Moreover, for the method of SIBTEST and IRT-LR in all conditions, it was found that the error obtained by purifying items with C level DIF is lower than the error obtained by purifying items with both B and C level DIF.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:The aim of the study is to examine differential item functioning (DIF) detection methods—the simultaneous item bias test (SIBTEST), Item Response Theory likelihood ratio (IRT-LR), Lord chi square (χ2), and Raju area measures—based on ability estimates when purifying items with DIF from the test, considering conditions of ratio of the items with DIF, effect size of DIF, and type of DIF.
📊
Key Findings & Data:This study is a simulation study and 50 replications were conducted for each condition.
💡
Conclusion & Impact:Moreover, for the method of SIBTEST and IRT-LR in all conditions, it was found that the error obtained by purifying items with C level DIF is lower than the error obtained by purifying items with both B and C level DIF.
🔬Design:Research Article
📍Scope:Comprehensive Sample
⚙️Variables:Differential item functioning · simulation · ratio of the items with DIF

Abstract

The aim of the study is to examine differential item functioning (DIF) detection methods—the simultaneous item bias test (SIBTEST), Item Response Theory likelihood ratio (IRT-LR), Lord chi square (χ2), and Raju area measures—based on ability estimates when purifying items with DIF from the test, considering conditions of ratio of the items with DIF, effect size of DIF, and type of DIF. This study is a simulation study and 50 replications were conducted for each condition. In order to compare DIF detection methods, error (RMSD) and coefficient of concordance (Pearson’s correlation coefficient) were calculated according to estimated and initial abilities for the reference group. As a result of the study, the lowest error and the highest concordance were seen in the case of 10% uniform DIF in the test and the method of IRT-LR, considering all other conditions. Moreover, for the method of SIBTEST and IRT-LR in all conditions, it was found that the error obtained by purifying items with C level DIF is lower than the error obtained by purifying items with both B and C level DIF. Similarly, for the method of SIBTEST and IRT-LR in all conditions, it was seen that the concordance coefficient found by purifying C level DIF is higher than the coefficient by purifying items with both B and C level DIF.

🔖

Cite This Article

UYSAL, İ., ERTUNA, L., ERTAŞ, F. G., KELECİOĞLU, H. (2019). Performances Based on Ability Estimation of the Methods of Detecting Differential Item Functioning: A Simulation Study. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, Cilt: 10 - Sayı: 2, 133–148. https://ajindex.com/articles/137311

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (2)

  1. Atalay Kabasakal, K., Arsan, N., Gök, B., & Kelecioğlu, H. (2014). Comparing performances (type I error and power) of IRT Likelihood Ratio SIBTEST and Mantel-Haenszel methods in the determination of differential item functioning. Educational Sciences: Theory & Practice, 14(6), 2175–2193. doi: 10.12738/estp.2014.6.2165 Camilli, G. (1993). The case against item bias detection techniques based on internal criteria: Do item bias procedures obscure test fairness issues? In P. W. Holland & H. Wainer (Eds.), Differential Item Functioning (pp. 397–418). New York: Routledge. Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items. California: Sage. Cheng, C. M. (2005). A study on Differential Item Functioning of the basic mathematical competence test for junior high schools in Taiwan (Doctoral Dissertation). Available from ProOuest Dissertations and Theses database (UMI No. 3189625). Clauser, B. E., & Mazor, K. M. (1998). Using statistical procedures to identify differentially functioning test items. Educational Measurement: Issues and Practice, 17(1), 31-47. doi: 10.1111/j.1745-3992.1998.tb00619.x Cohen, A. S., & Kim, S. (1993). A comparison of Lord’s Chi Square and Raju’s Area Measures in detection of DIF. Applied Psychological Measurement, 17(1), 39–52. doi: 10.1177/014662169301700109 Cromwell, S. (2002, February). A primer on ways to explore item bias. Paper presented at the Annual Meeting of the Southwest Educational Research Association, Austin, TX. Downing, S. M., & Haladyna, T. M. (2004). Validity threats: Overcoming interference with proposed interpretations of assessment data. Medical Education, 38(3), 327-333. https://doi.org/10.1046/j.1365-2923.2004.01777.x
  2. Finch, H. (2005). The MIMIC model as a method for detecting DIF: Comparison with Mantel-Haenszel, SIBTEST, and the IRT Likelihood Ratio. Applied Psychological Measurement, 29(4), 278-295. doi: 10.1177/0146621605275728 Finch, W. H., & French, B. F. (2007). Detection of crossing differential item functioning a comparison of four methods. Educational and Psychological Measurement, 67(4), 565-582. doi: 10.1177/0013164406296975 Glas, C. A. W., & Meijer, R. R. (2003). A Bayesian approach to person fit analysis in Item Response Theory models. Applied Psychological Measurement, 27(3), 217-233. doi: 10.1177/0146621603252216 Golia S. (2010). The assessment of DIF on Rasch measures with an application to job satisfaction. Electronic Journal of Applied Statistical Analysis: Decision Suppo