AI Summary & Key Takeaways
AI InsightResults indicated severe DIF flagging prevalence differences for items across DIF detection techniquesemploying Wald test, Raju’s area measures, and Mantel-Haenzsel statistics.
📌 3 Key Takeaways
Abstract
Richer diagnostic information about examinees’ cognitive strength and weaknesses are obtained from cognitivelydiagnostic assessments (CDA) when a proper cognitive diagnosis model (CDM) is used for response data analysis.To do so, researchers state that a preset cognitive model specifying the underlying hypotheses about response datastructure is needed. However, many real data CDM applications are adds-on to simulation studies and retrofittedto data obtained from non-CDAs. Such a procedure is referred to as retrofitting, and fitting CDMs to traditionaltest data is not uncommon. To deal with a major validity concern of item/test bias in CDAs, some recent DIFdetection techniques compatible with various CDMs have been proposed. This study employs several DIFdetection techniques developed based on CTT, IRT, and CDM frameworks and compares the results to understandthe extent to which DIF flagging behavior of items is affected by retrofitting. A secondary purpose of this study isto gather evidence about test booklet effects (i.e., item ordering) on items’ psychometric properties through DIFanalyses. Results indicated severe DIF flagging prevalence differences for items across DIF detection techniquesemploying Wald test, Raju’s area measures, and Mantel-Haenzsel statistics. The largest numbers of DIF caseswere observed when the data were retrofitted to a CDM. The results further revealed that an item might be flaggedas DIF in one booklet, whereas it might not be flagged in another