AI Summary & Key Takeaways
AI InsightThus, it was concluded that the two assessment theories (CTT focuses on measurement errors while the IRT focuses on individual traits) could let test designers obtain valid and reliable measurement scores, while the latter approach was observed to be slightly better at testing language achievement and/or proficiency.
📌 3 Key Takeaways
Abstract
Foreign language testing is a multi-dimensional phenomenon and obtaining objective and error-free scores on learners’ language skills is often problematic. While assessing foreign language performance on high-stakes tests, using different testing approaches including Classical Test Theory (CTT), Generalizability Theory (GT) and/or Item Response Theory (IRT) may help both to obtain results closer to true scores on students’ proficiency levels and to minimize the amount of measurement error on test results, depending on item parameters, testing objectives and the amount of time and resources for valid and reliable evaluation. In this study, two popular testing theories the CTT and IRT were compared in testing language proficiency. Multi-dimensionality of two multiple-choice language tests taken by 2032 low-int and intermediate level language students in the spring term of 2018-2019 academic year was examined via CTT and IRT. In the first step of the analyses, the dimensionality test results revealed that test results were multidimensional. As a result of the NOHARM test, carried out to analyze which IRT model the data fit finest, it was determined that the test data fit the 3- parameter-logistic model. Eventually, it was determined that the correlation coefficients between foreign language proficiency estimations based on the CTT and IRT varied between 0.806 and 0.891. Thus, it was concluded that the two assessment theories (CTT focuses on measurement errors while the IRT focuses on individual traits) could let test designers obtain valid and reliable measurement scores, while the latter approach was observed to be slightly better at testing language achievement and/or proficiency.