Journals / International Journal of Assessment Tools in Education / 2022 / Cilt: 9 - Sayı: 4

Investigation of the effect of parameter estimation and classification accuracy in mixture IRT models under different conditions

Pages
1013–1029
Publication Date
2022-12-22
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

Additionally, a factorial ANOVA was used to evaluate each factor's main effects and interaction effects.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:This study aims to examine the effects of mixture item response theory (IRT) models on item parameter estimation and classification accuracy under different conditions.
📊
Key Findings & Data:The manipulated variables of the simulation study are set as mixture IRT models (Rasch, 2PL, 3PL); sample size (600, 1000); the number of items (10, 30); the number of latent classes (2, 3); missing data type (complete, missing at random (MAR) and missing not at random (MNAR)), and the percentage of missing data (10%, 20%).
💡
Conclusion & Impact:Additionally, a factorial ANOVA was used to evaluate each factor's main effects and interaction effects.
🔬Design:Modeling & Simulation
📍Scope:1000 sample
⚙️Variables:Latent Class · Mixture Item Response Theory Models · Maximum Likelihood Estimation

Abstract

This study aims to examine the effects of mixture item response theory (IRT) models on item parameter estimation and classification accuracy under different conditions. The manipulated variables of the simulation study are set as mixture IRT models (Rasch, 2PL, 3PL); sample size (600, 1000); the number of items (10, 30); the number of latent classes (2, 3); missing data type (complete, missing at random (MAR) and missing not at random (MNAR)), and the percentage of missing data (10%, 20%). Data were generated for each of the three mixture IRT models using the code written in R program. MplusAutomation package, which provides the automation of R and Mplus program, was used to analyze the data. The mean RMSE values for item difficulty, item discrimination, and guessing parameter estimation were determined. The mean RMSE values as to the Mixture Rasch model were found to be lower than those of the Mixture 2PL and Mixture 3PL models. Percentages of classification accuracy were also computed. It was noted that the Mixture Rasch model with 30 items, 2 classes, 1000 sample size, and complete data conditions had the highest classification accuracy percentage. Additionally, a factorial ANOVA was used to evaluate each factor's main effects and interaction effects.

Özet

This study aims to examine the effects of mixture item response theory (IRT) models on item parameter estimation and classification accuracy under different conditions. The manipulated variables of the simulation study are set as mixture IRT models (Rasch, 2PL, 3PL); sample size (600, 1000); the number of items (10, 30); the number of latent classes (2, 3); missing data type (complete, missing at random (MAR) and missing not at random (MNAR)), and the percentage of missing data (10%, 20%). Data were generated for each of the three mixture IRT models using the code written in R program. MplusAutomation package, which provides the automation of R and Mplus program, was used to analyze the data. The mean RMSE values for item difficulty, item discrimination, and guessing parameter estimation were determined. The mean RMSE values as to the Mixture Rasch model were found to be lower than those of the Mixture 2PL and Mixture 3PL models. Percentages of classification accuracy were also computed. It was noted that the Mixture Rasch model with 30 items, 2 classes, 1000 sample size, and complete data conditions had the highest classification accuracy percentage. Additionally, a factorial ANOVA was used to evaluate each factor's main effects and interaction effects.

🔖

Cite This Article

SAATÇİOĞLU, F. M., ATAR, H. Y. (2022). Investigation of the effect of parameter estimation and classification accuracy in mixture IRT models under different conditions. International Journal of Assessment Tools in Education, Cilt: 9 - Sayı: 4, 1013–1029. https://ajindex.com/articles/866159

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (11)

  1. Alexeev, N., Templin, J., & Cohen, A.S. (2011). Spurious latent classes in the mixture Rasch model. Journal of Educational Measurement, 48, 313–332.
  2. Cohen, J. (1988). Statistical power analysis for the behavioural sciences (2nd ed.). Academic. Cohen, A.S., & Bolt, D.M. (2005). A mixture model analysis of differential item functioning. Journal of Educational Measurement, 42,133–148.
  3. Cho, S.-J., Cohen, A.S., & Kim, S.-H. (2013). Markov chain Monte Carlo estimation of a mixture item response theory model. Journal of Statistical Computation and Simulation, 83, 278–306. https://doi.org/10.1080/00949655.2011.603090
  4. Cho, H.J., Lee, J., & Kingston, N. (2012). Examining the effectiveness of test accommodation using DIF and a mixture IRT model. Applied Measurement in Education, 25(4), 281–304. https://doi.org/10.1080/08957347.2012.714682
  5. Cho, S.-J., Cohen, A.S., & Kim, S.-H. (2013). Markov chain Monte Carlo estimation of a mixture Rasch model. Journal of Statistical Computation and Simulation, 83, 278–306. https://doi.org/10.1080/00949655.2011.603090 Choi Y.J., & Cohen, A.S. (2020). Comparison of scale identification methods in Mixture IRT models. Journal of Modern Applied Statistical Methods, 18(1), eP2971. https://doi.org/10.22237/jmasm/1556669700
  6. Collins, L.M., & Lanza, S.T. (2010). Latent class and latent transition analysis. John Wiley & Sons. De Ayala, R.J., Plake, B.S. & Impara, J.C. (2001). The impact of omitted responses on the accuracy of ability estimation in item response theory. Journal of Educational Measurement,38, 213–234. https://doi.org/10.1111/j.1745-3984.2001.tb01124.x
  7. De Ayala, R.J. & Santiago, S.Y. (2017). An introduction to mixture item response theory models. Journal of School Psychology, 60, 25 40. https://doi.org/10.1016/j.jsp.2016.01.002
  8. Edwards, J.M., & Finch, W.H. (2018). Recursive partitioning methods for data imputation in the context of item response theory: A Monte Carlo simulation. Psicológica, 39(1), 88-117. https://doi.org/10.2478/psicolj-2018-0005
  9. Finch, H. (2008). Estimation of item response theory parameters in the presence of missing data. Journal of Educational Measurement, 45(3), 225 245. https://doi.org/10.1111/j.1745-3984.2008.00062.x
  10. Finch, W.H., & French, B.F. (2012). Parameter estimation with mixture item response theory models: A monte carlo comparison of maximum likelihood and bayesian methods. Journal of Modern Applied Statistical Methods, 11(1), 167-178.
  11. Hallquist, M.N., & Wiley, J.F. (2018). MplusAutomation: An R p