KaydedilenlerDerginizi ekleyin

Dergiler / International Journal of Assessment Tools in Education / 2020 / Volume: 7 Issue: 4

Measurement Invariance Testing with Alignment Method: Many Groups Comparison

Gözde Sırgancı(Yozgat Bozok University)0000-0003-4824-5413
Gizem Uyumaz(Giresun University)0000-0003-0792-2289
Alperen Yandı(BOLU ABANT İZZET BAYSAL ÜNİVERSİTESİ)0000-0002-1612-4249
Sayfa
—
Yayın Tarihi
2020-12-20
🏛️Bu makalenin bibliyografik verileri ve tam metin bağlantısı DergiPark üzerinden sağlanmıştır.
DergiPark Kaydı ↗
✨

Yapay Zeka Hızlı Bakış & Çıkarımlar

AI Insight
⚡ TL;DR(1 Cümlede Hızlı Çıkarım)

Besides it provides information which items have the most invariants for groups that hold MI.

📌 3 Maddede Bu Makale

🎯
Amaç & Hipotez:It is necessary to examine the measurement invariance (MI) among groups in studies where different groups are compared by using a measurement instrument.
📊
Temel Bulgular & Veri:In this study, the aim is to explain the basic concepts and processes of the alignment method which is offered as a new method for testing MI and illustrate an application on the real data set.
💡
Sonuç & Katkı:Besides it provides information which items have the most invariants for groups that hold MI.
🔬Tür:Modelleme & Sayısal Simülasyon
📍Kapsam:2015 Verileri
⚙️Değişkenler:multi-group confirmatory factor analysis · validity · modification indexes

Özet

It is necessary to examine the measurement invariance (MI) among groups in studies where different groups are compared by using a measurement instrument. Most of the studies, measurement invariance is tested with multiple group confirmatory factor analysis. This model applies many model adjustments based on the modification indexes. Therefore, it is not practical due to too many large modification indexes while testing MI over many groups. Besides scalar model is a poor model fit when comparing many groups and so does not hold MI. In this study, the aim is to explain the basic concepts and processes of the alignment method which is offered as a new method for testing MI and illustrate an application on the real data set. In this study, measurement invariance among 56 countries including Turkey is tested with alignment method in order to set an example for researchers. For this purpose, the Instrumental Motivation Scale data, which is one of the psychological measurement instruments used in PISA 2015, was used. As a result of MG-CFA, it was found that configural invariance was ensured. The fit indexes of CFI and TLI were calculated as 0.982 and 0.946 respectively in this stage. After that, metric invariance was tested by considering the difference of fit indices obtained for the two stages. It was found that the metric invariance could not be provided. Alignment results show which countries hold MI and which do not. Besides it provides information which items have the most invariants for groups that hold MI.

Abstract

It is necessary to examine the measurement invariance (MI) among groups in studies where different groups are compared by using a measurement instrument. Most of the studies, measurement invariance is tested with multiple group confirmatory factor analysis. This model applies many model adjustments based on the modification indexes. Therefore, it is not practical due to too many large modification indexes while testing MI over many groups. Besides scalar model is a poor model fit when comparing many groups and so does not hold MI. In this study, the aim is to explain the basic concepts and processes of the alignment method which is offered as a new method for testing MI and illustrate an application on the real data set. In this study, measurement invariance among 56 countries including Turkey is tested with alignment method in order to set an example for researchers. For this purpose, the Instrumental Motivation Scale data, which is one of the psychological measurement instruments used in PISA 2015, was used. As a result of MG-CFA, it was found that configural invariance was ensured. The fit indexes of CFI and TLI were calculated as 0.982 and 0.946 respectively in this stage. After that, metric invariance was tested by considering the difference of fit indices obtained for the two stages. It was found that the metric invariance could not be provided. Alignment results show which countries hold MI and which do not. Besides it provides information which items have the most invariants for groups that hold MI.

🔖

Bu Makaleye Atıf Yap

Sırgancı, G., Uyumaz, G., Yandı, A. (2020). Measurement Invariance Testing with Alignment Method: Many Groups Comparison. International Journal of Assessment Tools in Education, Volume: 7 Issue: 4. https://doi.org/10.21449/ijate.714218

📄 Tam Metin PDF Okuyucu

📄

PDF görüntüleyici hazırlanıyor...

Kaynakça (36)

  1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
  2. Asil, M., & Gelbal, S. (2012). PISA öğrenci anketinin kültürler arası eşdeğerliği. Eğitim ve Bilim, 37(166), 236-249.
  3. Asil, M., & Brown, G. T. L. (2015). Comparing OECD PISA reading in English to other languages: Identifying potential sources of non-invariance. International Journal of Testing, (16)1, 71-93. https://doi.org/10.1080/15305058.2015.1064431
  4. Asparouhov, T., & Muthén, B. (2014). Multiple-group factor analysis alignment. Structural Equation Modeling: A Multidisciplinary Journal, 21(4), 495 508. https://doi.org/10.1080/10705511.2014.919210
  5. Byrne, B.M., Shavelson, R.J., & Muthén, B.O. (1989). Testing for equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105(3), 456-466.
  6. Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items (4th ed.). Thousand Oaks, California, USA: Sage Publications.
  7. Drasgow, F., & Kanfer, R. (1985). Equivalence of psychological measurement in heterogeneous populations. Journal of Applied Psychology, 70(4), 662-80.
  8. Ercikan, K., & Koh, K. (2005). Examining the construct comparability of the English and French versionf of TIMSS. International Journal of Testing, 5(1), 23-35. https://doi.org/10.1207/s15327574ijt0501_3
  9. Fan, X., & Sivo, S. A. (2009). Using Δ-goodness-of-fit indexes in assessing mean structure invariance. Structural Equation Modeling: A Multidisciplinary Journal, 16(1), 54-69, https://doi.org/10.1080/10705510802561311
  10. Gülleroğlu, H. D. (2017). PISA 2012 matematik uygulamasına katılan türk öğrencilerin duyuşsal özelliklerinin cinsiyete göre ölçme değişmezliğinin incelenmesi, Gazi Üniversitesi Gazi Eğitim Fakültesi Dergisi, 37(1), 151-175.
  11. Gür, E. (2019). PISA 2015 uygulamasındaki maddelerin kültüre göre değişen madde fonksiyonu acısından incelenmesi (Master's thesis). Hacettepe University, Graduate School of Educational Sciences, Ankara, Türkiye.
  12. Hansson, Å., & Gustafsson, J. (2013). Measurement invariance of socioeconomic status across migrational background, Scandinavian Journal of Educational Research, 57(2), 148-166. https://doi.org/10.1080/00313831.2011.625570
  13. İmrol, F. (2017). Pisa 2012 Türkiye örnekleminde matematiğe yönelik motivasyon ve öz-inanç yapılarının ölçme değişmezliğinin incelenmesi (Master's thesis). Ankara University Graduate School of Educational Sciences, Ankara, Türkiye.
  14. Jennrich, R. I. (2006). Rotation to simple loadings using component loss functions: The oblique case. Psychometrika, 71(1), 173-191. https://doi.org/10.1007/s11336-003-1136-B
  15. Jöreskog, K.G., Sörbom, D., Du Toit, S.H.C., & Du Toit, M. (2001). LISREL 8: New statistical features (3rd ed.). Lincolnwood, IL: Scientific Software International.
  16. Kıbrıslıoğlu, N. (2015). PISA 2012 matematik öğrenme modelinin kültürlere ve cinsiyete göre ölçme değişmezliğinin incelenmesi: Türkiye, Çin (Şangay)-Endonezya örneği. (Master’s thesis, Hacettepe University, Graduate School of Educational Sciences, Ankara, Türkiye). Retrieved from http://openaccess.hacettepe.edu.tr:8080/xmlui/bitstream/handle/11655/1843/70bd9399-b70f-414a-a11f-b81218adb77c.pdf?sequence=1
  17. Kim, E. S., Cao, C., Wang, Y., & Nguyen, D. T. (2017). Measurement invariance testing with many groups: a comparison of five approaches, Structural Equation Modeling: A Multidisciplinary Journal, 24(4), 524 544. https://doi.org/10.1080/10705511.2017.1304822
  18. Luo, C. (2010). Measurement invariance of Rosenberg Self-Esteem Scale between British and Chinese college students. (Master's thesis The University of Edinburgh, Edinburgh, Scotland). Retrieved from https://pdfs.semanticscholar.org/274b/015fedaa0c835d94680df2a3c153aff36e5a.pdf
  19. Marsh, H. W., Guo, J., Parker, P. D., Nagengast, B., Asparouhov, T., Muthén, B., & Dicke, T. (2018). What to do when scalar invariance fails: The extended alignment method for multi-group factor analysis comparison of latent means across many groups. Psychological Methods, 23(3), 524-545. https://doi.org/10.1037/met0000113
  20. Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. ETS Research Report Series, 2, i-28. https://doi.org/10.1002/j.2333-8504.1994.tb01618.x
  21. Muthén, B., & Asparouhov, T. (2018). Recent methods for the study of measurement invariance with many groups: alignment and random effects. Sociological Methods & Research, 47(4) 637-664. https://doi.org/10.1177/0049124117701488
  22. Pauwels, J. (2018) Do you really measure the same? (Doctoral dissertation, Ghent University, Gent, Belgium). Retrieved from: https://lib.ugent.be/fulltxt/RUG01/002/481/799/RUG01-002481799_2018_0001_AC.pdf
  23. OECD (2016). PISA 2015 Results (Volume II): Policies and Practices for Successful Schools, PISA, OECD Publishing, Paris, https://doi.org/10.1787/9789264267510-en
  24. Rutkowski, L., & Svetina, D. (2014). Assessing the hypothesis of measurement invariance in the context of large-scale international surveys. Educational and Psychological Measurement, 74(1), 31–57. https://doi.org/10.1177/ 0013164413498257
  25. Sırgancı, G., & Çakan, M. (2020). Sirali lojistik regresyon ve poly-sibtest yöntemleri ile değişen madde fonksiyonunun belirlenmesi. Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 20(1), 705-717.
  26. Sokolov, B. (2019). Sensitivity of goodness of fit indices to lack of measurement invariance with categorical indicators and many groups. Higher School of Economics Research Paper No. WP BRP 86/SOC/2019. Retrieved from https://wp.hse.ru/data/2019/07/09/1480015921/86SOC2019.pdf
  27. Sörbom, D. (1989). Model modification. Psychometrika, 54, 371–384.
  28. Tiryaki, F. (2019). PISA 2015 öğrenci tutum anketlerinin değişen madde fonksiyonu ve ölçme değişmezliğinin incelenmesi (Master’s Thesis). Ankara University Graduate School of Educational Sciences, Ankara, Türkiye
  29. Uyar, Ş. (2011). PISA 2009 Türkiye örnekleminde öğrenme stratejileri modelinin farklı gruplarda ölçme değişmezliğinin incelenmesi (Master’s Thesis). Hacettepe University Graduate School of Educational Sciences, Ankara, Türkiye
  30. Uyar, Ş., & Uyanık, G. K. (2019). Fen bilimlerine yönelik öğrenme modelinin ölçme değişmezliğinin incelenmesi: Pisa 2015 örneği. Kastamonu Eğitim Dergisi, 27(2), 497-507. https://doi.org/10.24106/kefdergi.2570
  31. Uzun, B., & Öğretmen, T. (2010). Fen başarısı ile ilgili bazı değişkenlerin timss-r türkiye örnekleminde cinsiyete göre ölçme değişmezliğinin değerlendirilmesi. Eğitim ve Bilim, 35(155), 26-35.
  32. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the MI literature: suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-70. https://doi.org/10.1177/109442810031002
  33. Widaman, K. F., & Reise, S. P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In K. J. Bryant, M. Windle, & S. G. West (Eds.), The science of prevention: Methodological advances from alcohol and substance abuse research (p. 281–324). Washington DC, USA: American Psychological Association. https://doi.org/10.1037/10222-009
  34. Wu, A. D., Li, Z., & Zumbo, B. D. (2007). Decoding the meaning of factorial ınvariance and updating the practice of multigroup confirmatory factor analysis: A demonstration with TIMSS data. Practical Assessment, Research and Evaluation, 12, 1-26. https://doi.org/10.7275/mhqa-cd89
  35. Yandı, A., Köse, İ. A., & Uysal, Ö. (2017). Farklı yöntemlerle ölçme değişmezliğinin incelenmesi: PISA 2012 örneği. Mersin Üniversitesi Eğitim Fakültesi Dergisi, 13(1), 243-253. https://doi.org/10.17860/mersinefd.305952
  36. Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and likert-type (Ordinal) item scores. Ottawa ON: Directorate of Human Resources Research and Evaluation, Department of National Defense.