Journals / International Journal of Assessment Tools in Education / 2020 / Cilt: 7 - Sayı: 4

Measurement Invariance Testing with Alignment Method: Many Groups Comparison

Pages
657–673
Publication Date
2022-10-19
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

As a result of MG-CFA, it was found that configural invariance was ensured.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:It is necessary to examine the measurement invariance (MI) among groups in studies where different groups are compared by using a measurement instrument.
📊
Key Findings & Data:In this study, the aim is to explain the basic concepts and processes of the alignment method which is offered as a new method for testing MI and illustrate an application on the real data set.
💡
Conclusion & Impact:As a result of MG-CFA, it was found that configural invariance was ensured.
🔬Design:Modeling & Simulation
📍Scope:2015 Verileri
⚙️Variables:Multi-group confirmatory factor analysis · Validity · Modification indexes

Abstract

It is necessary to examine the measurement invariance (MI) among groups in studies where different groups are compared by using a measurement instrument. Most of the studies, measurement invariance is tested with multiple group confirmatory factor analysis. This model applies many model adjustments based on the modification indexes. Therefore, it is not practical due to too many large modification indexes while testing MI over many groups. Besides scalar model is a poor model fit when comparing many groups and so does not hold MI. In this study, the aim is to explain the basic concepts and processes of the alignment method which is offered as a new method for testing MI and illustrate an application on the real data set. In this study, measurement invariance among 56 countries including Turkey is tested with alignment method in order to set an example for researchers. For this purpose, the Instrumental Motivation Scale data, which is one of the psychological measurement instruments used in PISA 2015, was used. As a result of MG-CFA, it was found that configural invariance was ensured. The fit indexes of CFI and TLI were calculated as 0.982 and 0.946 respectively in this stage. After that, metric invariance was tested by considering the difference of fit indices obtained for the two stages. It was found that the metric invariance could not be provided. Alignment results show which countries hold MI and which do not. Besides it provides information which items have the most invariants for groups that hold MI.

Özet

It is necessary to examine the measurement invariance (MI) among groups in studies where different groups are compared by using a measurement instrument. Most of the studies, measurement invariance is tested with multiple group confirmatory factor analysis. This model applies many model adjustments based on the modification indexes. Therefore, it is not practical due to too many large modification indexes while testing MI over many groups. Besides scalar model is a poor model fit when comparing many groups and so does not hold MI. In this study, the aim is to explain the basic concepts and processes of the alignment method which is offered as a new method for testing MI and illustrate an application on the real data set. In this study, measurement invariance among 56 countries including Turkey is tested with alignment method in order to set an example for researchers. For this purpose, the Instrumental Motivation Scale data, which is one of the psychological measurement instruments used in PISA 2015, was used. As a result of MG-CFA, it was found that configural invariance was ensured. The fit indexes of CFI and TLI were calculated as 0.982 and 0.946 respectively in this stage. After that, metric invariance was tested by considering the difference of fit indices obtained for the two stages. It was found that the metric invariance could not be provided. Alignment results show which countries hold MI and which do not. Besides it provides information which items have the most invariants for groups that hold MI.

🔖

Cite This Article

Uyumaz, G., Sırgancı, G., Yandı, A. (2022). Measurement Invariance Testing with Alignment Method: Many Groups Comparison. International Journal of Assessment Tools in Education, Cilt: 7 - Sayı: 4, 657–673. https://ajindex.com/articles/540134

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (10)

  1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Asil, M., & Gelbal, S. (2012). PISA öğrenci anketinin kültürler arası eşdeğerliği. Eğitim ve Bilim, 37(166), 236-249.
  2. Asil, M., & Brown, G. T. L. (2015). Comparing OECD PISA reading in English to other languages: Identifying potential sources of non-invariance. International Journal of Testing, (16)1, 71-93. https://doi.org/10.1080/15305058.2015.1064431
  3. Asparouhov, T., & Muthén, B. (2014). Multiple-group factor analysis alignment. Structural Equation Modeling: A Multidisciplinary Journal, 21(4), 495 508. https://doi.org/10.1080/10705511.2014.919210
  4. Byrne, B.M., Shavelson, R.J., & Muthén, B.O. (1989). Testing for equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105(3), 456-466.
  5. Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items (4th ed.). Thousand Oaks, California, USA: Sage Publications. Drasgow, F., & Kanfer, R. (1985). Equivalence of psychological measurement in heterogeneous populations. Journal of Applied Psychology, 70(4), 662-80.
  6. Ercikan, K., & Koh, K. (2005). Examining the construct comparability of the English and French versionf of TIMSS. International Journal of Testing, 5(1), 23-35. https://doi.org/10.1207/s15327574ijt0501_3
  7. Fan, X., & Sivo, S. A. (2009). Using Δ-goodness-of-fit indexes in assessing mean structure invariance. Structural Equation Modeling: A Multidisciplinary Journal, 16(1), 54-69, https://doi.org/10.1080/10705510802561311
  8. Gülleroğlu, H. D. (2017). PISA 2012 matematik uygulamasına katılan türk öğrencilerin duyuşsal özelliklerinin cinsiyete göre ölçme değişmezliğinin incelenmesi, Gazi Üniversitesi Gazi Eğitim Fakültesi Dergisi, 37(1), 151-175.
  9. Gür, E. (2019). PISA 2015 uygulamasındaki maddelerin kültüre göre değişen madde fonksiyonu acısından incelenmesi (Master's thesis). Hacettepe University, Graduate School of Educational Sciences, Ankara, Türkiye. Hansson, Å., & Gustafsson, J. (2013). Measurement invariance of socioeconomic status across migrational background, Scandinavian Journal of Educational Research, 57(2), 148-166. https://doi.org/10.1080/00313831.2011.625570
  10. İmrol, F. (2017). Pisa 2012 Türkiye örnekleminde matematiğe yönelik motivasyon ve öz-inanç yapılarının ölçme değişmezli