SavedAdd your journal

Journals / Kuram ve Uygulamada Eğitim Bilimleri / 2016 / Cilt: 16 - Sayı: 3

The Impact of Test Dimensionality, Common-Item Set Format, and Scale Linking Methods on Mixed-Format

NEŞE ÖZTÜRK GÜBEŞ
HÜLYA KELECİOĞLU
Pages
715–734
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

The results showed that: (i) The FOE and SOE properties were best preserved under the unidimensional condition, were poorly preserved when the degree of multidimensionality was severe. (ii) The TSE and OSE results, which were provided by using a mixed-format common-item set, preserved FOE better compared to equating results, which provided only a multiple-choice common item set. (iii) Under the unidimensional and multidimensional test structure, characteristic curve methods performed significantly better than moment scale linking methods in terms of preserving FOE and SOE properties

📌 3 Key Takeaways

🎯
Objective & Hypothesis:The purpose of this study was to examine the impact of dimensionality, common-item set format, and different scale linking methods on preserving equity property with mixed-format test equating.
📊
Key Findings & Data:A simulation study was conducted based on actual item parameter estimates obtained from the TIMSS 2011 8th grade mathematics assessment.
💡
Conclusion & Impact:The results showed that: (i) The FOE and SOE properties were best preserved under the unidimensional condition, were poorly preserved when the degree of multidimensionality was severe. (ii) The TSE and OSE results, which were provided by using a mixed-format common-item set, preserved FOE better compared to equating results, which provided only a multiple-choice common item set. (iii) Under the unidimensional and multidimensional test structure, characteristic curve methods performed significantly better than moment scale linking methods in terms of preserving FOE and SOE properties
🔬Design:Research Article
📍Scope:2011 Verileri
⚙️Variables:Domain Parameters

Abstract

The purpose of this study was to examine the impact of dimensionality, common-item set format, and different scale linking methods on preserving equity property with mixed-format test equating. Item response theory (IRT) true-score equating (TSE) and IRT observed-score equating (OSE) methods were used under common-item nonequivalent groups design. The equity property was evaluated based on firstorder equity (FOE) and second-order equity (SOE) properties. A simulation study was conducted based on actual item parameter estimates obtained from the TIMSS 2011 8th grade mathematics assessment. The results showed that: (i) The FOE and SOE properties were best preserved under the unidimensional condition, were poorly preserved when the degree of multidimensionality was severe. (ii) The TSE and OSE results, which were provided by using a mixed-format common-item set, preserved FOE better compared to equating results, which provided only a multiple-choice common item set. (iii) Under the unidimensional and multidimensional test structure, characteristic curve methods performed significantly better than moment scale linking methods in terms of preserving FOE and SOE properties

🔖

Cite This Article

GÜBEŞ, N. Ö., KELECİOĞLU, H. (2016). The Impact of Test Dimensionality, Common-Item Set Format, and Scale Linking Methods on Mixed-Format. Kuram ve Uygulamada Eğitim Bilimleri, Cilt: 16 - Sayı: 3, 715–734. https://ajindex.com/articles/346021

References (5)

  1. Andrews, B. J. (2011). Assessing first- and second-order equity for the common item nonequivalent groups design using multidimensional IRT (Doctoral dissertation). Available from ProQuest Dissertation and Theses database. (UMI No. 3473138) Baker, F. B., & Al-Karni, A. (1991). A comparison of two procedures for computing IRT equating coefficients. Journal of Educational Measurement, 28(2), 147-162.
  2. Balch, J. (1964). The influence of the evaluating instrument on students' learning. American Educational Research Journal, 1(3), 169-182.
  3. Bastari, B. (2000). Linking multiple-choice and constructed-response items a common proficiency scale (Doctoral dissertation). Available from ProQuest Dissertation and Theses database. (UMI No. 9960735) Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 397- 479). Reading, MA: Addison-Wesley. Bolt, D. M. (1999). Evaluating the effects of multidimensionality on IRT true-score equating. Applied Measurement in Education, 12(4), 383-407. Retrieved from http://dx.doi. org/10.1207/S15324818AME1204_4 Braun, H. I., & Holland, P. W. (1982). Observed-score test equating: A mathematical analysis of some ETS equating procedures. In P. W. Holland & D. B. Rubin (Eds.), Test equating (pp. 9-49).
  4. New York, NY: Academic. Brennan, R. L. (2010). First-order and second-order equity in equating (Report No. 30). Iowa City: Center for Advanced Studies in Measurement and Assessment. Cao, Y. (2008). Mixed-format test equating: Effects of test dimensionality and common item sets (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database. (UMI No. 3341415) Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Dorans, N. J., Moses, T. P., & Eignor, D. R. (2011). Equating test scores: Toward best practices. In A. A. von Davier (Ed.), Statistical models for test equating, scaling, and linking (pp. 21-42).
  5. New York, NY: Springer. Haebara, T. (1980). Equating logistic ability scales by a weighted least squares method. Japanese Psychological Research, 22, 144-149. Retrieved from https://www.jstage.jst. go.jp/article/psycholres1954/22/3/22_3_144/_pdf Hagge, S. L. (2010). The impact of equating method and format representation of common items on the adequacy of mixed-format test equating using nonequivalent groups (Doctoral dissertation). Retr