SavedAdd your journal

Journals / Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi / 2019 / Cilt: 10 - Sayı: 2

The Effect of Item Weighting on Reliability and Validity

Nuri DOĞAN
Abdullah Faruk KILIÇ
Pages
149–164
Publication Date
2022-10-12
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

According to the results of confirmatory factor analysis, the comparative fit index (CFI) and the root mean square error of approximation (RMSEA) values were improved.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:The purpose of this study is to examine the effect of the item weighting method developed by researchers on the construct validity of the test.
📊
Key Findings & Data:Item weighting method was defined as follows: If average score of the individuals (calculated as individual's test score/the number of items) plus item difficulty index is 1 and over then item reliability index added to individual’s item score (1 or 0); if not, then the item score of the individual (1 if 1, 0 if 0) is preserved.
💡
Conclusion & Impact:According to the results of confirmatory factor analysis, the comparative fit index (CFI) and the root mean square error of approximation (RMSEA) values were improved.
🔬Design:Research Article
📍Scope:Comprehensive Sample
⚙️Variables:Domain Parameters

Abstract

The purpose of this study is to examine the effect of the item weighting method developed by researchers on the construct validity of the test. For this purpose, a Monte Carlo simulation study was carried out. Test length, average factor loadings, and sample size were considered as simulation conditions. Item weighting method was defined as follows: If average score of the individuals (calculated as individual's test score/the number of items) plus item difficulty index is 1 and over then item reliability index added to individual’s item score (1 or 0); if not, then the item score of the individual (1 if 1, 0 if 0) is preserved. As a result of the research, it was observed that the weighting method contributes to the construct validity. According to the results of confirmatory factor analysis, the comparative fit index (CFI) and the root mean square error of approximation (RMSEA) values were improved. According to the research findings, the weighting method used in this research can be recommended.

🔖

Cite This Article

DOĞAN, N., KILIÇ, A. F. (2022). The Effect of Item Weighting on Reliability and Validity. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, Cilt: 10 - Sayı: 2, 149–164. https://ajindex.com/articles/497206

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (8)

  1. American Educational Research Association (AERA), American Educational Research Association (APA), & National Council on Measurement In Education (NCME). (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Akkuş, O., & Baykul, Y. (2001). Çoktan seçmeli test maddelerinin puanlamada, seçenekleri farklı biçimde ağırlıklandırmanın madde ve test istatistiklerine olan etkisinin incelenmesi. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 20, 9–15.
  2. Birnbaum, A. (1968). Some latent models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick (Eds.), Statistical Theories of Mental Test Scores (pp. 397–479). Addison-Wesley: Reading, MA. Burt, C. (1950). The influence of differential weighting. British Journal of Statistical Psychology, 3(2), 105–125. https://doi.org/10.1111/j.2044-8317.1950.tb00288.x
  3. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York: Lawrence Erlbaum Associates. Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159. https://doi.org/10.1037/0033-2909.112.1.155
  4. Comrey, A. L. (1988). Factor-analytic methods of scale development in personality and clinical psychology. Journal of Consulting and Clinical Psychology, 56(5), 754–761. https://doi.org/10.1037/0022-006X.56.5.754
  5. Daniel, L. G. (1989). Comparisons of exploratory and confirmatory factor analysis. In Mid-South Educational Research Association. Little Rock, AR: (ERIC Document Reproduction Service No. ED314447). Dick, W. (1965). Item weighting: Test parameter effects and comparison of the efficiency of various weighting methods. (Doctoral Dissertation). Available from ProOuest Dissertations and Theses database. (UMI No. 3564813). Erdem, B., Ertuna, L., & Doğan, N. (2016, September). Çoktan seçmeli testlerde seçenek ağırlıklandırma yöntemlerinin testin faktör yapısına etkisinin incelenmesi (pp. 148–149). Paper presented at the Fifth International Congress on Measurement And Evaluation in Education And Psychology, Antalya, TR. Abstract retrieved from http://epod2016.akdeniz.edu.tr/_dinamik/333/53.pdf.
  6. Erkuş, A. (2014). Psikolojide ölçme ve ölçek geliştirme-I: Temel kavramlar ve işlemler (2nd ed.). Ankara:
  7. Pegem Akademi. Floyd, F. J., & Widaman, K. F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7(3), 286–299. https://doi.org/10.1037/1040-3590.7.3.286
  8. Ghiselli, E. E. (1964).