SavedAdd your journal

Journals / Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi / 2026 / Volume: 10 Issue: 2

The Effect of Item Weighting on Reliability and Validity

Abdullah Faruk Kılıç(HACETTEPE ÜNİVERSİTESİ, EĞİTİM BİLİMLERİ ENSTİTÜSÜ)0000-0003-3129-1763
Nuri Doğan(HACETTEPE ÜNİVERSİTESİ, EĞİTİM FAKÜLTESİ)0000-0001-6274-2016
Pages
—
Publication Date
2019-06-28
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

According to the results of confirmatory factor analysis, the comparative fit index (CFI) and the root mean square error of approximation (RMSEA) values were improved.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:The purpose of this study is to examine the effect of the item weighting method developed by researchers on the construct validity of the test.
📊
Key Findings & Data:Item weighting method was defined as follows: If average score of the individuals (calculated as individual's test score/the number of items) plus item difficulty index is 1 and over then item reliability index added to individual’s item score (1 or 0); if not, then the item score of the individual (1 if 1, 0 if 0) is preserved.
💡
Conclusion & Impact:According to the results of confirmatory factor analysis, the comparative fit index (CFI) and the root mean square error of approximation (RMSEA) values were improved.
🔬Design:Research Article
📍Scope:Comprehensive Sample
⚙️Variables:item weighting · validity · reliability

Abstract

The purpose of this study is to examine the effect of the item weighting method developed by researchers on the construct validity of the test. For this purpose, a Monte Carlo simulation study was carried out. Test length, average factor loadings, and sample size were considered as simulation conditions. Item weighting method was defined as follows: If average score of the individuals (calculated as individual's test score/the number of items) plus item difficulty index is 1 and over then item reliability index added to individual’s item score (1 or 0); if not, then the item score of the individual (1 if 1, 0 if 0) is preserved. As a result of the research, it was observed that the weighting method contributes to the construct validity. According to the results of confirmatory factor analysis, the comparative fit index (CFI) and the root mean square error of approximation (RMSEA) values were improved. According to the research findings, the weighting method used in this research can be recommended.

🔖

Cite This Article

Kılıç, A. F., Doğan, N. (2019). The Effect of Item Weighting on Reliability and Validity. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, Volume: 10 Issue: 2. https://doi.org/10.21031/epod.516057

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (23)

  1. AERA, APA, & NCME. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
  2. Akkuş, O., & Baykul, Y. (2001). Çoktan seçmeli test maddelerinin puanlamada, seçenekleri farklı biçimde ağırlıklandırmanın madde ve test istatistiklerine olan etkisinin incelenmesi. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 20, 9–15.
  3. Birnbaum, A. (1968). Some latent models and their use in inferring an examinee’s ability. In F. M. Lord & M. R. Novick (Eds.), Statistical Theories of Mental Test Scores (pp. 397–479). Addison-Wesley: Reading, MA.
  4. Burt, C. (1950). The influence of differential weighting. British Journal of Statistical Psychology, 3(2), 105–125. https://doi.org/10.1111/j.2044-8317.1950.tb00288.x
  5. Daniel, L. G. (1989). Comparisons of of exploratory and confirmatory factor analysis. In Mid-South Educational Research Association. Little Rock, AR: (ERIC Document Reproduction Service No. ED314447.
  6. Dick, W. (1965). Item weighting: Test parameter effects and comparison of the efficiency of various weighting methods. The Pennsylvania State University.
  7. Erdem, B., Ertuna, L., & Doğan, N. (2016, September). Çoktan seçmeli testlerde seçenek ağırlıklandırma yöntemlerinin testin faktör yapısına etkisinin incelenmesi. Paper presented at the V. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Kongresi, Antalya, Turkey. Retrived from http://epod2016.akdeniz.edu.tr/_dinamik/333/53.pdf.
  8. Ghiselli, E. E. (1964). Theory of psychological measurement. New York: McGraw-Hill.
  9. Gözen-Çıtak, G. (2010). Klasik Test ve Madde Tepki Kuramlarına göre çoktan seçmeli testlerde farklı puanlama yöntemlerinin karşılaştırılması. İlköğretim Online, 9(1), 170–187.
  10. Guilford, J. P. (1954). Psychometric methods (2. Edition). New York: McGraw-Hill.
  11. Guilford, J. P., Lovell, C., & Williams, R. M. (1942). Completely weighted versus unweighted scoring in an achievement examination. Educational and Psychological Measurement, 2, 15–21.
  12. Gulliksen, H. (1950). Theory of mental tests. New York: Wiley.
  13. Hallquist, M., & Wiley, J. (2017). MplusAutomation: Automating Mplus Model Estimation and Interpretation. Retrieved from https://cran.r-project.org/package=MplusAutomation
  14. Horst, P. (1936). Obtaining a composite measure from a number of different measures of the same attribute. Psychometrika, 1(1), 53–60. https://doi.org/10.1007/BF02287924
  15. Messick, S. (1995). Validity of psychological assessment. American Psychologist, 50(9), 741–749. https://doi.org/10.1037//0003-066X.50.9.741
  16. Nunnally, J. C. (1978). Psychometric theory (2. Baskı). New York: McGraw-Hill.
  17. Özdemir, D. (2004). Çoktan seçmeli testlerin Klasik Test Teorisi ve Örtük Özellikler Teorisine göre hesaplanan psikometrik özelliklerinin iki kategorili ve ağırlıklandırılmış puanlanması yönünden karşılaştırılması. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 26, 117–123.
  18. Phillips, A. J. (1943). Further evidence regarding weighted versus unweighted scoring of examinations. Educational and Psychological Measurement, 3(1), 151–155. https://doi.org/10.1177/001316444300300114
  19. R Core Team. (2018). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.r-project.org/.
  20. Revelle, W. (2016). psych: Procedures for psychological, psychometric, and personality research. Evanston, Illinois. Retrieved from https://cran.r-project.org/package=psych
  21. Rotou, O., Headrick, T. C., & Elmore, P. B. (2002). An investigation of difference scores for a grade-level testing program. International Journal of Testing, 2(2), 83–105. https://doi.org/10.1207/S15327574IJT0202
  22. Stevens, J. P. (2009). Applied multivariate statistics for the social science (5. Baskı). London: Routledge.
  23. Yurdugül, H. (2010). Farklı madde puanlama yöntemlerinin ve farklı test puanlama yöntemlerinin karşılaştırılması. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 1(1), 1–8.