SavedAdd your journal

Journals / International Journal of Assessment Tools in Education / 2021 / Cilt: 8 - Sayı: 3

Comparison of G and Phi coefficients estimated in generalizability theory with real cases

Kaan Zulfikar DENİZ
Emel ILICAN
Pages
583–595
Publication Date
2022-10-18
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

In other cases, where the mean difficulty of the test changed because of the addition of easy or difficult tasks, it was determined that the reliability coefficients estimated in the D study and obtained in real cases were similar, but they had different values.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:This study aims to compare the G and Phi coefficients as estimated by D studies for a measurement tool with the G and Phi coefficients obtained from real cases in which items of differing difficulty levels were added and also to determine the conditions under which the D studies estimated reliability coefficients closer to reality.
📊
Key Findings & Data:The study group for this research consisted of 80 seventh-grade students from various public and private secondary schools in the provinces of Ankara, Istanbul, and Adana in Turkey.
💡
Conclusion & Impact:In other cases, where the mean difficulty of the test changed because of the addition of easy or difficult tasks, it was determined that the reliability coefficients estimated in the D study and obtained in real cases were similar, but they had different values.
🔬Design:Research Article
📍Scope:Comprehensive Sample
⚙️Variables:Reliability · Generalizability theory · Decision study

Abstract

This study aims to compare the G and Phi coefficients as estimated by D studies for a measurement tool with the G and Phi coefficients obtained from real cases in which items of differing difficulty levels were added and also to determine the conditions under which the D studies estimated reliability coefficients closer to reality. The study group for this research consisted of 80 seventh-grade students from various public and private secondary schools in the provinces of Ankara, Istanbul, and Adana in Turkey. Four raters who served as Turkish teachers in various public secondary schools in Ankara were included in this study. A data collection tool consisting of 12 tasks was prepared to measure the participating seventh grade students’ written expression skills in Turkish. The equation of the G and Phi coefficients estimated in the D study and obtained through the real cases was observed only when six tasks with item difficulty indexes close to the mean difficulty of the test were added in such a way that the mean difficulty of the test never changed. In other cases, where the mean difficulty of the test changed because of the addition of easy or difficult tasks, it was determined that the reliability coefficients estimated in the D study and obtained in real cases were similar, but they had different values.

Özet

This study aims to compare the G and Phi coefficients as estimated by D studies for a measurement tool with the G and Phi coefficients obtained from real cases in which items of differing difficulty levels were added and also to determine the conditions under which the D studies estimated reliability coefficients closer to reality. The study group for this research consisted of 80 seventh-grade students from various public and private secondary schools in the provinces of Ankara, Istanbul, and Adana in Turkey. Four raters who served as Turkish teachers in various public secondary schools in Ankara were included in this study. A data collection tool consisting of 12 tasks was prepared to measure the participating seventh grade students’ written expression skills in Turkish. The equation of the G and Phi coefficients estimated in the D study and obtained through the real cases was observed only when six tasks with item difficulty indexes close to the mean difficulty of the test were added in such a way that the mean difficulty of the test never changed. In other cases, where the mean difficulty of the test changed because of the addition of easy or difficult tasks, it was determined that the reliability coefficients estimated in the D study and obtained in real cases were similar, but they had different values.

🔖

Cite This Article

DENİZ, K. Z., ILICAN, E. (2022). Comparison of G and Phi coefficients estimated in generalizability theory with real cases. International Journal of Assessment Tools in Education, Cilt: 8 - Sayı: 3, 583–595. https://ajindex.com/articles/540071

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (6)

  1. Aiken, L., R. (2009). Psychological testing and assessment (Twelfth ed.). Pearson. Anastasi, A., & Urbina, S. (1997). Psychological testing. Pearson. Ankenmann, R. D., & Stone, C. A. (1992, April). A monte carlo study of marginal maximum likelihood parameter estimates fort he graded model. Paper presented at the Annual Meeting of the Council on Measurement in Education, San Francisco, CA. Atılgan, H., & Tezbaşaran, A. A. (2005). Genellenebilirlik kuramı alternatif karar çalışmaları ile senaryolar ve gerçek durumlar için elde edilen G ve Phi katsayılarının tutarlılığının incelenmesi [An investigation on consistency of G and Phi coefficients obtained by generalizability theory alternative decisions study for scenarios and actual cases]. Eurasian Journal of Educational Research, 18, 236-252.
  2. Baykul, Y. (2015). Eğitimde ve psikolojide ölçme: Klasik test teorisi ve uygulaması. Pegem. Bıkmaz Bilgen, Ö., & Doğan, N. (2017). Çok kategorili parametrik ve parametrik olmayan madde tepki kuramı modellerinin karşılaştırılması [Comparison of polytomous parametric and nonparametric item response theory models]. Journal of Measurement and Evaluation in Education and Psychology, 8(4), 354-372.
  3. Brennan, R. L. (2001). Generalizability theory. Springer-Verlag. Büyüköztürk, Ş., Çakmak, E.K., Akgün, Ö.E., Karadeniz, Ş. ve Demirel, F. (2012). Bilimsel Araştırma Yöntemleri. Pegem. Choi, J., & Wilson, M. R. (2018). Modeling rater effects using a combination of generalizability theory and IRT. Psychological Test and Assessment Modeling, 60(1), 53-80.
  4. Crocker, L., & Algina J. (1986). Introduction to classical and modern test theory. Harcourt Brace Jovanovich Inc. Çakıcı Eser, D., & Gelbal, S. (2013). Genellenebilirlik kuramı ve lojistik regresyona dayalı hesaplanan puanlayıcılar arası tutarlılığın karşılaştırılması [Comparison of interrater agreement calculated with generalizability theory and logistic regression]. Kastamonu Education Journal, 21(2), 421-438.
  5. Çakır, M., & Aldemir, B. (2011). İki aşamalı genetik kavramlar tanı testi geliştirme ve geçerlik çalışması [Developing and validating a two tier mendel genetics diagnostic test]. Mustafa Kemal University Journal of Social Sciences Institute, 8(16), 335-353.
  6. Deliceoğlu, G., & Çıkrıkçı Demirtaşlı, N. (2012). Futbol yetilerine ilişkin dereceleme ölçeğinin güvenirliğinin genellenebilirlik kuramına ve klasik test kuramına dayalı olarak karşılaştırılması [The comparison of the reliability of the soccer abilities’ rating scale based o