SavedAdd your journal

Journals / Kastamonu Education Journal / 2017 / Cilt: 25 Sayı: 1

Comparing Fully Crossed and Nested Designs Where Items Nested in Raters in Generalizability Theory

GENELLENEBİLİRLİK KURAMINDA TÜMÜYLE ÇAPRAZLANMIŞ ve MADDELERİN PUANLAYICILARA YUVALANDIĞI DESENLERİN KARŞILAŞTIRILMASI

Celal Deha DOĞAN
Hatice Özlem ANADOL
Pages
361–372
Publication Date
2017-01-15
DOI
—
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

According to the results, G and Phi coefficients of thefully crossed design were higher.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:This study aims to investigate the comparison of G and Phi coefficients calculated bothby a fully crossed (bxpxm) design (b: person; p: rater; m:item) and a nested (bx(m:p)) designin which items are nested in raters but individuals are crossed with items and raters in theprocess of grading English Composition Writing Skill.
📊
Key Findings & Data:The study consists of students whoattend a private university and 3 raters.
💡
Conclusion & Impact:According to the results, G and Phi coefficients of thefully crossed design were higher.
🔬Design:Research Article
📍Scope:Comprehensive Sample
⚙️Variables:Puanlayıcılar arası Güvenirlik · Genellenebilirlik Kuramı · Tümüyle Çaprazlanmış Desenler

Abstract

This study aims to investigate the comparison of G and Phi coefficients calculated bothby a fully crossed (bxpxm) design (b: person; p: rater; m:item) and a nested (bx(m:p)) designin which items are nested in raters but individuals are crossed with items and raters in theprocess of grading English Composition Writing Skill. The study consists of students whoattend a private university and 3 raters. According to the results, G and Phi coefficients of thefully crossed design were higher. In terms of sources of variance, while person has the highestpercentage in total variance in fully crossed design, occasion has relatively low percentage.Findings indicate that fully crossed design yields more reliable results in classroom practices.In this respect, fully crossed design is recommended for classroom practices.

Özet

Bu çalışmada İngilizce Kompozisyon Yazma Becerisinin puanlanması sürecinde tümüyle çaprazlanmış desenin (bxpxm) ve maddelerin puanlayıcıya yuvalandığı ancak bireylerin maddeler ve puanlayıcılar ile çaprazlanmış olduğu yuvalanmış desenin (bx(m:p)) kullanıldığı durumlarda elde edilen G ve Phi katsayılarının karşılaştırılması amaçlanmıştır. Çalışmaya bir vakıf üniversitesi hazırlık okulunda öğrenim gören ve 3 puanlayıcı dahil olmuştur. Çalışma sonucunda tümüyle çaprazlamış desen ile elde edilen G ve Phi katsayıları daha yüksek çıkmıştır. Değişkenlik kaynaklarına göre varyans bileşenleri incelendiğinde birey ana etkisine ilişkin varyans tümüyle çaprazlanmış desen için daha yüksekken kalan etkiye ilişkin varyans değeri daha düşüktür. Bu bulgular sınıf içi uygulamalarda tümüyle çaprazlanmış desenin daha güvenilir sonuçlar verdiğini göstermektedir. Bu bağlamda sınıf içi uygulamalarda zorunlu kalınmadıkça tümüyle çaprazlanmış desenin kullanılması önerilmektedir.

🔖

Cite This Article

DOĞAN, C. D., ANADOL, H. Ö. (2017). Comparing Fully Crossed and Nested Designs Where Items Nested in Raters in Generalizability Theory. Kastamonu Education Journal, Cilt: 25 Sayı: 1, 361–372. https://ajindex.com/articles/940723

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (10)

  1. Akın, Ö.,&Baştürk, R. (2010). Assessment of research assignment by many-facet Rasch measurement approach. Journal of Measurement and Evaluation in Education and Psychology, 1(1), 51–57.
  2. Akın, Ö.,&Baştürk, R. (2012). The evaluation of the basic skills in violin training by many-facetrasch model. Pamukkale Üniversity Journal of Education, 31, 175–187.
  3. Arter, J. A. Ve Mctighe, J. (2001). Scoring Rubrics in TheClassroom:Using Peformance Criteria for Assessing and Improving Student Performance, Thousand Oaks, CA: Corvin Press Büyükkıdık, S., & Anıl, D. (2015). Investigation of reliability in generalizability theory with different designs on performance based assessment. Education and Science, 40(177), 285–296.
  4. Doğan, C. D., &Yosmaoğlu, B. (2015). The effect of the analytical rubrics on the objectivity in physiotherapy practical examination.TurkiyeKlinikleri Journal of Sports Science, 7(1), 9–15.
  5. Engelhard, G. (1994). Examining rater errors in the assessment of written composition with a many‐faceted Rasch model. Journal of Educational Measurement, 31(2), 93–112.
  6. Engelhard, G., &Myford, C. M. (2003). Monitoring faculty consultant performance in the advanced placement English Literature and composition program with a many‐faceted Rasch model. ETS Research Report Series (1), i-60. Goodrich, H. (1996).Students Self Assessment: At theintersection of metacognition and authentic assessment. Doctoraldisertation. Cambrdige, MA: HarwardUniversity Güler, N., &Gelbal, S. (2010a). Studying reliability of open-ended mathematics items according to the classical test theory and generalizability. Educational Sciences: Theory & Practice, 10(2), 989–1019.
  7. Güler, N., &Gelbal, S. (2010b). A study based on the classical test theory and many facet Rasch model. Egitim Eurasian Journal of Educational Research, 38, 108–125.
  8. Güler, N, Uyanık,G. K., Teker, G. T. (2012). Genellenebilirlik Kuramı. Pegem Akademi, Ankara Türkiye Grounlond, N. E. (1998).Assessment of StudentAchievement. USA: By Allyn & Bacon Viacom Company Iramaneerat, C., Yudkowsky, R., Myford, C. M., & Downing, S. M. (2008). Quality control of an OSCE using generalizability theory and many-faceted Rasch measurement. Advances in Health Sciences Education, 13(4), 479–493.
  9. Iramaneerat, C., Myford, C. M., Yudkowsky, R., & Lowenstein, T. (2009). Evaluating the effectiveness of rating instruments for a communication skills assessment of medical residents. Advances in Health Sciences Education, 14 (4), 575–594.
  10. Kan, A. (2005a). The