KaydedilenlerDerginizi ekleyin

Dergiler / Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi / 2025 / Volume: 16 Issue: 4

Examining The Rater Reliability of Laboratory Activities of Gifted Students In Science Courses With The Many Facet Rasch Model

Başak Akar(GAZİANTEP ÜNİVERSİTESİ, GAZİANTEP EĞİTİM FAKÜLTESİ)0009-0002-5798-9705
Bayram Çetin(GAZİANTEP ÜNİVERSİTESİ, GAZİANTEP EĞİTİM FAKÜLTESİ)0000-0001-5321-8028
Sayfa
—
Yayın Tarihi
2025-12-31
🏛️Bu makalenin bibliyografik verileri ve tam metin bağlantısı DergiPark üzerinden sağlanmıştır.
DergiPark Kaydı ↗
✨

Yapay Zeka Hızlı Bakış & Çıkarımlar

AI Insight
⚡ TL;DR(1 Cümlede Hızlı Çıkarım)

Keywords: Many facet Rasch measurement model, gifted students, rater effects

📌 3 Maddede Bu Makale

🎯
Amaç & Hipotez:In this study, it was aimed to examine the inter-rater reliability of the same performance tasks scored by seven-raters with poster scoring rubrics, science-chemistry laboratory applications performance scoring rubrics, and open-ended achievement test scoring rubrics within the framework of the many facet Rasch model.
📊
Temel Bulgular & Veri:The participants were 27 sixth grade gifted-students attending BİLSEM selected by criterion sampling from purposive sampling types and seven teachers serving as raters.
💡
Sonuç & Katkı:Keywords: Many facet Rasch measurement model, gifted students, rater effects
🔬Tür:Modelleme & Sayısal Simülasyon
📍Kapsam:Genişletilmiş Akademik Örneklem
⚙️Değişkenler:Many facet Rasch measurement model · gifted students · rater effects

Abstract

In this study, it was aimed to examine the inter-rater reliability of the same performance tasks scored by seven-raters with poster scoring rubrics, science-chemistry laboratory applications performance scoring rubrics, and open-ended achievement test scoring rubrics within the framework of the many facet Rasch model. The participants were 27 sixth grade gifted-students attending BİLSEM selected by criterion sampling from purposive sampling types and seven teachers serving as raters. Density Open-Ended Achievement Test, Poster Scoring rubrics and Science/Chemistry Laboratory Applications Process Performance Analytical Rubric were used to collect data of the study. In the study, FACETS was used for the MFRM, JASP-0.17.2.1 was used for model – data fit analyses and assumption tests. As a result, it was seen that individuals could be distinguished at the group level in terms of ability levels, the items differed in terms of difficulty levels and there was a difference between the raters in terms of severity/leniency. As a result, it was seen that individuals could be distinguished in terms of ability levels at the group level, the items differed in terms of difficulty levels and there was a difference between the raters in terms of severity/leniency except for the open-ended achievement test. It was seen that the scoring based on the three scoring tools distinguished the individuals in terms of ability levels and the items differed in terms of difficulty levels. In the scoring of the poster and experimental performance evaluation scoring tool, scorers numbered six and seven had scoring severity/leniency errors, while no scoring errors Halo effect in open-ended and doing experiment performance scoring were found in all scoring. There were no inconsistent scoring errors in the scoring of the open-ended achievement test, poster evaluation and experiment performances according to the scoring rubric of the density of irregularly shaped solids. Keywords: Many facet Rasch measurement model, gifted students, rater effects

🔖

Bu Makaleye Atıf Yap

Akar, B., Çetin, B. (2025). Examining The Rater Reliability of Laboratory Activities of Gifted Students In Science Courses With The Many Facet Rasch Model. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, Volume: 16 Issue: 4. https://doi.org/10.21031/epod.1585418

📄 Tam Metin PDF Okuyucu

📄

PDF görüntüleyici hazırlanıyor...

Kaynakça (47)

  1. Akben, N., (2011). Developing a scientific inquiry supported laboratory course for prospective teachers. (Thesis number: 290633). [Doctoral dissertation, T. C. Gazi University]. HigherEducation Institution National Thesis Centre. https://tez.yok.gov.tr/UlusalTezMerkezi/giris.jsp
  2. Aktaş, M. & Alıcı, D. (2017). Investigation of Rater Reliability in Checklist, Analytical Rubric and Rating Scales According to Generalisability Theory, International Journal Of Eurasia Social Sciences, Vol: 8, Issue: 29, pp. (991-1010). https://search.trdizin.gov.tr/tr/yayin/detay/263050/kontrol-listesi-analitik-rubrik-ve-dereceleme-olceklerinde-puanlayici-guvenirliginin-genellenebilirlik-kuramina-gore-incelenmesi
  3. Altunova, N., Artun, H., 2019. Rubric Development Study for the Evaluation of Experiments in Science Laboratory Practices Course. 6th International Multidisciplinary Studies Congress.
  4. Atılgan, 2004. A research on the comparison of generalisability theory and multivariate Rasch model.(Thesis number: 144251). [Doctoral dissertation, T. C. Hacettepe University]. Higher Education Institution National Thesis Centre. https://tez.yok.gov.tr/UlusalTezMerkezi/giris.jsp
  5. Bacanak A., 2008. Investigation of the effectiveness of the web-based programme created for science and technology course performance evaluation forms.(Thesis number: 179206). [Doctoral dissertation, T. C. Karadeniz Technical University]. Higher Education Institution National Thesis Centre. https://tez.yok.gov.tr/UlusalTezMerkezi/giris.jsp
  6. Baştürk, R. (2008). Applying the many facet Rasch model to evaluate PowerPoint presentation performance in higher education. Assessment and Evaluation in Higher Education, 33(4), 431-444. https://doi.org/10.1080/02602930701562775
  7. Baştürk, R. & Işıkoğlu, N. (2008). Analysing the functional quality of pre-school education institutions with Many Facet rasch measurement model. Educational Sciences in Theory and Practice, 8(1), 7-32. https://search.trdizin.gov.tr/tr/yayin/detay/78688
  8. Bıkmaz Bilgen, Ö., & Doğan, N. (2017). Comparison of Inter-rater Reliability Determination Techniques. Journal of Measurement and Evaluation in Education and Psychology, 8(1), 63-78. https://doi.org/10.21031/epod.294847
  9. Brentari, E., & Golia, S. (2008). Measuring job satisfaction in the social services sector with the Rasch model. Journal of Applied Measurement, 9(1), 45-56. https://www.researchgate.net/publication/5668940_Measuring_job_satisfaction_in_the_social_services_sector_with_the_Rasch_Model
  10. Cohen. J. R., Swerdlik M. E., & Phillips, S. M. (1996). Psychological testing and assessment. (3rd Ed.). London: Mayfield
  11. Covill. A. E. (2012). College Students' Use of a Writing Rubric: Effect on Quality of Writing, Self-Efficacy, and Writing Practices. The Journal of Writing Assessment. (5)1. https://escholarship.org/uc/item/5f79g6kz
  12. Çaylak, B., (2009). Investigation of science activities applied in science and art centres. (Thesis number: 246649). [Master's thesis, T. C. İnönü University]. Higher Education Institution National Thesis Centre. https://tez.yok.gov.tr/UlusalTezMerkezi/giris.jsp
  13. Çetin, B., & İlhan, M. (2017). Investigation of rater rigour and generosity in open-ended mathematics questions scored with rubrics based on standard and SOLO Taxonomy. Education and Science, 42(189). http://dx.doi.org/10.15390/EB.2017.5082
  14. Deliceoğlu, G., & Çıkrıkçı Demirtaşlı. N,. (2012). Comparison of the reliability of the rating scale for football skills based on generalisability and classical test theory. Hacettepe Journal of Sport Sciences. 23 (1),1-12. https://dergipark.org.tr/tr/download/article-file/151222
  15. Esfandiari, R. (2015). Rater errors among peer-assessors: applying the many-facet Rasch measurement model. Iranian Journal of Applied Linguistics, 18(2), 77-107. https://doi.org/10.18869/acadpub.ijal.18.2.77
  16. Farrokhi, F., & Esfandiari, R. (2011). A many-facet Rasch model to detect halo effect in three types of raters. Theory & Practice in Language Studies, 1(11), 1531-1540. https://doi.org/10.4304/tpls.1.11.1531-1540
  17. Gisev, N., Bell, J. S., & Chen, T. F. (2013). Interrater agreement and interrater reliability: key concepts, approaches, and applications. Research in Social and Administrative Pharmacy, 9(3), 330-338. https://doi:10.1016/j.sapharm.2012.04.004
  18. Göktürk, Ö., Uzun, N. B., & Aktaş, M. (2022). Performance scale development study for science-chemical laboratory applications. Online Science Education Journal, 7(2),72-84. https://dergipark.org.tr/en/download/article-file/2796015
  19. Güler, N. (2008). Comparison of Reliability According to Generalisability Theory and Classical Test Theory on Random Data. Education and Science, 36 (162) https://egitimvebilim.ted.org.tr/index.php/EB/article/view/993
  20. Hernández-Ureña, O., & Montero-Rojas, E. (2023). Validation of a standardised performance test for selection of Architecture students with the Many-Facet Rasch Measurement Model. Revista de Arquitectura (Bogotá), 25(1), 3-11. https://doi.org/10.14718/RevArq.2023.25.4040
  21. Hung, S.P., Chen, P.S., & Chen, H.C. (2012). Improving creativity performance assessment: A rater effect examination with many facet Rasch model. Creativity Research Journal, 24(4), 345-357. https://doi.org/10.1080/10400419.2012.730331
  22. İlhan, M. (2015). Investigation of rater effects in open-ended mathematics questions scored with rubrics based on standard and solo taxonomy with Many Facet rasch model.(Thesis number: 398394). [Doctoral dissertation, Gaziantep University]. Higher Education Institution National Thesis Centre. https://tez.yok.gov.tr/UlusalTezMerkezi/giris.jsp
  23. Karakaya, İ. (2015). Comparison of self, peer and instructor assessments in the portfolio assessment by using many facet Rasch model. Journal of Education and Human Development, 4(2), 182-192. https://doi.org10.15640/jehd.v4n2a22
  24. Linacre, J.M. (2014). A user's guide to FACETS Rasch-model computer programmes. Retrieved on 13.03.2024 from http://www.winsteps.com/a/facets-manual.pdf.
  25. Linacre, J. M. (2018). A user's guide to Winsteps. Chicago, IL: Winsteps.com
  26. Ministry of National Education General Directorate of Special Education and Guidance Services, Draft Framework Education Programme for Individuals with Special Abilities, (2014). http://orgm.meb.gov.tr/meb_iys_dosyalar/2014_01/27021830_ereveprogramtasla.pdf (Access Date: 12.01.2024).
  27. MEB. (2022). 2022- 2023 science and art centres student identification guide. Retrieved from https://orgm.meb.gov.tr.
  28. Myford, C.M., & Wolfe, E.W. (2003). Detecting and measuring rater effects using many-facet Rasch measurement: Part I. Journal of Applied Measurement, 4(4), 386-422. https://pubmed.ncbi.nlm.nih.gov/14523257/
  29. Myford, C. M., & Wolfe, E. W. (2003). Detecting and measuring rater effects using manyfacet Rasch measurement: Part I. Journal of Applied Measurement, 4(4), 386-422. Retrieved from http://psycnet.apa.org/record/2003-09517-007.
  30. Myford, C.M., & Wolfe, E.W. (2004). Detecting and measuring rater effects using manyfacet Rasch measurement: Part II. Journal of Applied Measurement, 5(2), 189-227. Retrieved from https://europepmc.org/abstract/med/15064538.
  31. Nakamura, N. (2002). Teacher Assessment and Peer Assessment in Practice. Educational Studies. 44, 143, 204-215.
  32. Özyer, K. K. (2018). Scoring open-ended questions in the open education system with the Many Facet Rasch Model analysis method. Journal of Open Education Applications and Research, 4(1), 61-77. https://dergipark.org.tr/en/download/article-file/419501
  33. Sarıtaş, S., 2015. Investigation of Self, Peer and Teacher Scores in the Evaluation of Problem Solving Skills with Many Facet Rasch Measurement Model. (Thesis number: 429376). [Master's Thesis, Gazi University]. Higher Education Institution National Thesis Centre. https://tez.yok.gov.tr/UlusalTezMerkezi/giris.jsp
  34. Schumacker, R. E. (1996). Many-Facet Rasch Model Selection Criteria: Examining Residuals and More. https://files.eric.ed.gov/fulltext/ED397117.pdf
  35. Semerci, Ç. (2012). Evaluation of Students' Opinions on the Department of ITET According to Rasch Measurement Model (Fırat University Example). NWSA: Education Sciences, 7(2), 777-784. https://dergipark.org.tr/tr/pub/nwsaedu/issue/19816/211967
  36. Semerci, Ç., Semerci, N., & Duman, B. (2013). Analysing graduate students' seminar presentation performances with Many FacetRasch model. Sakarya University Journal of Faculty of Education, 25, 7-22. https://dergipark.org.tr/tr/download/article-file/115660
  37. Sudweeks, R. R., Reeve, S., & Bradshaw, W. S. (2004). A comparison of generalisability theory and many-facet Rasch measurement in an analysis of college sophomore writing. Assessing Writing, 9(3), 239-261. https://doi.org/10.1016/j.asw.2004.11.001
  38. Şata, M., (2019). Investigation of the Effect of Rater Training on Rater Behaviours in Performance Evaluation Process (Thesis number: 626117). [Doctoral dissertation, Gazi University]. Higher Education Institution National Thesis Centre. AVAILABLE AT https://tez.yok.gov.tr/UlusalTezMerkezi/giris.jsp
  39. Sata, M., & Karakaya, İ. (2021). Investigating the Effect of Rater Training on Differential Rater Function in Assessing Academic Writing Skills of Higher Education Students. Journal of Measurement and Evaluation in Education and Psychology, 12(2), 163-181. AVAILABLE AT https://doi.org/10.21031/epod.842094
  40. Sata, M. & Karakaya, I. (2022). Investigating the impact of rater training on rater errors in the process of assessing writing skill. International journal of assessment tools in education, 9(2), 492-514. AVAILABLE AT https://doi.org/10.21449/ijate.877035
  41. T.C. Ministry of National Education (MEB), General Directorate of Special Education and Guidance Services, 2022. Education of gifted individuals, strategy and implementation guide. https://orgm.meb.gov.tr/meb_iys_dosyalar/2013_11/25034903_zelyeteneklibireylerineitimistratejiveuygulamaklavuzu.pdf
  42. Tekin D., (2023). Laboratory Activities and Application Performances of Students: Application Status of General Chemistry II Course at University (Thesis number: 813890). [Master's thesis, Çanakkale On Sekiz Mart University]. Higher Education Institution National Thesis Centre. https://tez.yok.gov.tr/UlusalTezMerkezi/giris.jsp
  43. Tobaş C., (2020). Investigation of Differentiating Rater Behaviours in the Evaluation of Performance with Many Facet Rasch Measurement Model (Thesis number: 629923). [Master's thesis, Gazi University]. Higher Education Institution National Thesis Centre. https://tez.yok.gov.tr/UlusalTezMerkezi/giris.jsp
  44. Ulukök, Ş., Çelik, H. & Sarı, U. (2013). The effect of computer-assisted applications related to simple electrical circuits on the development of experimental process skills. Journal of Theoretical Education Science, 6(1), 77-101. http://hdl.handle.net/11630/5434
  45. Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement: Transactions of the Rasch Measurement SIG, 8(3), 370. https://rasch.org/rmt/rmt83.htm
  46. Wu, S. M., & Tan, S. (2016). Managing rater effects through the use of FACETS analysis: the case of a university placement test. Higher Education Research & Development, 35(2), 380-394. https://doi.org/10.1080/07294360.2015.1087381
  47. Yüzüak, A. V., Yüzüak, B., & Kaptan, F. (2015). Analysing the performance task in the direction of peer groups and teacher approaches with Many Facet rasch measurement model. Journal of Measurement and Evaluation in Education and Psychology, 6(1), 1-11. https://doi.org/10.21031/epod.57425