SavedAdd your journal

Journals / Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi / 2026 / Volume: 14 Issue: 3

Analysis of Peer and Self-Assessments Using the Many-facet Rasch Measurement Model and Student Opinions

Seda Demir(TOKAT GAZİOSMANPAŞA ÜNİVERSİTESİ)0000-0003-4230-5593
Pages
—
Publication Date
2023-09-30
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

According to the results of the study, it was determined that there was a statistically significant difference between the students in terms of their oral presentation skills, between the assessors in terms of their strictness/generosity in scoring, and between the criteria (items) in terms of the level of difficulty in realization.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:The aim of this study is to analyze the peer and self-assessments of higher education students' oral presentation skills with the many-facet Rasch measurement model and to determine students' opinions on peer and self-assessment.
📊
Key Findings & Data:The study group consisted of 11 university students studying at a state university in the 2022-2023 academic year.
💡
Conclusion & Impact:According to the results of the study, it was determined that there was a statistically significant difference between the students in terms of their oral presentation skills, between the assessors in terms of their strictness/generosity in scoring, and between the criteria (items) in terms of the level of difficulty in realization.
🔬Design:Modeling & Simulation
📍Scope:2022-2023 Verileri
⚙️Variables:Peer assessment · Self-assessment · Many-facet Rasch measurement model

Abstract

The aim of this study is to analyze the peer and self-assessments of higher education students' oral presentation skills with the many-facet Rasch measurement model and to determine students' opinions on peer and self-assessment. In the study, convergent parallel method, one of the mixed-method research approaches, was used. The study group consisted of 11 university students studying at a state university in the 2022-2023 academic year. The FACETS program was used to analyze the data. The three facets identified in the study were the assessee (11 students), the assessor (11 students) and the items (16 items). Therefore, 11 participants scored (peer and self-assessment) on a 16-item assessment form. In addition, students' opinions on peer and self-assessment were obtained through three open-ended interview questions prepared by the researcher. According to the results of the study, it was determined that there was a statistically significant difference between the students in terms of their oral presentation skills, between the assessors in terms of their strictness/generosity in scoring, and between the criteria (items) in terms of the level of difficulty in realization. In addition, the participant opinions obtained from each interview question were analyzed through themes and sub-themes formed according to the general thoughts on peer and self-assessment, experiences, and whether the participants considered themselves as a reliable rater or not. In terms of practice, it can be suggested to provide detailed and enlightening information to students before peer and/or self-assessment in the classroom environment, and to give quick feedback to those who have not done the assessment appropriately. In addition, the reasons for the biases identified in peer and self-assessments in the current study can be investigated in future studies.

🔖

Cite This Article

Demir, S. (2023). Analysis of Peer and Self-Assessments Using the Many-facet Rasch Measurement Model and Student Opinions. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, Volume: 14 Issue: 3. https://doi.org/10.21031/epod.1344196

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (54)

  1. Akın, Ö. ve Baştürk, R. (2012). The evaluation of the basic skills in violin training by many facet Rasch model. Pamukkale University Journal of Education, 31(1), 175-187. https://dergipark.org.tr/en/pub/pauefd/issue/11112/132860
  2. Atılgan, H. (2005). Analysis of Special Ability sellection examination for music education department using many-facets Racsh measurement (İnönü University case). Eurasian Journal of Educational Measurement, 0(20), 62-73. https://web.s.ebscohost.com/ehost/pdfviewer/pdfviewer?vid=0&sid=1c78c0f3-93e0-43b5-9499-026878d2fcb1%40redis
  3. Baştürk, R. (2008). Applying the many facet Rasch model to evaluate PowerPoint presentation performance in higher education. Assessment and Evaluation in Higher Education, 33(4), 431-444. https://doi.org/10.1080/02602930701562775
  4. Baştürk, R. (2010). Bilimsel araştırma ödevlerinin çok yüzeyli Rasch ölçme modeli ile değerlendirilmesi. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 1(1), 51-57. https://dergipark.org.tr/en/pub/epod/issue/5808/77254
  5. Boud, D., & Falchikov, N. (1989). Quantitative studies of student self-assessment in higher education: a critical analysis of findings. Higher Education, 18(5), 529-549. https://doi.org/10.1007/BF00138746
  6. Brown, S. (2004). Assessment for learning. Learning and Teaching in Higher Education, 5(1), 81-89. https://eprints.glos.ac.uk/3607/
  7. Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65(3), 245-281. https://doi.org/10.3102/00346543065003245
  8. Cheong, C. M., Luo, N., Zhu, X., Lu, Q., & Wei, W. (2023). Self-assessment complements peer assessment for undergraduate students in an academic writing task. Assessment & Evaluation in Higher Education, 48(1), 135-148. https://doi.org/10.1080/02602938.2022.2069225
  9. Creswell, J. W. (2014). A concise introduction to mixed methods research. SAGE.
  10. Crisp, V., & Ward, C. (2008). The development of a formative scenario-based computer assisted assessment tool in psychology for teachers: The PePCAA project. Computers & Education, 50(4), 1509-1526. https://doi.org/10.1016/j.compedu.2007.02.004
  11. Eckes, T. (2005). Examining rater effects in TestDaF writing and speaking performance assessments: A many-facet Rasch analysis. Language Assessment Quarterly: An International Journal, 2(3), 197-221. https://doi.org/10.1207/s15434311laq0203_2
  12. Elhan, A. H., & Atakurt, Y. (2005). Why is it necessary to use Rasch analysis when evaluating measures? Ankara Üniversitesi Tıp Fakültesi Mecmuası, 58(1), 47-50. https://doi.org/10.1501/Tipfak_0000000134
  13. Evans, A. T., Mcnutt, R. A., Fletcher, S. W., & Fletcher, R. H. (1993). The characteristics of peer reviewers who produce good-quality reviews. Journal of General Internal Medicine, 8(8), 422-428. https://doi.org/10.1007/bf02599618
  14. Falchikov, N., & Goldfinch, J. (2000). Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks. Review of Educational Research, 70(3), 287-322. https://doi.org/10.3102/00346543070003287
  15. Fete, M. G., Haight, R. C., Clapp, P., & McCollum, M. (2017). Peer evaluation instrument development, administration, and assessment in a team-based learning curriculum. American Journal of Pharmaceutical Education, 81(4), 1-10. https://doi.org/10.5688/ajpe81468
  16. Gu, C. (2020). Student Peer Assessment. Review of Educational Theory, 3(2), 74-78. https://doi.org/10.30564/ret.v3i2.1762
  17. Güler, N. (2014). Analysis of open-ended statistics questions with many facet Rasch model. Eurasian Journal of Educational Research, 55, 73-90. http://dx.doi.org/10.14689/ejer.2014.55.5
  18. Gürlen, E., Boztunç Öztürk, N. & Eminoğlu, E. (2019). Investigation of the reliability of teachers, self and peer assessments at primary school level with Generalizability Theory. Journal of Measurement and Evaluation in Education and Psychology, 10(4), 406-421. http://dx.doi.org/10.21031/epod.583891
  19. Hanrahan, S. J., & Isaacs, G. (2001). Assessing self- and peer-assessment: The students' views. Higher Education Research & Development, 20(1), 53-70. https://doi.org/10.1080/07294360123776
  20. Hoo, H. T., Deneen, C., & Boud, D. (2022). Developing student feedback literacy through self and peer assessment interventions. Assessment & Evaluation in Higher Education, 47(3), 444-457. https://doi.org/10.1080/02602938.2021.1925871
  21. Köse, İ. A., Usta, H. G., & Yandı A. (2016). Evaluation of presentation skills by using many facets Rasch model. Bolu Abant İzzet Baysal University Journal of Faculty of Education, 16(4), 1853-1864. https://dergipark.org.tr/en/pub/aibuefd/issue/28550/304600
  22. Linacre, J. M. (1993). Rasch-based generalizability theory. Rasch Measurement Transactions, 7(1), 283-284. https://www.rasch.org/rmt/rmt71h.htm
  23. Linacre, J. M. (1994). Many-facet rasch model (2nd ed.). Mesa Press.
  24. Linacre, J. M. (2023). A user’s guide to FACETS: Rasch-model computer programs (Program manual 3.85. 1). Chicago, IL. https://www.winsteps.com/a/Facets-Manual.pdf
  25. Liu, N. F., and D. Carless. 2006. “Peer feedback: The learning element of peer assessment.” Teaching in Higher Education 11(3), 279-290. https://doi.org/10.1080/13562510600680582
  26. Liu, E. Z.-F., Lin, S. S. J., Chiu, C.-H., & Yuan, S.-M. (2001). Web-based peer review: the learner as both adapter and reviewer. IEEE Transactions on Education, 44(3), 246-251. https://doi.org/10.1109/13.940995
  27. Lu, J., & Law, N. (2011). Online peer assessment: effects of cognitive and affective feedback. Instructional Science, 40(2), 257-275. https://doi.org/10.1007/s11251-011-9177-2
  28. Lundstrom, K., and W. Baker. 2009. “To give is better than to receive: The benefits of peer review to the reviewer’s own writing.” Journal of Second Language Writing 18(1), 30-43. https://doi.org/10.1016/j.jslw.2008.06.002
  29. Mulqueen C., Baker D. & Dismukes R.K. (2000) Using multifacet Rasch analysis to examine the effectiveness of rater training. SIOP.
  30. Mumpuni, K. E., Priyayi, D. F., & Widoretno, S. (2022). How do students perform a peer assessment? International Journal of Instruction, 15(3), 751-766. https://doi.org/10.29333/iji.2022.15341a
  31. Nakamura, Y. (2000). Many-facet Rasch based analysis of communicative language testing results. Journal of Communication Students, 12, 3-13. https://eric.ed.gov/?id=ED449678
  32. Nakamura, N. (2002). Teacher assessment and peer assessment in practice. Educational Studies, 44, 204-215. https://files.eric.ed.gov/fulltext/ED464483.pdf
  33. Nicol, D. J., & Macfarlane‐Dick, D. (2006). Formative assessment and self‐regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218. https://doi.org/10.1080/03075070600572090
  34. Orluwene, G. W., & Ekim, D. K. (2020). Promoting self-regulated learning through self-and peer-assessment techniques among secondary school students. International Journal of Arts and Commerce, 9(4), 1-16. https://ijac.org.uk/articles/9.4.1.1-16.pdf
  35. Panadero, E., Alqassab, M., Fernández Ruiz, J., & Ocampo, J. C. (2023). A systematic review on peer assessment: intrapersonal and interpersonal factors. Assessment & Evaluation in Higher Education, 1-23. https://doi.org/10.1080/02602938.2023.2164884
  36. Pantiwati, Y., & Husamah. (2017). Self and peer assessments in active learning model to increase metacognitive awareness and cognitive abilities. International Journal of Instruction, 10(4), 185-202. https://doi.org/10.12973/iji.2017.10411a
  37. Prieto, G., & Nieto, E. (2014). Analysis of rater severity on written expression exam using many faceted Rasch measurement. Psicologica, 35(2), 385-397. https://www.redalyc.org/pdf/169/16931314011.pdf
  38. Randall, J. & Engelhard, G. Jr. (2009). Examining teacher grades using Rasch measurement theory. Journal of Educational Measurement, 46(1), 1-18. https://doi.org/10.1111/j.1745-3984.2009.01066.x
  39. Sande, J. C. G., & Godino-Llorente, J. I. (2014). Peer assessment and self-assessment: Effective learning tools in higher education. International Journal of Engineering Education, 30(3), 711-721. https://oa.upm.es/35804/
  40. Semerci, Ç. (2011a). Doktora yeterlikler çerçevesinde öğretim üyesi, akran ve öz değerlendirmelerin Rasch ölçme modeliyle analizi. Journal of Measurement and Evaluation in Education and Psychology, 2(2), 164-171. https://dergipark.org.tr/en/pub/epod/issue/5804/77226
  41. Semerci, Ç. (2011b). Analyzing microteaching applications with many-facet Rasch measurement model. Education and Science, 36 (161), 14-25. http://egitimvebilim.ted.org.tr/index.php/EB/article/view/145
  42. Semerci, Ç., Semerci, N., & Duman, B. (2013). Analysis of seminar presentation performances of postgraduate students with many-facet Rasch model. The Journal of Sakarya Unıversity Education Faculty, 0(25), 7-22. https://dergipark.org.tr/tr/pub/sakaefd/issue/11221/133978
  43. Stefani, L. A. J. (1994). Peer, self and tutor assessment: Relative reliabilities. Studies in Higher Education, 19(1), 69-75. https://doi.org/10.1080/03075079412331382153
  44. Sudweeks, R. R., Reeve, S., & Bradshaw, W. S. (2004). A comparison of generalizability theory and many-facet Rasch measurement in an analysis of college sophomore writing. Assessing Writing, 9(3), 239-261. https://doi.org/10.1016/j.asw.2004.11.001
  45. Topping, K. J., Smith, E. F., Swanson, I., & Elliot, A. (2000). Formative Peer Assessment of Academic Writing Between Postgraduate Students. Assessment & Evaluation in Higher Education, 25(2), 149-169. https://doi.org/10.1080/713611428
  46. Tseng, S.-C., & Tsai, C.-C. (2007). On-line peer assessment and the role of the peer feedback: A study of high school computer course. Computers & Education, 49(4), 1161-1174. https://doi.org/10.1016/j.compedu.2006.01.007
  47. Turner, J. (2003). Examining on art portfolio assessment using a many facet Rasch measurement model [Unpublished dissertation]. Boston College.
  48. Uyanık, G. K., Güler, N., Teker, G. T., & Demir, S. (2019). The analysis of elementary science education course activities through many-facet Rasch model. Kastamonu Education Journal, 27(1), 139-150. https://doi.org/10.24106/kefdergi.2417
  49. Wright, B.D., & Linacre, J.M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370-382. https://www.rasch.org/rmt/rmt83b.htm
  50. Yan, Z., & Brown, G. T. (2017). A cyclical self-assessment process: Towards a model of how students engage in self-assessment. Assessment & Evaluation in Higher Education, 42(8), 1247-1262. https://doi.org/10.1080/02602938.2016.1260091
  51. Yan, Z., Wang, X., Boud, D., & Lao, H. (2023). The effect of self-assessment on academic performance and the role of explicitness: a meta-analysis. Assessment & Evaluation in Higher Education, 48(1), 1-15. https://doi.org/10.1080/02602938.2021.2012644
  52. Yüzüak, A. V., Yüzüak, B., & Kaptan, F. (2015). A many-facet Rasch measurement approach to analyze peer and teacher assessment for authentic assessment task. Journal of Measurement and Evaluation in Education and Psychology, 6(1), 1-11. https://doi.org/10.21031/epod.57425
  53. Zhan, Y., Yan, Z., Wan, Z. H., Wang, X., Zeng, Y., Yang, M., & Yang, L. (2023). Effects of online peer assessment on higher‐order thinking: A meta‐analysis. British Journal of Educational Technology, 54, 817-835. https://doi.org/10.1111/bjet.13310
  54. Gürlen, E., Boztunç Öztürk, N. & Eminoğlu, E. (2019). Investigation of the reliability of teachers, self and peer assessments at primary school level with Generalizability Theory. Journal of Measurement and Evaluation in Education and Psychology, 10(4), 406-421. http://dx.doi.org/10.21031/epod.583891