KaydedilenlerDerginizi ekleyin

Dergiler / Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi / 2020 / Cilt: 11 - Sayı: 2

Rating Performance among Raters of Different Experience Through Multi-Facet Rasch Measurement (MFRM) Model

Muhamad Firdaus Bin MOHD NOH
Mohd Effendi Ewan Bin MOHD MATORE
Sayfa
147–162
Yayın Tarihi
2022-10-12
DOI
—
🏛️Bu makalenin bibliyografik verileri ve tam metin bağlantısı DergiPark üzerinden sağlanmıştır.
DergiPark Kaydı ↗
✨

Yapay Zeka Hızlı Bakış & Çıkarımlar

AI Insight
⚡ TL;DR(1 Cümlede Hızlı Çıkarım)

Independent t-test indicated that therewas a significant difference in ratings provided by the inexperienced and the experienced raters, t-value = -0.96,df = 28, p

📌 3 Maddede Bu Makale

🎯
Amaç & Hipotez:One’s experience can greatly contribute to a diversified rating performance in educational scoring.
📊
Temel Bulgular & Veri:The aim of the study is to examine raters’ rating performance inassessing oral tests among lower secondary school students using Multi-facet Rasch Measurement (MFRM) modelindicated by raters’ severity.
💡
Sonuç & Katkı:Independent t-test indicated that therewas a significant difference in ratings provided by the inexperienced and the experienced raters, t-value = -0.96,df = 28, p
🔬Tür:Modelleme & Sayısal Simülasyon
📍Kapsam:Genişletilmiş Akademik Örneklem
⚙️Değişkenler:Alan Parametreleri

Abstract

One’s experience can greatly contribute to a diversified rating performance in educational scoring. Heterogeneousratings can negatively affect examinees’ results. The aim of the study is to examine raters’ rating performance inassessing oral tests among lower secondary school students using Multi-facet Rasch Measurement (MFRM) modelindicated by raters’ severity. Respondents are thirty English Language teachers clustered into two groups basedon their rating experience in high-stakes assessment. The respondents listened to ten examinees’ recorded answersof three oral test items and provided their ratings. Instruments include items, examinees’ answers, scoring rubric,and scoring sheet used to appraise examinees’ competence in three domains which are vocabulary, grammar, andcommunicative competence. MFRM analysis showed that raters exhibited diversity in their severity level with chisquareχ2=2.661. Raters’ severity measures ranged from 2.13 to -1.45 logits. Independent t-test indicated that therewas a significant difference in ratings provided by the inexperienced and the experienced raters, t-value = -0.96,df = 28, p

🔖

Bu Makaleye Atıf Yap

NOH, M. F. B. M., MATORE, M. E. E. B. M. (2022). Rating Performance among Raters of Different Experience Through Multi-Facet Rasch Measurement (MFRM) Model. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, Cilt: 11 - Sayı: 2, 147–162. https://ajindex.com/articles/497184

📄 Tam Metin PDF Okuyucu

📄

PDF görüntüleyici hazırlanıyor...

Kaynakça (8)

  1. Ahmadi Shirazi, M. (2019). For a greater good: Bias analysis in writing assessment. SAGE Open, 9(1), 1–14. https://doi.org/10.1177/2158244018822377
  2. Albano, A. D., & Rodrigues, M. (2018). Item development research and practice. Handbook of Accessible Instruction and Testing Practices: Issues, Innovations, and Applications, 181–198. https://doi.org/10.1007/978-3-319-71126-3_12
  3. Alp, P., Epner, A., & Pajupuu, H. (2018). The influence of rater empathy, age and experience on writing performance assessment. Linguistics Beyond And Within, 3(2017), 7–19. Retrieved from https://www.ceeol.com/search/article-detail?id=716601
  4. Anthony, L., & Miriam, S. (2019). Drill in English Skills Practice: CEFR-Alligned Curriculum. Selangor: Oxford Fajar Attali, Y. (2016). A comparison of newly-traind and experienced raters on a standardized writing assessment. Language Testing, 33(1), 99–115. https://doi.org/10.1177/0265532215582283
  5. Bond, T. G., & Fox, C. M. (2015). Applying the Rasch Model Fundamental Measurement in the Human Sciences. New Jersey: Lawrence Erlbaum Associates. Creswell, J. W., & Creswell, J. D. (2018). Research Design: Qualitative, Quantitative, And Mixed Methods Approaches (5th ed.). California: Sage Publications. Davis, L. (2016). The influence of training and experience on rater performance in scoring spoken language. Language Testing, 33(1), 117–135. https://doi.org/10.1177/0265532215582282
  6. Eckes, T. (2015). Introduction to Many-Facet Rasch Measurement: Analyzing and Evaluating Rater-Mediated Assessments (2nd ed.). Peter Lang. Eckes, T. (2019). Implications for rater-mediated language assessment. In Aryadoust, V., & Raquel, M. (Eds.), Quantitative Data Analysis for Language Assessment Volume I: Fundamental Techniques (pp. 153-175). London & New York: Routledge. Engelhard, G., & Wind, S. A. (2018). Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments. Routledge. New York & London: Routledge. https://doi.org/10.1017/CBO9781107415324.004
  7. Fisher, J. W. P. 2007. Rating scale instrument quality criteria. Rasch Measurement Transactions 21(1): 1095. Govindasamy, P., Salazar, M. D. C., Lerner, J., & Green, K. E. (2019). Assessing the reliability of the framework for equitable and effective teaching with the many-facet rasch model. Frontiers in Psychology, 10(June), 1–10. https://doi.org/10.3389/fpsyg.2019.01363
  8. Haladyna, T. M., & Rodrigues, M. C. (2013). Developing and Validating Test. New York: Routledge. Huang, L., Kubelec,