Dergiler / International Journal of Assessment Tools in Education / 2019 / Cilt: 6 - Sayı: 2

An Empirical Study for the Statistical Adjustment of Rater Bias Mustafa İlhan

Sayfa
193–201
Yayın Tarihi
2022-10-18
DOI
—
🏛️Bu makalenin bibliyografik verileri ve tam metin bağlantısı DergiPark üzerinden sağlanmıştır.
DergiPark Kaydı ↗
✨

Yapay Zeka Hızlı Bakış & Çıkarımlar

AI Insight
⚡ TL;DR(1 Cümlede Hızlı Çıkarım)

This result indicates that the bias adjustment usingmany-facet Rasch analysis is a viable way to control rater bias.

📌 3 Maddede Bu Makale

🎯
Amaç & Hipotez:This study investigated the effectiveness of statisticaladjustments applied to rater bias in many-facet Rasch analysis.
📊
Temel Bulgular & Veri:Somechanges were first made in the dataset that did not include rater × examineebias to cause to have rater × examinee bias.
💡
Sonuç & Katkı:This result indicates that the bias adjustment usingmany-facet Rasch analysis is a viable way to control rater bias.
🔬Tür:Araştırma Makalesi
📍Kapsam:Genişletilmiş Akademik Örneklem
⚙️Değişkenler:Alan Parametreleri

Abstract

This study investigated the effectiveness of statisticaladjustments applied to rater bias in many-facet Rasch analysis. Somechanges were first made in the dataset that did not include rater × examineebias to cause to have rater × examinee bias. Later, bias adjustment wasapplied to rater bias included in the data file, and the effectiveness of thestatistical adjustment was further examined. The outcomes pertaining to thedatasets with and without bias, and to which the bias adjustment wasapplied, were compared. It was concluded that diversities created by rater× examinee bias in examinees’ ability estimation, item difficulty indices andmeasures of rater severity and leniency were, to a large extent, eliminatedby bias adjustment. This result indicates that the bias adjustment usingmany-facet Rasch analysis is a viable way to control rater bias.

🔖

Bu Makaleye Atıf Yap

İlhan, M. (2022). An Empirical Study for the Statistical Adjustment of Rater Bias Mustafa İlhan. International Journal of Assessment Tools in Education, Cilt: 6 - Sayı: 2, 193–201. https://ajindex.com/articles/540159

📄 Tam Metin PDF Okuyucu

📄

PDF görüntüleyici hazırlanıyor...

Kaynakça (9)

  1. Aubin, A. S., St-Onge, C., & Renaud, J. S. (2018). Detecting rater bias using a person-fit statistic: A Monte Carlo simulation study. Perspectives on Medical Education, 7(2), 83- 92. http://dx.doi.org/10.1007/s40037-017-0391-8
  2. Bailey, K. (1994). Methods of social research. New York: The Free. Bennett, R. E. (1991). On the meanings of constructed response. ETS Research Report Series, 2, 1-46. http://dx.doi.org/10.1002/j.2333-8504.1991.tb01429.x
  3. Bennett, R. E., Ward, W. C., Rock, D. A., & LaHart, C. (1990). Toward a framework for constructed response items. ETS Research Report Series, 1, 1 - 29. http://dx.doi.org/10. 1002/j.2333-8504.1990.tb01348.x Connaway, L. S., & Powell, R. R. (2010). Basic research methods for librarians. Santa Barbara, CA: Libraries Unlimited. DeMars, C. (2010).
  4. Item response theory. Oxford, UK: Oxford University. Eckes, T. (2005). Examining rater effects in TestDaF writing and speaking performance assessments: A many-facet Rasch analysis. Language Assessment Quarterly, 2(3), 197- 221. http://dx.doi.org/10.1207/s15434311laq0203_2
  5. Fahim, M., & Bijani, H. (2011). The effects of rater training on raters’ severity and bias in second language writing assessment. Iranian Journal of Language Testing, 1(1), 1-16. Retrieved from http://www.ijlt.ir/portal/files/401-2011-01-01.pdf
  6. Güler, N., İlhan, M., Güneyli, A., & Demir, S. (2017). An evaluation of the psychometric properties of three different forms of Daly and Miller’s writing apprehension test through Rasch analysis. Educational Sciences: Theory & Practice, 17(3), 721-744. http://dx.doi.org/10.12738/estp.2017.3.0051
  7. Haiyang, S. (2010). An application of classical test theory and many facet Rasch measurement in analyzing the reliability of an English test for non-English major graduates. Chinese Journal of Applied Linguistics, 33(2), 87 - 102. Retrieved from http://www.celea.org.cn/ teic/90/10060807.pdf Haladyana, T. M. (1997). Writing test items to evaluate higher order thinking. Needham Heights, MA: Allyn & Bacon. Hogan, T. P., & Murphy, G. (2007) Recommendations for preparing and scoring constructedresponse items: What the experts say. Applied Measurement in Education, 20(4), 427- 441. http://dx.doi.org/10.1080/08957340701580736
  8. Houston, W. M., Raymond, M.R., & Svec, J. C. (1991). Adjustments for rater effects in performance assessment. Applied Psychological Measurement, 15(4), 409-421. http://dx.doi.org/10.1177/014662169101500411
  9. Hoyt, W. T. (2000). Rater bias in psychological research: