An Empirical Study for the Statistical Adjustment of Rater Bias Mustafa İlhan

Abstract

This study investigated the effectiveness of statisticaladjustments applied to rater bias in many-facet Rasch analysis. Somechanges were first made in the dataset that did not include rater × examineebias to cause to have rater × examinee bias. Later, bias adjustment wasapplied to rater bias included in the data file, and the effectiveness of thestatistical adjustment was further examined. The outcomes pertaining to thedatasets with and without bias, and to which the bias adjustment wasapplied, were compared. It was concluded that diversities created by rater× examinee bias in examinees’ ability estimation, item difficulty indices andmeasures of rater severity and leniency were, to a large extent, eliminatedby bias adjustment. This result indicates that the bias adjustment usingmany-facet Rasch analysis is a viable way to control rater bias.