An Evaluation of Pass/Fail Decisions through Norm- and CriterionReferenced Assessments

Abstract

The institutions in education use various assessment methods to decideon the proficiency levels of students in a particular construct. This studyinvestigated whether the decisions differed based on the type of assessment: normand criterion-referenced assessment. An achievement test with 20 multiple-choiceitems was administered to 107 students in guidance and psychological counselingdepartment to assess their mastery in the course of measurement and evaluation.First, the raw scores were transformed into T-scores for the decisions from normreferenced assessments. Two decisions were made to classify students aspassed/failed comparing each student’s T-score with two common cutoffs ineducation: 50 and 60. Second, two standard-setting methods (i.e., Angoff andNedelsky) were conducted to get two cut scores for the criterion-referencedassessment with the help of experts in measurement and evaluation. Two moredecisions were made on the classification of students into pass/fail group bycomparing the raw scores and the cut scores from two standard-setting methods.The proportions of students in pass/fail categories were found to be statisticallydifferent across each pair of four decisions from norm- and criterion-referencedassessments. Cohen’s Kappa showed that the decisions based on Nedelsky methodindicated a moderate agreement with the pass/fail decisions from the students’semester scores in measurement and evaluation while the other three decisionsshowed a lower agreement. Therefore, the criterion-referenced assessment withNedelsky method might be considered in making pass/fail decisions in educationdue to its criterion validity from the agreement with the semester scores.