Using Alignment Index and Polytomous Item Response Theory on Statistics Essay Test

Abstract

Purpose: Essay test in mathematics, both in the form ofrestricted-response and extended-response, generallyconsist of polytomous scored items. However, the essay test used by teachers in Indonesia has not been fully supported by sufficient quality evidence. There have been many studies focusing on the development of the essay test, but not many of them have applied the use of relevant measurement theory for the polytomous data.The evidence of content validity also has not beensupported by its alignment with the curriculum. Thisstudy used alignment index toprove the content validity and IRT polytomous GPCM to determine the characteristics of test itemsin order to produce an essay test that could accurately measure the achievement of students onstatistical materials.Method: Procedures of this study: (1) preparation of preliminary test, (2) trials, (3) interpretation.Trial was conducted involving 688 Junior High School students in Yogyakarta, Indonesia.Results: The content validity of the test was good, supported by V Aiken index of 0.88–1.00 andPorter alignment index of 0.93. The test items had good construct validity. Test reliability wascategorized as good with the Construct Reliability coefficient of 0.88 and the Alpha coefficient of0.78. Judging from its characteristics, all test items were categorized as good.Implications for Research and Practice: The use of the alignment index contribution to theverification of content validity of essay test and the use of the IRT polytomous GPCM may providereference for the use of appropriate measurement theory to determine the item characteristics ofessay test.