The Importance of Sample Weights and Plausible Values in Large-Scale Assessments

Abstract

International large-scale assessments such as PISA (The Programme for International Student Assessment),PIAAC (The Programme for the International Assessment of Adult Competencies) and TIMSS (Trends inInternational Mathematics Science Study), play a key role in determining educational policies besides theirprimary objectives of measuring, evaluating and monitoring the educational process. Therefore, it is critical toanalyze the data gathered from the large scale assessments using scientifically accurate statistical methods as theresults have the potential to influence millions of stakeholders through major policy changes. Analysis of thesedata that consists of hundreds of different genuine variables requires expertise and using specific methods. Thisstudy illustrates issues to be considered while analyzing PISA, PIAAC and TIMSS data by presenting relevantsyntax and exemplifying the possible incorrect results that might be encountered. In Turkey, there are verylimited courses that focus on large scale data analysis. Workshops are also very limited to reach major groups.The aim of this study is to raise awareness related to sample weights and plausible values. Comparative findingsof the study showed that without using sample weights and plausible values there is a high probability to getincorrect results. In this study, t-test and multiple regression analyses conducted by IDB Analyzer and multilevelregression analysis by Mplus were exemplified.