Analytics in International Large-Scale Assessments: Item Response Theory and Population Models

Abstract

International large-scale assessments (ILSAs) report results in terms of estimated average scale score in a subject area for all examinees in the target population in participating countries, and also as the percentage of examinees who attain predefined levels of proficiency. In addition, ILSA results include estimates for subgroups within countries based on characteristicsCONTENTSIntroduction ......................................................................................................... 155 Latent Regression Item Response Theory ....................................................... 157Item Response Theory ................................................................................... 157 Population Modeling Using Latent Regressions ....................................... 160 Current Estimation Approaches and the MGROUP Programs ............... 163 Estimating Group Results Using Plausible Values .................................... 165 Variance Estimation ....................................................................................... 166Variance due to Latency of the Proficiency θv ....................................... 166 Variance due to Sampling of Examinees ................................................ 167Current Research on Alternative Approaches ................................................ 167 Application of an Importance Sampling EM Method .............................. 167 Multilevel IRT Using Markov Chain Monte Carlo Methods ................... 168 Stochastic Approximation ............................................................................. 169 Other Approaches .......................................................................................... 170Outlook ................................................................................................................. 170 References ............................................................................................................. 171such as gender, race/ethnicity, and school type. ILSAs do not report scores for individual students, however, and are typically of no consequence to the students participating in these assessments.