Validity Coefficients and Correlated Errors in Test Theory

Abstract

AbstractSimple formulas for test validity in predictor-criterion test situations are derived from the expected-value concept of true score without assuming that error scores on distinct tests are uncorrelated. The formulas apply when scores are not experimentally independent and reduce to the usual results of the classical model if the correlation between error scores is zero. These more general formulas reveal some paradoxical properties of test validity under conditions where errors are correlated and have some implications for the interpretation of validity coefficients in practical testing situations. Under some conditions there is no attenuation of the correlation between test scores produced by error of measurement, and in some cases the correlation between observed scores can become higher than the correlation between true scores. Under other conditions where errors are correlated, even if only to a small degree, increasing test length can decrease test validity. A redefined validity coefficient that avoids some of these paradoxes is suggested.