Study of the measurement bias of two standardized psychological tests.

Abstract

Psychological tests are subject to two distinct forms of bias. The first form, measurement bias, occurs when individuals with equal standing on the trait measured by the test, but sampled from different subpopulations, have different expected test scores. Relational bias, the second type of bias, exists with respect to a second variable if a measure of bivariate association differs across groups. Empirical studies have found little evidence of relational bias. Two recent court cases, however, seem to have been more influenced by considerations of measurement bias than by the literature concerning relational bias. Unfortunately, a consequence of both court cases is that the respective test makers must select items for future tests on the basis of a statistic (proportion correct) that is inappropriate for evaluating measurement bias. More sophisticated approaches may also suffer from methodological difficulties unless special precautions are taken. In this article, tests of English and Mathematics Usage are analyzed by measurement bias methods in which several steps are taken to reduce methodological artifacts. Many items are found to be biased. Nonetheless, the sizes of these effects are very small and no cumulative bias across items is found.