Dependability of Measurement in Counseling Psychology

Abstract

Classical approaches to the assessment of reliability neglect to take into account multiple sources of error and to consider diverse measurement contexts. Generalizability theory (GT) offers a flexible framework for assessing dependability of measurement. With GT, investigators can estimate the total proportion of variance in ratings that is due to error rather than focusing on one source of error at a time. Simultaneous consideration of multiple sources of error allows investigators to assess the overall impact of measurement error in terms of attenuation of study findings and reduction of statistical power. Estimation of variance components allows for flexible application of findings to a variety of possible future research designs. Illustrative analyses demonstrate the special advantages of GT for planning studies in which observer ratings will be used.