The validity of conclusions in evaluation research: A further development of Chen and Rossi's theory-driven approach

The general conclusion is that despite the more demanding requirements of the modelling strategy, it promises a greater yield of policy relevant conclusions than do evaluation designs in the “black box” tradition.

Abstract

The classical concepts of internal and external validity are explicated with causal models to illustrate the requirements of using a theoretically based model for enhancing the internal and external validity of conclusions drawn from program evaluation studies. Causal models are further used to explicate the utility of incorporating measures pertaining to the rationale or theory of a program into the analysis of data. The implications of random and correlated measurement error for drawing conclusions from models of program effects are explored, again using causal models with unmeasured variables. Corrections for measurement error impose more demanding data requirements in the form of multiple indicators for each dimension or construct incorporated into the analysis, but the omission of such corrections for flaws in measures allows for potentially misleading conclusions ostensibly supported by the empirical data. The general conclusion is that despite the more demanding requirements of the modelling strategy, it promises a greater yield of policy relevant conclusions than do evaluation designs in the "black box" tradition.