Avoiding type III error in program evaluation

Abstract

A common, yet questionable assumption underlying many evaluations of service intervention programs is that program clients uniformly receive the services purportedly available. The authors draw upon the experience of a randomized field experiment to point out the hazards of that assumption. They found marked differences among clients in the amount of actual service received during participation in the program evaluated. Moreover, the data suggest that program outcomes varied as a function of the amount of service received. These findings are offered as a cautionary note to other evaluators; the amount of service actually received by clients should be accurately recorded and incorporated into the analyses of program outcomes.