BEYOND FORMAL EXPERIMENTAL DESIGN: TOWARDS AN EXPANDED VIEW OF THE TRAINING EVALUATION PROCESS

Abstract

Textbook treatments of training evaluation typically equate evaluation with the measurement of change and focus on formal experimental design as the mechanism for controlling threats to the inference that the training intervention produced whatever change was observed. This paper notes that two separate questions may be of interest: How much change has occurred? and, Has a target performance level been reached? We show that the evaluation mechanisms needed to answer the two types of questions are markedly different, and discuss circumstances under which an evaluator's interests will focus on one, the other, or both of these questions. We then discuss alternatives to formal design as mechanisms for reducing various threats to validity, and discuss trade‐offs between internal validity and statistical conclusion validity.