Reply to the devil's advocates: Don't confound model testing and measurement.

Abstract

Replies by E. F. Alf and N. M. Abrahams and by L. G. Rorer defend the correlation-regression approach to model testing, contending that if a priori measurements are assumed to be proper psychological values and if the correct model is known, correlations can be higher for the better model. But since psychologists cannot know in advance the correct scales and models, popular correlational techniques are inappropriate for investigating psychological processes. It is necessary to separate measurement from the evaluation of a model. A further attempt is made here to clarify the relationships between different methods of analysis. Birnbaum (1973) criticized a currently popular use of correlation that confounds measurement with model testing, demonstrating that a poorer model can achieve higher correlations with the data when a priori measurements are used. Recent replies by Alf and Abrahams (1974) and by Rorer (1974) correctly contend that once the data have been properly diagnosed by other techniques, it may be possible to use regression so that the correlation coefficient is higher for the correct model. But the fundamental question should be: What are the advantages or disadvantages of correlational techniques for exploring psychological theories under conditions where the correct models and psychological values of the stimuli are unknown? Under these conditions, correlations of fit can be misleading since they depend on such factors as (a) unreliability of response, (b) experimental design (which includes variation and covariation of independent variables), (c) stimulus metric, (d) response metric, and (e) number of estimated parameters, as well as (/) the goodness of the model. When correlational analyses are reported, the journal reader has no way of knowing what the original data (and the pattern of devia