8. A New Approach to Evaluating the Quality of Measurement Instruments: The Split-Ballot MTMM Design

Abstract

Two distinctly different quantitative approaches are used to evaluate measurement instruments: the split-ballot experiment and the multitrait-multimethod (MTMM) approach. The first approach is typically used to indicate whether variation in the method causes differences in the response distribution; the second approach evaluates the reliability and validity of different methods. The new approach, suggested in this paper, combines the more attractive features of both methods. The strength of the split-ballot experiment is its use of independent random samples from the same population to provide information about differences in response distributions. This is also possible with the new approach, but this approach provides more detailed information about the reasons