Alternative designs for evaluating computer-based instruction

Five designs which can be used effectively when a randomized two-group design is not practical are presented and a Decision Flowchart is provided for selecting the most appropriate design for the evaluation question and constraints of the situation.

Abstract

As more computer-based materials and methods appear in the educational marketplace, there is an increasing need for studies to validate the effectiveness of these products of technology and to provide more substantial guidelines for future development. However, efforts to study the usefulness of computer-based methods and materials with students are rare in comparison with the opportunities for such studies. Although many reasons have been hypothesized for why more research on the effects of various computer-based methods and materials has not been accomplished, two which seem most likely are: (1) practical difficulties in arranging studies with traditional randomized two-group designs in non-laboratory settings, and (2) lack of available expertise to develop and employ alternative designs and analyze resulting data. This paper: