Evaluating Information System Effectiveness — Part I: Comparing Evaluation Approaches

Evaluation of system effectiveness in terms of a conceptual hierarchy of system objectives is examined, used to discuss problems in, and recommendations for, evaluating system effectiveness, and to compare MIS evaluation approaches.

Abstract

While the use and costs of Management Information Systems (MIS) have become highly visible, little attention has been paid to assessing and communicating system effectiveness. Evaluation of system effectiveness is difficult due to its multidimensionality, its quantitative and qualitative aspects, and the multiple, and often conflicting, evaluator viewpoints. This article provides an overview of what system effectiveness means and how it should be measured. It is the first of two articles to appear in consecutive issues of the MIS Quarterly. Starting with a definition of system effectiveness, this article examines evaluation of system effectiveness in terms of a conceptual hierarchy of system objectives. The hierarchy is used to discuss problems in, and recommendations for, evaluating system effectiveness, and to compare MIS evaluation approaches. The second article characterizes and compares the evaluator viewpoints on system effectiveness for decision makers in several functional groups involved in MIS implementation — user, MIS, internal audit, and management. The second article recommends several MIS evaluation approaches for incorporating multiple dimensions and multiple evaluator viewpoints into evaluations of information system effectiveness.