A comparison of the four prominent user-based methods for evaluating the usability of computer software

Four evaluation methods were used to evaluate three different business software types using a between-groups design, involving 148 individuals of mixed age and gender, and the verbal protocol analysis was found to be most efficient.

Abstract

Abstract The aim was to examine four prominent user-based computer software usability evaluation methods. Four evaluation methods (logged data, questionnaire, inter view, and verbal protocol analysis) were used to evaluate three different business software types (spreadsheet, word processor, and database) using a between-groups design, involving 148 individuals of mixed age and gender. Comparisons were made to examine the efficiency of each evaluation method in terms of its ability to highlight usability problems both between and within the evaluation strategy. Here, the verbal protocol analysis was found to be most efficient. The possibility of further efficiency gains by using two evaluation methods was also examined, where it was found that no statistically significant improvement was obtained over the verbal protocol analysis used by itself. Ways in which the utility of the methods may be enhanced was also discussed. Keywords: Usability evaluation. Notes Present address and author for correspondence: Centre for Applied Psychology, University of Canberra, PO Box 1 Belconnen, ACT 2616, Australia. §Now at the Department of Psychology, Victoria University, New Zealand.