Evaluation of online catalogues : an assessment of methods

A project investigating tools, techniques and methods for the evaluation of interactive library catalogues is described, and the emphasis was on diagnostic methods, and on the use of the catalogue in the wider context of user information seeking behaviour.

Abstract

A project investigating tools, techniques and methods for the evaluation of interactive library catalogues is described. Methods used in other projects are reviewed. The emphasis of the investigation was on diagnostic methods, and on the use of the catalogue in the wider context of user information seeking behaviour. Studies on users of the City University Library catalogue indicated that (a) traditional transaction logging is quite inadequate as a method of evaluation; (b) various enhancements can be introduced to greatly increase the usefulness of transaction logging; (c) a fuller understanding of the processes involved requires study of the user's behaviour away from the catalogue. Useful enhancements to transaction logging include: a) full-screen logging b) a playback facility c) preor post-search, online or offline questionnaires d) interactive, in-search questionnaires e) talk-aloud. A front-end system (Olive) has been developed to allow the first four enhancements. The experiments also emphasise the extent and importance of subject-searching or hybrid searching activity, as opposed to specific item searches.