Changing perspectives on evaluation in HCI

The challenges the community needs to address to develop adequate methods for evaluating systems in modern use contexts are outlined and important research directions that will shape evaluation's future are discussed.

Abstract

Evaluation has been a dominant theme in HCI for decades, but it is far from being a solved problem. As interactive systems and their uses change, the nature of evaluation must change as well. In this paper, we outline the challenges our community needs to address to develop adequate methods for evaluating systems in modern (and future) use contexts. We begin by tracing how evaluation efforts have been shaped by a continuous adaptation to technological and cultural changes and conclude by discussing important research directions that will shape evaluation's future.