A structured process for transforming usability data into usability information
Journal of Usability Studies archivePublished 1 November 2007
Jonathan R. Howarth, Terence S. Andre, Rex Hartson
Citations30
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This article presents a structured process for transforming raw usability data into usability information that is based on a new way of thinking about usability problem data and indicates that it helps improve the effectiveness of novice usability practitioners.
Abstract
Much research has been devoted to developing usability evaluation methods that are used in evaluating interaction designs. More recently, however, research has shifted away from evaluation methods and comparisons of evaluation methods to issues of how to use the raw usability data generated by these methods. Associated with this focus is the assumption that the transformation of the raw usability data into usability information is relatively straightforward. We would argue that this assumption is incorrect, especially for novice usability practitioners. In this article, we present a structured process for transforming raw usability data into usability information that is based on a new way of thinking about usability problem data. The results of a study of this structured process indicate that it helps improve the effectiveness of novice usability practitioners.
Keywords
Computer ScienceSocial Sciences
Finding usability problems through heuristic evaluation
1,042 Citations1992Jakob Nielsen
Usability specialists were better than non-specialists at performing heuristic evaluation, and “double experts” with specific expertise in the kind of interface being evaluated performed even better.
International Journal of Man-Machine StudiesCognitive walkthroughs: a method for theory-based evaluation of user interfaces
805 Citations1992Peter G. Polson, Clayton Lewis +2 more
The cognitive walkthrough methodology, described in detail, is an adaptation of the design walkthrough techniques that have been used for many years in the software engineering community and is based on a theory of learning by exploration presented.
John Wiley & Sons, Inc. eBooksDeveloping user interfaces: ensuring usability through product & process
795 Citations1993Deborah Hix, H. Rex Hartson
Ensuring Usability in Human-Computer Interaction, a Handbook of Iterative, Evaluation-Centered User Interaction Development, and Techniques for Representing user Interaction Designs.
interactionsEvaluating usability methods
182 Citations2003Dennis Wixon
Dennis Wixon argues that consideration of the factors that determine success of usability efforts in product development organizations will fundamentally change the terms of the debate about the relative effectiveness of different methodologies.
Comparing usability problems and redesign proposals as input to practical systems development
148 Citations2005Kasper Hornbæk, Erik Frøkjær
A study of how developers of a large web application assess usability problems and redesign proposals as input to their systems development concludes that redesign proposals should be developed and used as an integral part of usability evaluation.
The evaluator effect in usability tests
145 Citations1998Niels Ebbe Jacobsen, Morten Hertzum +1 more
In this study, four evaluators analyzed four videotaped usability test sessions and found that the evaluator effect had little effect on the reliability of usability tests.
Behaviour and Information TechnologyComparison of evaluation methods using structured usability problem reports
138 Citations1997Darryn Lavery, Gilbert Cockton +1 more
It is shown that the matching of predicted and actual problems requires careful attention, and that current approaches lack rigour or generality, and a solution is presented: a new report structure for usability problems.
Remote usability evaluation
91 Citations1998José Carlos Castillo, H. Rex Hartson +1 more
The findings indicate that users can, in fact, identify and report their own critical incidents, and this method for remote usability evaluation.
Iterative methodology and designer training in human-computer interface design
61 Citations1993Gregg Bailey
The results of this study indicate that iterative design methodology can improve the usability of a product and also supports the use of human factors specialists in user interface design.
International Journal of Human-Computer StudiesSupporting novice usability practitioners with usability engineering tools
57 Citations2009Jonathan R. Howarth, Tonya L. Smith‐Jackson +1 more
This work introduces a tool feature, usability problem instance records, to better support novice usability practitioners and describes the results of a study of this feature, which suggest that the feature helps to improve two aspects of the effectiveness of novice usability practitioner: reliability and quality.
International Journal of Human-Computer InteractionThe Impact of Usability Reports and User Test Observations on Developers' Understanding of Usability Data: An Exploratory Study
48 Citations2006Rune Thaarup Høegh, Christian M. Nielsen +3 more
Results from an exploratory study of 2 ways of providing feedback from a usability evaluation: observation of user tests and reading usability reports indicate that observation ofuser tests facilitated a rich understanding of usability problems and created empathy with the users and their work.
Towards the design of effective formative test reports
48 Citations2005Mary Theofanos, Whitney Quesenbery
This paper defines elements in these reports and presents some early guidelines on making design decisions for a formative report, based on considerations of the business context, the relationship between author and audience, the questions that the evaluation is trying to answer, and the techniques used in the evaluation.
International Journal of Human-Computer InteractionEvaluating the Downstream Utility of User Tests and Examining the Developer Effect: A Case Study
43 Citations2006Effie Lai‐Chong Law
A case study on evaluating the downstream utility of user tests performed on a digital library with reference to the developer effect, where developer effect is defined as developers' bias toward fixing UPs with particular characteristics.
Reconditioned merchandise
37 Citations2004Gilbert Cockton, Alan Woolrych +1 more
Two versions of an Extended Structured Report Format for usability problems are reported on, briefly noting their impact on analyst behaviour, but more extensively presenting insights into decision making during usability inspection, thus validating and refining a model of evaluation performance.
Journal of Usability Studies archivePost-modern usability
22 Citations2006Arnold M. Lund
A few years ago, I was talking to my boss about the iterative, user-centered design plan the authors had laid out for their products, and he leaned toward me, took another bite of his sandwich, and said in all seriousness, "Don't people in your field actually know anything?"
Proceedings of the Human Factors and Ergonomics Society Annual MeetingComparing Usability Problem Identification and Description by Practitioners and Students
10 Citations2007Miranda G. Capra
There was no difference in the number of problems reported by students and practitioners, but there was a difference in their ratings for following several of the guidelines, which provides a more complete assessment of usability reports.
Lecture notes in computer scienceFeedback from Usability Evaluation to User Interface Design: Are Usability Reports Any Good?
8 Citations2005Christian M. Nielsen, Michael Overgaard +2 more
The findings indicate that detailed descriptions of problems and log descriptions of the user’s interaction with the system and of system interaction are useful for the designers when trying to understand the usability problems that the users have encountered.