Extracting usability and user experience information from online user reviews
Published 27 April 2013
Steffen Hedegaard, Jakob Grue Simonsen
Citations127
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The results suggest that a greater understanding of users' preoccupation with different dimensions of usability and UX may be inferred from the large volume of self-reported experiences online, and that research focused on identifying pertinent dimensions of UX may benefit further from empirical studies of user-generated experience reports.
Abstract
Internet review sites allow consumers to write detailed reviews of products potentially containing information related to user experience (UX) and usability. Using 5198 sentences from 3492 online reviews of software and video games, we investigate the content of online reviews with the aims of (i) charting the distribution of information in reviews among different dimensions of usability and UX, and (ii) extracting an associated vocabulary for each dimension using techniques from natural language processing and machine learning. We (a) find that 13%-49% of sentences in our online reviews pool contain usability or UX information; (b) chart the distribution of four sets of dimensions of usability and UX across reviews from two product categories; (c) extract a catalogue of important word stems for a number of dimensions. Our results suggest that a greater understanding of users' preoccupation with different dimensions of usability and UX may be inferred from the large volume of self-reported experiences online, and that research focused on identifying pertinent dimensions of usability and UX may benefit further from empirical studies of user-generated experience reports.
Keywords
Computer ScienceSocial Sciences
Machine LearningSupport-Vector Networks
33,035 Citations1995Corinna Cortes, Vladimir Vapnik
High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
American PsychologistSelf-determination theory and the facilitation of intrinsic motivation, social development, and well-being.
27,592 Citations2000Richard M. Ryan, Edward L. Deci
Journal of the American Statistical AssociationContent Analysis: An Introduction to its Methodology.
24,566 Citations1984Mack Shelley, Klaus Krippendorff
Usability Engineering
9,352 Citations1993Jakob Nielsen
Thumbs up?
6,987 Citations2002Bo Pang, Lillian Lee +1 more
This work considers the problem of classifying documents not by topic, but by overall sentiment, e.g., determining whether a review is positive or negative, and concludes by examining factors that make the sentiment classification problem more challenging.
Journal of Interactive MarketingElectronic word-of-mouth via consumer-opinion platforms: What motivates consumers to articulate themselves on the Internet?
5,779 Citations2004Thorsten Hennig‐Thurau, Kevin P. Gwinner +2 more
CERN Document Server (European Organization for Nuclear Research)Natural Language Processing with Python
3,449 Citations2009Steven Bird, Ewan Klein +1 more
Behaviour and Information TechnologyUser experience - a research agenda
2,899 Citations2006Marc Hassenzahl, Noam Tractinsky
The present introduction to the special issue on 'Empirical studies of the user experience' attempts to give a provisional answer to the question of what is meant by 'the user experience', and provides a cursory sketch of UX and how the authors think UX research will look like in the future.
Journal of Service ResearchCustomer Satisfaction and Word of Mouth
2,104 Citations1998Eugene W. Anderson
International Journal of Human-Computer StudiesCurrent practice in measuring usability: Challenges to usability studies and research
1,083 Citations2005Kasper Hornbæk
It is argued that increased attention to the problems identified and challenges discussed may strengthen studies of usability and usability research.
Journal of Quality TechnologyNonparametric Statistical Methods, 2nd, Ed.
877 Citations2001Connie M. Borror
Wiley Encyclopedia of Computer Science and EngineeringHuman–Computer Interaction
804 Citations2009Jeremy R. Cooperstock
Human–computer interaction (HCI) plays an increasingly important role in ensuring that computer systems are not only functional but also respect the needs and capabilities of the humans that use them.
User experience (UX)
757 Citations2008Marc Hassenzahl
This paper will present my particular perspective on UX and will discuss its implications for the field of Human-Computer Interaction.
Software for Use: A Practical Guide to the Models and Methods of Usage-Centered Design
735 Citations1999Larry L. Constantine
The most provocative material here is the coverage of the Internet--the book argues that many Web sites sacrifice usability for visual razzle-dazzle, and it offers ways to organize Web sites for better usability.
Short text classification in twitter to improve information filtering
727 Citations2010Bharath Sriram, Dave Fuhry +3 more
A small set of domain-specific features extracted from the author's profile and text is proposed to use to classify short text messages to a predefined set of generic classes such as News, Events, Opinions, Deals, and Private Messages.
Software Quality JournalUsability measurement and metrics: A consolidated model
634 Citations2006Ahmed Seffah, Mohammad Donyaee +2 more
This paper first reviews existing usability standards and models while highlighted the limitations and complementarities of the various standards, and explains how these various models can be unified into a single consolidated, hierarchical model of usability measurement.
Old wine in new bottles or novel challenges
623 Citations2011Javier A. Bargas-Avila, Kasper Hornbæk
This paper reviews how empirical research on User Experience (UX) is conducted, integrating products, dimensions of experience, and methodologies across a systematically selected sample of 51 publications from 2005-2009, reporting a total of 66 empirical studies.
Lecture notes in computer sciencePulse: Mining Customer Opinions from Free Text
444 Citations2005Michael Gamon, Anthony Aue +2 more
A simple but effective technique for clustering sentences, the application of a bootstrapping approach to sentiment classification, and a novel user-interface are described that enables the exploration of large quantities of customer free text.
Cambridge University Press eBooksCambridge University Press, 1696–1712
406 Citations2010Donald Francis McKenzie
Can online reviews reveal a product's true quality?
382 Citations2006Nan Hu, Paul A. Pavlou +1 more
An analytical model is derived to explain when the mean can serve as a valid representation of a product's true quality, and its implication on marketing practices is discussed.
National Bureau of Economic ResearchThe Effect of Word of Mouth on Sales: Online Book Reviews
381 Citations2003Judith A. Chevalier, Dina Mayzlin
Human-Computer InteractionThe Inference of Perceived Usability From Beauty
315 Citations2010Marc Hassenzahl, Andrew Monk
The present study's main objective is to re-examine the relation between beauty and usability, that is, the implication that “what is beautiful is usable,” and adopted an inference perspective to qualify underlying processes to examine the possibility that, under the circumstances pertaining in most studies of this kind where participants have limited experience of using a website or product, the relationship between Beauty and usability is mediated by goodness.
Journal of Consumer ResearchChanging Brand Attitudes Through Modification of Cognitive Structure
301 Citations1975Richard J. Lutz
Automatic identification of pro and con reasons in online reviews
225 Citations2006Soo-Min Kim, Eduard Hovy
A maximum entropy model is trained on the resulting labeled set to subsequently extract pros and cons from online review sites that do not explicitly provide them, and the resulting system identifies pros andcons with 66% precision and 76% recall.
interactionsFunctionality, usability, and user experience
134 Citations2006Niamh B. McNamara, Jurek Kirakowski
It is proposed that when evaluating technology, there are three primary elements that need to be considered, namely, the product, the interaction between the user and the product and the experience of using the product.
Annals of the New York Academy of SciencesClassifying and selecting UX and usability measures
117 Citations2008Nigel Bevan
The results suggest that the toxic potency of PCBs is related to their MC-type inducing ability, but the toxic characteristics are different from those of MC itself.
Remote usability evaluation
91 Citations1998José Carlos Castillo, H. Rex Hartson +1 more
The findings indicate that users can, in fact, identify and report their own critical incidents, and this method for remote usability evaluation.
Narratives of satisfying and unsatisfying experiences of current mobile augmented reality applications
89 Citations2012Thomas Olsson, Markus Salo
An online survey to explore the user experience of early stage mobile AR applications available in the market in spring 2011 indicates that the applications have not yet reached their potential in evoking a multifaceted user experience that is characteristic especially to AR.
Software Process Improvement and PracticeA framework for capturing the relationship between usability and software architecture
70 Citations2003Eelke Folmer, Jilles van Gurp +1 more
A framework is presented that expresses the relationship between usability and software architecture and consists of an integrated set of design solutions such as usability patterns and usability properties that have been identified in various cases in industry, modern day software and literature surveys.
RePEc: Research Papers in EconomicsThe Effect of Word of Mouth on Sales: Online Book Reviews
53 Citations2003Dina Mayzlin, Judith A. Chevalier
Let users tell the story
28 Citations2010Hannu Korhonen, Juha Arrasvuori +1 more
This paper describes a field study in which 21 participants wrote 116 experience reports about UX with their personal products such as smart phones and MP3 players and discusses the initial findings on the applicability of the method to evaluate UX.
The effect of task assignments and instruction types on remote asynchronous usability testing
13 Citations2012Anders Bruun, Jan Stage
A comparative study of the effect of task assignments and instruction types on the problems identified in remote asynchronous usability testing of a website for information retrieval, involving 53 prospective users shows that users solving predefined tasks identified significantly more usability problems with a significantly higher level of agreement than those working on their own authentic tasks.