Sentence and Expression Level Annotation of Opinions in User-Generated Discourse
TUbilio (Technical University of Darmstadt)Published 11 July 2010
Cigdem Toprak, Niklas Jakob, Iryna Gurevych
Citations149
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A corpus of consumer reviews from the rateitall and the eopinions websites annotated with opinion-related information is introduced and a two-level annotation scheme is presented for pinpointing the properties and functional components of the evaluations.
Abstract
In this paper, we introduce a corpus of consumer reviews from the rateitall and the eopinions websites annotated with opinion-related information. We present a two-level annotation scheme. In the first stage, the reviews are analyzed at the sentence level for (i) relevancy to a given topic, and (ii) expressing an evaluation about the topic. In the second stage, on-topic sentences containing evaluations about the topic are further investigated at the expression level for pinpointing the properties (semantic orientation, \nintensity), and the functional components of the evaluations (opinion terms, targets and holders). We discuss the annotation scheme, the inter-annotator agreement for different subtasks and our observations.
Keywords
Computer Science
Journal of the American Statistical AssociationContent Analysis: An Introduction to its Methodology.
24,566 Citations1984Mack Shelley, Klaus Krippendorff
Mining and summarizing customer reviews
7,714 Citations2004Minqing Hu, Bing Liu
This research aims to mine and to summarize all the customer reviews of a product, and proposes several novel techniques to perform these tasks.
College Composition and CommunicationA Comprehensive Grammar of the English Language
5,535 Citations1987P Beauvais, Randolph Quirk +3 more
Recognizing contextual polarity in phrase-level sentiment analysis
3,379 Citations2005Theresa Wilson, Janyce Wiebe +1 more
A new approach to phrase-level sentiment analysis is presented that first determines whether an expression is neutral or polar and then disambiguates the polarity of the polar expressions.
SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining
2,489 Citations2006Andrea Esuli, Fabrizio Sebastiani
SENTIWORDNET is a lexical resource in which each WORDNET synset is associated to three numerical scores Obj, Pos and Neg, describing how objective, positive, and negative the terms contained in the synset are.
Computers and the HumanitiesAnnotating Expressions of Opinions and Emotions in Language
1,770 Citations2005Janyce Wiebe, Theresa Wilson +1 more
The manual annotation process and the results of an inter-annotator agreement study on a 10,000-sentence corpus of articles drawn from the world press are presented.
A holistic lexicon-based approach to opinion mining
1,388 Citations2008Xiaowen Ding, Bing Liu +1 more
This paper proposes a holistic lexicon-based approach to solving the problem of determining the semantic orientations (positive, negative or neutral) of opinions expressed on product features in reviews by exploiting external evidences and linguistic conventions of natural language expressions.
Learning extraction patterns for subjective expressions
1,001 Citations2003Ellen Riloff, Janyce Wiebe
A bootstrapping process that learns linguistically rich extraction patterns for subjective (opinionated) expressions while maintaining high precision is presented.
Movie review mining and summarization
909 Citations2006Zhuang Li, Jing Feng +1 more
A multi-knowledge based approach is proposed, which integrates WordNet, statistical analysis and movie knowledge, and the experimental results show the effectiveness of the proposed approach in movie review mining and summarization.
Extracting opinions, opinion holders, and topics expressed in online news media text
460 Citations2006Soo-Min Kim, Eduard Hovy
This method uses semantic role labeling as an intermediate step to label an opinion holder and topic using data from FrameNet, and decomposes the task into three phases: identifying an opinion-bearing word, labeling semantic roles related to the word in the sentence, and then finding the holder and the topic of the opinion word among the labeled semantic roles.
Identifying sources of opinions with conditional random fields and extraction patterns
366 Citations2005Yejin Choi, Claire Cardie +2 more
This work adopts a hybrid approach that combines Conditional Random Fields (Lafferty et al., 2001) and a variation of AutoSlog (Riloff, 1996a), and shows that the combination of these two methods performs better than either one alone.
International Joint Conference on Artificial IntelligenceIdentifying expressions of opinion in context
253 Citations2007Eric Breck, Yejin Choi +1 more
An approach for identifying opinion expressions that uses conditional random fields and achieves expression-level performance within 5% of the human interannotator agreement is presented.
Topic identification for fine-grained opinion analysis
192 Citations2008Veselin Stoyanov, Claire Cardie
This paper provides an operational definition of opinion topic and presents an algorithm for opinion topic identification that, following the new definition, treats the task as a problem in topic coreference resolution.
D-Scholarship@Pitt (University of Pittsburgh)Fine-grained subjectivity and sentiment analysis: recognizing the intensity, polarity, and attitudes of private states
137 Citations2008Theresa Wilson
This dissertation investigates the manual and automatic identification of linguistic expressions of private states in a corpus of news documents from the world press with results in automatic systems for performing fine-grained subjectivity analysis that significantly outperform baseline systems.
Annotating attributions and private states
103 Citations2005Theresa Wilson, Janyce Wiebe
Extensions to a corpus annotation scheme for the manual annotation of attributions, as well as opinions, emotions, sentiments, speculations, evaluations and other private states in language are described.
Computing Reliability for Coreference Annotation
90 Citations2004Rebecca J. Passonneau
The solution I present accommodates a wide range of coding choices for the annotator, while preserving the same units across codings, permits a straightforward application of reliability measurement and incorporates a distance metric to scale disagreements.
Discourse level opinion relations
59 Citations2008Swapna Somasundaran, Josef Ruppenhofer +1 more
An annotation scheme is presented that realizes the opinion frames and via human annotation studies, it is shown that these can be reliably identified.
Annotating Subjective Content in Meetings
55 Citations2008Theresa Wilson
The scheme adapts concepts from the Multi-perspective Question Answering (MPQA) Annotation Scheme, an annotation scheme for marking opinions and attributions in the news, to mark subjective content in meetings.
Distilling Opinion in Discourse: A Preliminary Study
51 Citations2008Nicholas Asher, Farah Benamara +1 more
A preliminary study for a discourse based opinion categorization and a new annotation schema for a deep contextual opinion analysis using discourse relations are described and proposed.
Fine-grained Opinion Topic and Polarity Identification
13 Citations2008Xiwen Cheng, Feiyu Xu
OMINE, an opinion mining system which aims to identify concepts such as products and their attributes, and analyze their corresponding polarities, achieves 10% higher recall with the same level precision on the topic extraction task.