Annotating Topics of Opinions
Published 1 May 2008Open access
Veselin Stoyanov, Claire Cardie
Citations59
SJR quartileQ1
SJR score0.48
SNIP1.79
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper proposes a methodology for the manual annotation of opinion topics and uses it to annotate a portion of an existing general-purpose opinion corpus with opinion topic information to suggest that the annotations are reliable.
Abstract
Fine-grained subjectivity analysis has been the subject of much recent research attention. As a result, the field has gained a number of working definitions, technical approaches and manually annotated corpora that cover many facets of subjectivity. Little work has been done, however, on one aspect of fine-grained opinions – the specification and identification of opinion topics. In particular, due to the difficulty of manual opinion topic annotation, no general-purpose opinion corpus with information about topics of fine-grained opinions currently exists. In this paper, we propose a methodology for the manual annotation of opinion topics and use it to annotate a portion of an existing general-purpose opinion corpus with opinion topic information. Inter-annotator agreement results according to a number of metrics suggest that the annotations are reliable. 1.
Keywords
Computer Science
Journal of the American Statistical AssociationContent Analysis: An Introduction to its Methodology.
24,566 Citations1984Mack Shelley, Klaus Krippendorff
Thumbs up?
6,987 Citations2002Bo Pang, Lillian Lee +1 more
This work considers the problem of classifying documents not by topic, but by overall sentiment, e.g., determining whether a review is positive or negative, and concludes by examining factors that make the sentiment classification problem more challenging.
College Composition and CommunicationA Comprehensive Grammar of the English Language
5,535 Citations1987P Beauvais, Randolph Quirk +3 more
Meeting of the Association for Computational LinguisticsThumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews
3,654 Citations2002Peter Peter, Turney
A sentimental education
3,343 Citations2004Bo Pang, Lillian Lee
A novel machine-learning method is proposed that applies text-categorization techniques to just the subjective portions of the document, which greatly facilitates incorporation of cross-sentence contextual constraints.
Mining the peanut gallery
1,908 Citations2003Kushal Dave, Steve Lawrence +1 more
This work develops a method for automatically distinguishing between positive and negative reviews and draws on information retrieval techniques for feature extraction and scoring, and the results for various metrics and heuristics vary depending on the testing situation.
Computers and the HumanitiesAnnotating Expressions of Opinions and Emotions in Language
1,770 Citations2005Janyce Wiebe, Theresa Wilson +1 more
The manual annotation process and the results of an inter-annotator agreement study on a 10,000-sentence corpus of articles drawn from the world press are presented.
Extracting product features and opinions from reviews
1,720 Citations2005Ana-Maria Popescu, Oren Etzioni
Opine is introduced, an unsupervised information-extraction system which mines reviews in order to build a model of important product features, their evaluation by reviewers, and their relative quality across products.
Mining opinion features in customer reviews
1,273 Citations2004Minqing Hu, Bing Liu
This project aims to summarize all the customer reviews of a product by mining opinion/product features that the reviewers have commented on and a number of techniques are presented to mine such features.
Towards answering opinion questions
1,051 Citations2003Hong Yu, Vasileios Hatzivassiloglou
A Bayesian classifier for discriminating between documents with a preponderance of opinions such as editorials from regular news stories is presented, and three unsupervised, statistical techniques for the significantly harder task of detecting opinions at the sentence level are described.
Learning extraction patterns for subjective expressions
1,001 Citations2003Ellen Riloff, Janyce Wiebe
A bootstrapping process that learns linguistically rich extraction patterns for subjective (opinionated) expressions while maintaining high precision is presented.
Sentiment analyzer: extracting sentiments about a given topic using natural language processing techniques
720 Citations2004Junbo Yi, Tetsuya Nasukawa +2 more
This work presents sentiment analyzer (SA) that extracts sentiment (or opinion) about a subject from online text documents using natural language processing (NLP) techniques.
A model-theoretic coreference scoring scheme
682 Citations1995Marc Vilain, John D. Burger +3 more
This note describes a scoring scheme for the coreference task in MUC6 that improves on the original approach by grounding the scoring scheme in terms of a model; producing more intuitive recall and precision scores; and not requiring explicit computation of the transitive closure of coreference.
Lecture notes in computer scienceCreating Subjective and Objective Sentence Classifiers from Unannotated Texts
543 Citations2005Janyce Wiebe, Ellen Riloff
The results of developing subjectivity classifiers using only unannotated texts for training rivals that of previous supervised learning approaches and advances the state of the art in objective sentence classification.
On coreference resolution performance metrics
535 Citations2005Xiaoqiang Luo
The paper proposes a Constrained Entity-Alignment F-Measure (CEAF) for evaluating coreference resolution and shows that the best alignment is a maximum bipartite matching problem which can be solved by the Kuhn-Munkres algorithm.
Extracting opinions, opinion holders, and topics expressed in online news media text
460 Citations2006Soo-Min Kim, Eduard Hovy
This method uses semantic role labeling as an intermediate step to label an opinion holder and topic using data from FrameNet, and decomposes the task into three phases: identifying an opinion-bearing word, labeling semantic roles related to the word in the sentence, and then finding the holder and the topic of the opinion word among the labeled semantic roles.
Just how mad are you? finding strong and weak opinion clauses
435 Citations2004Theresa Wilson, Janyce Wiebe +1 more
This paper presents the first experimental results classifying the strength of opinions and other types of subjectivity and classifies the subjectivity of deeply nested clauses using new syntactic features developed for opinion recognition.
Identifying sources of opinions with conditional random fields and extraction patterns
366 Citations2005Yejin Choi, Claire Cardie +2 more
This work adopts a hybrid approach that combines Conditional Random Fields (Lafferty et al., 2001) and a variation of AutoSlog (Riloff, 1996a), and shows that the combination of these two methods performs better than either one alone.
Automatic Extraction of Opinion Propositions and their Holders
216 Citations2004Steven Bethard, Hong Yu +3 more
An extension of semantic parsing techniques, coupled with additional lexical and syntactic features, that can produce labels for propositional opinions as opposed to other syntactic constituents is proposed.
Lecture notes in computer scienceCollecting Evaluative Expressions for Opinion Extraction
189 Citations2005Nozomi Kobayashi, Kentaro Inui +3 more
A semi-automatic method that uses particular cooccurrence patterns of evaluated subjects, focused attributes and value expressions to accelerate the process of collecting evaluative expressions by applying a text mining technique is proposed.
Multi-perspective question answering using the OpQA corpus
135 Citations2005Veselin Stoyanov, Claire Cardie +1 more
It is argued that traditional fact-based QA approaches may have difficulty in an MPQA setting without modification, and the use of machine learning and rule-based subjectivity and opinion source filters are investigated and shown that they can be used to guideMPQA systems.
Computing Reliability for Coreference Annotation
90 Citations2004Rebecca J. Passonneau
The solution I present accommodates a wide range of coding choices for the annotator, while preserving the same units across codings, permits a straightforward application of reliability measurement and incorporates a distance metric to scale disagreements.