Annotations for Opinion Mining Evaluation in the Industrial Context of the DOXA project
Published 1 May 2010
Patrick Paroubek, Alexander Pak, Djamel Mostefa
Citations13
SJR quartileQ1
SJR score0.48
SNIP1.79
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The two level opinion and sentiment model that will be used for evaluation in the DOXA project and the annotation interface the authors use for hand annotating a reference corpus are presented.
Abstract
After presenting opinion and sentiment analysis state of the art and the DOXA project, we review the few evaluation campaigns that have dealt in the past with opinion mining. Then we present the two level opinion and sentiment model that we will use for evaluation in the DOXA project and the annotation interface we use for hand annotating a reference corpus. We then present the corpus which will be used on DOXA and report on the hand-annotation task on a corpus of comments on video games and the solution adopted to obtain a sufficient level of inter-annotator agreement. 1.
Keywords
Computer Science
College Composition and CommunicationA Comprehensive Grammar of the English Language
5,535 Citations1987P Beauvais, Randolph Quirk +3 more
Meeting of the Association for Computational LinguisticsThumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews
3,654 Citations2002Peter Peter, Turney
Verbs semantics and lexical selection
3,086 Citations1994Zhibiao Wu, Martha Palmer
This paper will focus on the semantic representation of verbs in computer systems and its impact on lexical selection problems in machine translation (MT), and sees the approach as closely aligned with knowledge-based MT approaches (KBMT), and as a separate component that could be incorporated into existing systems.
The Language of Evaluation: Appraisal in English
2,779 Citations2005James R. Martin, Peter Robert Rupert White
SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining
2,489 Citations2006Andrea Esuli, Fabrizio Sebastiani
SENTIWORDNET is a lexical resource in which each WORDNET synset is associated to three numerical scores Obj, Pos and Neg, describing how objective, positive, and negative the terms contained in the synset are.
Mining the peanut gallery
1,908 Citations2003Kushal Dave, Steve Lawrence +1 more
This work develops a method for automatically distinguishing between positive and negative reviews and draws on information retrieval techniques for feature extraction and scoring, and the results for various metrics and heuristics vary depending on the testing situation.
Computers and the HumanitiesAnnotating Expressions of Opinions and Emotions in Language
1,770 Citations2005Janyce Wiebe, Theresa Wilson +1 more
The manual annotation process and the results of an inter-annotator agreement study on a 10,000-sentence corpus of articles drawn from the world press are presented.
ACM Transactions on Information SystemsMeasuring praise and criticism
1,509 Citations2003Peter D. Turney, Michael L. Littman
This article introduces a method for inferring the semantic orientation of a word from its statistical association with a set of positive and negative paradigm words, based on two different statistical measures of word association.
Towards answering opinion questions
1,051 Citations2003Hong Yu, Vasileios Hatzivassiloglou
A Bayesian classifier for discriminating between documents with a preponderance of opinions such as editorials from regular news stories is presented, and three unsupervised, statistical techniques for the significantly harder task of detecting opinions at the sentence level are described.
Using WordNet to Measure Semantic Orientations of Adjectives
647 Citations2004Jaap Kamps, M. Marx +2 more
Feature subsumption for opinion analysis
263 Citations2006Ellen Riloff, Siddharth Patwardhan +1 more
It is shown that reducing the feature set improves performance on three opinion classification tasks, especially when combined with traditional feature selection.
Identifying and analyzing judgment opinions
228 Citations2006Soo-Min Kim, Eduard Hovy
This paper decomposes the task of opinion analysis into four parts: recognizing the opinion; identifying the valence; 3) identifying the holder; and 4) identifies the topic.
Automatic Extraction of Opinion Propositions and their Holders
216 Citations2004Steven Bethard, Hong Yu +3 more
An extension of semantic parsing techniques, coupled with additional lexical and syntactic features, that can produce labels for propositional opinions as opposed to other syntactic constituents is proposed.
Human judgment as a parameter in evaluation campaigns
1 Citations2008Jean-Baptiste Berthelin, Cyril Grouin +2 more
The relevance of human judgment in an evaluation campaign is illustrated here through the DEFT text mining campaigns by testing a topic for a campaign among a limited number of human evaluators.