Annotating Expressions of Opinions and Emotions in Language
Computers and the HumanitiesPublished 1 May 2005
Janyce Wiebe, Theresa Wilson, Claire Cardie
Citations1,770
SNIP1.79
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The manual annotation process and the results of an inter-annotator agreement study on a 10,000-sentence corpus of articles drawn from the world press are presented.
Abstract
This paper describes a corpus annotation project to study issues in the manual annotation of opinions, emotions, sentiments, speculations, evaluations and other private states in language. The resulting corpus annotation scheme is described, as well as examples of its use. In addition, the manual annotation process and the results of an inter-annotator agreement study on a 10,000-sentence corpus of articles drawn from the world press are presented.
Keywords
Computer Science
Journal of the American Statistical AssociationContent Analysis: An Introduction to its Methodology.
24,566 Citations1984Mack Shelley, Klaus Krippendorff
Language<b>Introduction to functional grammar</b> . By M. A. K. Halliday London & Baltimore: Edward Arnold, 1985. Pp. xxxv, 387.
9,924 Citations1989Christian M.I.M. Matthiessen
The Measurement of Meaning
8,716 Citations1957Charles E. Osgood, George J. Suci +1 more
An Introduction to Functional Grammar
8,212 Citations2014Μ. Α. Κ. Halliday, Christian M.I.M. Matthiessen +2 more
An Introduction to Functional Grammar (Halliday的代表作)
Thumbs up?
6,987 Citations2002Bo Pang, Lillian Lee +1 more
This work considers the problem of classifying documents not by topic, but by overall sentiment, e.g., determining whether a review is positive or negative, and concludes by examining factors that make the sentiment classification problem more challenging.
College Composition and CommunicationA Comprehensive Grammar of the English Language
5,535 Citations1987P Beauvais, Randolph Quirk +3 more
Content Analysis: An Introduction to Its Methodology
5,456 Citations2019Klaus Krippendorff
Meeting of the Association for Computational LinguisticsThumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews
3,654 Citations2002Peter Peter, Turney
English Text: System and structure
1,936 Citations1992James R. Martin
Mining the peanut gallery
1,908 Citations2003Kushal Dave, Steve Lawrence +1 more
This work develops a method for automatically distinguishing between positive and negative reviews and draws on information retrieval techniques for feature extraction and scoring, and the results for various metrics and heuristics vary depending on the testing situation.
Mental Spaces: Aspects of Meaning Construction in Natural Language
1,698 Citations1985Gilles Fauconnier
ACM Transactions on Information SystemsMeasuring praise and criticism
1,509 Citations2003Peter D. Turney, Michael L. Littman
This article introduces a method for inferring the semantic orientation of a word from its statistical association with a set of positive and negative paradigm words, based on two different statistical measures of word association.
Predicting the semantic orientation of adjectives
1,439 Citations1997Vasileios Hatzivassiloglou, Kathleen McKeown
A log-linear regression model uses constraints from conjunctions to predict whether conjoined adjectives are of same or different orientations, achieving 82% accuracy in this task when each conjunction is considered independently.
John Benjamins Publishing Company eBooksEnglish Text
1,425 Citations1992James R. Martin
English Text's major contribution is to outline one way in which a rich semantically oriented functional grammar can be systematically related to a theory of discourse semantics, including deconstruction of contextual issues.
Beyond Exchange: APPRAISAL Systems in English
1,151 Citations2000Joe Martin
Evaluation in Text : Authorial Stance and the Construction of Discourse
1,146 Citations2001Susan Hunston, Geoff Thompson
Towards answering opinion questions
1,051 Citations2003Hong Yu, Vasileios Hatzivassiloglou
A Bayesian classifier for discriminating between documents with a preponderance of opinions such as editorials from regular news stories is presented, and three unsupervised, statistical techniques for the significantly harder task of detecting opinions at the sentence level are described.
Learning extraction patterns for subjective expressions
1,001 Citations2003Ellen Riloff, Janyce Wiebe
A bootstrapping process that learns linguistically rich extraction patterns for subjective (opinionated) expressions while maintaining high precision is presented.
Computational LinguisticsLearning Subjective Language
743 Citations2004Janyce Wiebe, Theresa Wilson +3 more
This article shows that the density of subjectivity clues in the surrounding context strongly affects how likely it is that a word is subjective, and it provides the results of an annotation study assessing the subjectivity of sentences with high-density features.
Sentiment analyzer: extracting sentiments about a given topic using natural language processing techniques
720 Citations2004Junbo Yi, Tetsuya Nasukawa +2 more
This work presents sentiment analyzer (SA) that extracts sentiment (or opinion) about a subject from online text documents using natural language processing (NLP) techniques.
Cognition & EmotionThe language of emotions: An analysis of a semantic field
702 Citations1989P. N. Johnson‐Laird, Keith Oatley
The Handbook of Pragmatics
570 Citations2006
The German QuarterlyTransparent Minds: Narrative Modes for Presenting Consciousness in Fiction
556 Citations1981Sharon Jackiw, Dorrit Cohn
Learning Subjective Adjectives from Corpora
520 Citations2000Janyce Wiebe
This paper identifies strong clues of subjectivity using the results of a method for clustering words according to distributional similarity (Lin 1998), seeded by a small amount of detailed manual annotation.
Learning subjective nouns using extraction pattern bootstrapping
499 Citations2003Ellen Riloff, Janyce Wiebe +1 more
The goal of the research is to develop a system that can distinguish subjective sentences from objective sentences, and a Naive Bayes classifier is trained using the subjective nouns, discourse features, and subjectivity clues identified in prior research.
Cognitive ScienceThe Referential Structure of the Affective Lexicon
480 Citations1987Andrew Ortony, Gerald L. Clore +1 more
The overall goal was to develop a comprehensive taxonomy of the affective lexicon, with special attention being devoted to the isolation of terms that refer to emotions.
Just how mad are you? finding strong and weak opinion clauses
435 Citations2004Theresa Wilson, Janyce Wiebe +1 more
This paper presents the first experimental results classifying the strength of opinions and other types of subjectivity and classifies the subjectivity of deeply nested clauses using new syntactic features developed for opinion recognition.
arXiv (Cornell University)Tracking point of view in narrative
307 Citations1994Janyce Wiebe
This paper presents an algorithm to develop an algorithm that tracks point of view on the basis of the regularities found in naturally occurring narrative, and describes the results of some preliminary empirical studies, which lend support to the algorithm.
IEEE Transactions on Fuzzy SystemsAffect analysis of text using fuzzy semantic typing
291 Citations2001Pero Subašić, Alison K. Huettner
Linguistic VariationUnspeakable sentences
195 Citations2017Liliane Haegeman
The paper develops the cartographic analysis of register based subject omission proposed in Haegeman (2013) and based on Rizzi’s (2006b) ‘Privilege of the Root’ approach, hypothesis that there is a specialized projection for the encoding of subjecthood (SubjP).
Natural Language EngineeringRecognizing subjectivity: a case study in manual tagging
174 Citations1999Rebecca Bruce, Janyce Wiebe
A case study of a sentence-level categorization in which tagging instructions are developed and used by four judges to classify clauses from the Wall Street Journal as either subjective or objective.
Identifying Collocations for Recognizing Opinions
135 Citations2001Jens Wiebe
Promising results are shown for a straightforward method of identifying collocational clues of subjectivity, as well as evidence of the usefulness of these clues for recognizing opinionated documents.
Journal of Philosophical LogicBelief in discourse representation theory
104 Citations1986Nicholas Asher
DR theory provides a novel approach to the semantics of other propsitional attitude reports and enables one to approach the topic of anaphora within belief and other propositional attitude contexts in a novel way, thus combining the semantics developed here with one the theory's original motivations.
Combining Low-Level and Summary Representations of Opinions for Multi-Perspective Question Answering
91 Citations2003Claire Cardie, Janyce Wiebe +2 more
This paper proposes an approach to multi-perspective question answering that views the task as one of opinion-oriented information extraction, and describes an annotation scheme developed for the low-level representation of opinions.
Cognitive ScienceBeliefs, Points of View, and Multiple Environments*
83 Citations1983Yorick Wilks, Janusz S. Bień
It is argued that the mechanism proposed is compatible with a general least effort hypothesis concerning human mental functioning, and one which could explain the acquisition of unsupported beliefs.
Annotating Opinions in the World Press
83 Citations2003Theresa Wilson, Janyce Wiebe
This paper presents a detailed scheme for annotating expressions of opinions, beliefs, emotions, sentiment and speculation in the news and other discourse, and explores inter-annotator agreement for individual private state expressions.
Midwest Studies in PhilosophyOn the Philosophical Foundations of the Theory of Communication: Reference
77 Citations1977Héctor-Neri Castañeda
A corpus study of evaluative and speculative language
59 Citations2001Janyce Wiebe, Rebecca Bruce +3 more
This study yields knowledge needed to design effective machine learning systems for identifying subjective language, and analyses of annotator agreement and of characteristics of subjective language are performed.
Cognitive ScienceLogical Foundations for Belief Representation
43 Citations1986William J. Rapaport
This essay presents a philosophical and computational theory of the representation of de re, de dicto, nested, and quasi-indexical belief reports expressed in natural language and shows that all four of the above categories of belief reports can be handled by a single representational technique using belief spaces containing intensional entities.
Choice Reviews OnlineThe Power of words: unveiling the speaker and writer's hidden craft
28 Citations2004
Recognizing expressions of commonsense psychology in English Text
25 Citations2003Andrew S. Gordon, Abe Kazemzadeh +2 more
The efforts to create a robust, large-scale lexical-semantic resource for the recognition and classification of expressions of commonsense psychology in English Text are described and high levels of precision and recall are achieved.
Playing the telephone game
16 Citations2004Eric Breck, Claire Cardie
In this work, a learning approach is presented that correctly determines the hierarchical structure of information filtering expressions 78.30% of the time.