A Study of Information Retrieval Weighting Schemes for Sentiment Analysis
Published 11 July 2010
George Paltoglou, Mike Thelwall
Citations351
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It is shown that variants of the classic tf.idf scheme adapted to sentiment analysis provide significant increases in accuracy, especially when using a sublinear function for term frequency weights and document frequency smoothing.
Abstract
Most sentiment analysis approaches use as baseline a support vector machines (SVM) classifier with binary unigram weights. In this paper, we explore whether more sophisticated feature weighting schemes from Information Retrieval can enhance classification accuracy. We show that variants of the classic tf.idf scheme adapted to sentiment analysis provide significant increases in accuracy, especially when using a sublinear function for term frequency weights and document frequency smoothing. The techniques are tested on a wide selection of data sets and produce the best accuracy to our knowledge. 1
Keywords
Computer Science
Information Processing & ManagementTerm-weighting approaches in automatic text retrieval
9,532 Citations1988Gerard Salton, Chris Buckley
This paper summarizes the insights gained in automatic term weighting, and provides baseline single term indexing models with which other more elaborate content analysis procedures can be compared.
The Measurement of Meaning
8,716 Citations1957Charles E. Osgood, George J. Suci +1 more
ACM Computing SurveysMachine learning in automated text categorization
7,899 Citations2002Fabrizio Sebastiani
This survey discusses the main approaches to text categorization that fall within the machine learning paradigm and discusses in detail issues pertaining to three different problems, namely, document representation, classifier construction, and classifier evaluation.
Thumbs up?
6,987 Citations2002Bo Pang, Lillian Lee +1 more
This work considers the problem of classifying documents not by topic, but by overall sentiment, e.g., determining whether a review is positive or negative, and concludes by examining factors that make the sentiment classification problem more challenging.
now publishers, Inc. eBooksOpinion Mining and Sentiment Analysis
6,745 Citations2008Bo Pang, Lillian Lee
LIBLINEAR: A Library for Large Linear Classification
6,625 Citations2008Rong-En Fan, Kai‐Wei Chang +3 more
Meeting of the Association for Computational LinguisticsThumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews
3,654 Citations2002Peter Peter, Turney
Recognizing contextual polarity in phrase-level sentiment analysis
3,379 Citations2005Theresa Wilson, Janyce Wiebe +1 more
A new approach to phrase-level sentiment analysis is presented that first determines whether an expression is neutral or polar and then disambiguates the polarity of the polar expressions.
A sentimental education
3,343 Citations2004Bo Pang, Lillian Lee
A novel machine-learning method is proposed that applies text-categorization techniques to just the subjective portions of the document, which greatly facilitates incorporation of cross-sentence contextual constraints.
A comparison of event models for naive bayes text classification
3,224 Citations1998Andrew McCallum, Kamal Nigam
It is found that the multi-variate Bernoulli performs well with small vocabulary sizes, but that the multinomial performs usually performs even better at larger vocabulary sizes--providing on average a 27% reduction in error over the multi -variateBernoulli model at any vocabulary size.
The Language of Evaluation: Appraisal in English
2,779 Citations2005James R. Martin, Peter Robert Rupert White
Text REtrieval ConferenceOkapi at TREC
2,229 Citations1994Stephen Robertson, Steve Walker +3 more
Much of the work involved investigating plausible methods of applying Okapi-style weighting to phrases, and expansion using terms from the top documents retrieved by a pilot search on topic terms was used.
Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification
2,026 Citations2007John Blitzer, Mark Dredze +1 more
This work extends to sentiment classification the recently-proposed structural correspondence learning (SCL) algorithm, reducing the relative error due to adaptation between domains by an average of 30% over the original SCL algorithm and 46% over a supervised baseline.
Joint sentiment/topic model for sentiment analysis
972 Citations2009Chenghua Lin, Yulan He
A novel probabilistic modeling framework based on Latent Dirichlet Allocation (LDA), called joint sentiment/topic model (JST), which detects sentiment and topic simultaneously from text is proposed, which is fully unsupervised.
Information Processing & ManagementA probabilistic model of information retrieval: development and comparative experiments
947 Citations2000Karen Spärck Jones, Steve Walker +1 more
ACM Transactions on Information SystemsSentiment analysis in multiple languages
894 Citations2008Ahmed Abbasi, Hsinchun Chen +1 more
Stylistic features significantly enhanced performance across all testbeds while EWGA also outperformed other feature selection methods, indicating the utility of these features and techniques for document-level classification of sentiments.
Journal of InformetricsSentiment analysis: A combined approach
791 Citations2009Rudy Prabowo, Mike Thelwall
This paper combines rule-based classification, supervised learning and machine learning into a new combined method, and proposes a semi-automatic, complementary approach in which each classifier can contribute to other classifiers to achieve a good level of effectiveness.
Simple BM25 extension to multiple weighted fields
698 Citations2004Stephen Robertson, Hugo Zaragoza +1 more
This paper describes a simple way of adapting the BM25 ranking formula to deal with structured documents and proposes a much more intuitive alternative which weights term frequencies before the non-linear term frequency saturation function is applied.
Sentiment Analysis using Support Vector Machines with Diverse Information Sources
693 Citations2004Tony Mullen, Nigel Collier
Experiments on movie review data from the Internet Movie Database demonstrate that hybrid SVMs which combine unigram-style feature-based SVMs with those based on real-valued favorability measures obtain superior performance, producing the best results yet published using this data.
Using appraisal groups for sentiment analysis
562 Citations2005Casey Whitelaw, Navendu Garg +1 more
A new method for sentiment classification based on extracting and analyzing appraisal groups such as ``very good'' or ``not terribly funny'' is presented, based on several task-independent semantic taxonomies based on Appraisal Theory.
Get out the vote
497 Citations2006Matt Thomas, Bo Pang +1 more
It is found that the incorporation of sources of information regarding relationships between discourse segments, such as whether a given utterance indicates agreement with the opinion expressed by another, yields substantial improvements over classifying speeches in isolation.
ACM SIGIR ForumExploring the similarity space
386 Citations1998Justin Zobel, Alistair Moffat
It is demonstrated that it is surprisingly difficult to identify which techniques work best, and comment on the experimental methodology required to support any claims as to the superiority of one method over another.
Recent Advances in Natural Language ProcessingCustomizing Sentiment Classifiers to New Domains: a Case Study
370 Citations2005Anthony Aue, Michael Gamon
Four different approaches to customizing a sentiment classification system to a new target domain in the absence of large amounts of labeled data are surveyed and their advantages, disadvantages and performance are discussed.
Trinity's Access to Research Output (TARA) (Trinity College Dublin)Sentiment Polarity Identification in Financial News: A Cohesion-based Approach
275 Citations2007Ann Devitt, Khurshid Ahmad
This paper aims to explore a computable metric of positive or negative polarity in financial news text which is consistent with human judgments and can be used in a quantitative analysis of news sentiment impact on financial markets.
Which side are you on?
271 Citations2006Weihao Lin, Theresa Wilson +2 more
The results show that the proposed models successfully learn how perspectives are reflected in word usage and can identify the perspective of a document with high accuracy.
Maryland Shared Open Access Repository (USMAI Consortium)Delta TFIDF: An Improved Feature Space for Sentiment Analysis
266 Citations2009Justin Martineau, Tim Finin
Using ``Annotator Rationales'' to Improve Machine Learning for Text Categorization
258 Citations2007Omar F. Zaidan, Jason Eisner +1 more
Improvements that don't add up
238 Citations2009Timothy G. Armstrong, Alistair Moffat +2 more
This paper analyzes results achieved on the TREC Ad-Hoc, Web, Terabyte, and Robust collections as reported in SIGIR and CIKM and proposes a practice of regular longitudinal comparison to ensure measurable progress, or at least prevent the lack of it from going unnoticed.
Enlighten: Publications (The University of Glasgow)The TREC Blogs06 Collection: Creating and Analysing a Blog Test Collection
185 Citations2006Craig Macdonald, Iadh Ounis
This paper describes the creation of the Blogs06 collection by the University of Glasgow, and reports statistics of the collected data, and demonstrates how some characteristics of the collection vary across the spam and non-spam components.
More than words
172 Citations2009Stephan Greene, Philip Resnik
A strong predictive connection between linguistically well motivated features and implicit sentiment is established, and it is shown how computational approximations of these features can be used to improve on existing state-of-the-art sentiment classification results.
eCommons (Cornell University)Length Normalization in Degraded Text Collections
66 Citations1995Amit Singhal, Gerard Salton +1 more
This study examines the effects of the well known cosinenormalization method in the presence of OCR errors and proposes a new, more robust, normalization method that yields significant improvements in retrieval effectiveness over cosine normalization.