Combining lexicon-based and learning-based methods for twitter sentiment analysis

Published 29 June 2011

Lei Zhang, Riddhiman Ghosh, Mohamed Dekhil, Meichun Hsu, Bing Liu

Citations432

TL;DR

This paper proposes a new entity-level sentiment analysis method for Twitter that dramatically improves the recall and the F-score, and outperforms the state-of-the-art baselines.

Abstract

With the booming of microblogs on the Web, people have begun to express their opinions on a wide variety of topics on Twitter and other similar services. Sentiment analysis on entities (e.g., products, organizations, people, etc.) in tweets (posts on Twitter) thus becomes a rapid and effective way of gauging public opinion for business marketing or social studies. However, Twitter's unique characteristics give rise to new problems for current sentiment analysis methods, which originally focused on large opinionated corpora such as product reviews. In this paper, we propose a new entity-level sentiment analysis method for Twitter. The method first adopts a lexiconbased approach to perform entity-level sentiment analysis. This method can give high precision, but low recall. To improve recall, additional tweets that are likely to be opinionated are identified automatically by exploiting the information in the result of the lexicon-based method. A classifier is then trained to assign polarities to the entities in the newly identified tweets. Instead of being labeled manually, the training examples are given by the lexicon-based approach. Experimental results show that the proposed method dramatically improves the recall and the F-score, and outperforms the state-of-the-art baselines. External Posting Date: June 21, 2011 [Fulltext] Approved for External Publication Internal Posting Date: June 21, 2011 [Fulltext]  Copyright 2011 Hewlett-Packard Development Company, L.P. Combining Lexicon-based and Learning-based Methods for Twitter Sentiment Analysis Lei Zhang, Riddhiman Ghosh, Mohamed Dekhil, Meichun Hsu, Bing Liu Hewlett-Packard Laboratories University of Illinois at Chicago 1501 Page Mill Rd., Palo Alto, CA 851 S. Morgan St., Chicago, IL {riddhiman.ghosh, mohamed.dekhil, {lzhang3, liub}@cs.uic.edu meichun.hsu}@hp.com

Keywords

Computer Science

Mining and summarizing customer reviews

7,714 Citations2004Minqing Hu, Bing Liu

This research aims to mine and to summarize all the customer reviews of a product, and proposes several novel techniques to perform these tasks.

now publishers, Inc. eBooksOpinion Mining and Sentiment Analysis

6,745 Citations2008Bo Pang, Lillian Lee

Computational LinguisticsLexicon-Based Methods for Sentiment Analysis

3,255 Citations2011Maite Taboada, Julian Brooke +3 more

The Semantic Orientation CALculator (SO-CAL) uses dictionaries of words annotated with their semantic orientation (polarity and strength), and incorporates intensification and negation, and is applied to the polarity classification task.

View PDF

Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification

2,026 Citations2007John Blitzer, Mark Dredze +1 more

This work extends to sentiment classification the recently-proposed structural correspondence learning (SCL) algorithm, reducing the relative error due to adaptation between domains by an average of 30% over the original SCL algorithm and 46% over a supervised baseline.

Determining the sentiment of opinions

1,485 Citations2004Soo-Min Kim, Eduard Hovy

A system that, given a topic, automatically finds the people who hold opinions about that topic and the sentiment of each opinion and another module for determining word sentiment and another for combining sentiments within a sentence is presented.

View PDF

A holistic lexicon-based approach to opinion mining

1,388 Citations2008Xiaowen Ding, Bing Liu +1 more

This paper proposes a holistic lexicon-based approach to solving the problem of determining the semantic orientations (positive, negative or neutral) of opinions expressed on product features in reviews by exploiting external evidences and linguistic conventions of natural language expressions.

Robust Sentiment Detection on Twitter from Biased and Noisy Data

875 Citations2010Luciano Barbosa, Junlan Feng

This paper proposes an approach to automatically detect sentiments on Twitter messages (tweets) that explores some characteristics of how tweets are written and meta-information of the words that compose these messages and leverages sources of noisy labels as training data.

Cross-domain sentiment classification via spectral feature alignment

789 Citations2010Sinno Jialin Pan, Xiaochuan Ni +3 more

This work develops a general solution to sentiment classification when the authors do not have any labels in a target domain but have some labeled data in a different domain, regarded as source domain and proposes a spectral feature alignment (SFA) algorithm to align domain-specific words from different domains into unified clusters, with the help of domain-independent words as a bridge.

Enhanced Sentiment Learning Using Twitter Hashtags and Smileys

705 Citations2010Dmitry Davidov, Oren Tsur +1 more

A supervised sentiment classification framework which is based on data from Twitter, a popular microblogging service, is proposed, utilizing 50 Twitter tags and 15 smileys as sentiment labels, allowing identification and classification of diverse sentiment types of short texts.

Lecture notes in computer scienceCreating Subjective and Objective Sentence Classifiers from Unannotated Texts

543 Citations2005Janyce Wiebe, Ellen Riloff

The results of developing subjectivity classifiers using only unannotated texts for training rivals that of previous supervised learning approaches and advances the state of the art in objective sentence classification.