An efficient framework for real-time tweet classification

International Journal of Information TechnologyPublished 17 May 2017

Imran Khan, S. Kazim Naqvi, Mansaf Alam, S. N. A. Rizvi

DOI10.1007/s41870-017-0015-x

Source URL

Citations34

SJR quartileQ2

SJR score0.72

SNIP1.57

TL;DR

This paper proposes a framework that can analyze twitter data and classify tweets on some specific subject to generate trends and illustrates the use of framework by analyzing the tweets on “Politics” domain as a subject.

Abstract

Increasing popularity of social networking sites like facebook, twitter, google+ etc. is contributing in fast proliferation of big data. Amongst social Networking sites, twitter is one of the most common source of big data where people from across the world share their views on various topics and subjects. With daily Active user count of 100-million+ users twitter is becoming a rich information source for finding trends and current happenings around the world. Twitter does provide a limited "trends" feature. To make twitter trends more interesting and informative, in this paper we propose a framework that can analyze twitter data and classify tweets on some specific subject to generate trends. We illustrate the use of framework by analyzing the tweets on "Politics" domain as a subject. In order to classify tweets we propose a tweet classification algorithm that efficiently classify the tweets.

Keywords

Computer SciencePhysics and Astronomy

Journal of Machine Learning ResearchLatent dirichlet allocation

27,049 Citations2003David M. Blei, Andrew Y. Ng +1 more

arXiv (Cornell University)Supervised Topic Models

1,315 Citations2010David M. Blei, Jon McAuliffe

The supervised latent Dirichlet allocation (sLDA) model, a statistical model of labelled documents, is introduced, which derives a maximum-likelihood procedure for parameter estimation, which relies on variational approximations to handle intractable posterior expectations.

View PDF

Labeled LDA

1,208 Citations2009Daniel Ramage, David Hall +2 more

Labeled LDA is introduced, a topic model that constrains Latent Dirichlet Allocation by defining a one-to-one correspondence between LDA's latent topics and user tags that allows Labeled LDA to directly learn word-tag correspondences.

View PDF

Lecture notes in computer scienceComparing Twitter and Traditional Media Using Topic Models

1,206 Citations2011Wayne Xin Zhao, Jing Jiang +5 more

This paper empirically compare the content of Twitter with a traditional news medium, New York Times, using unsupervised topic modeling, and finds interesting and useful findings for downstream IR or DM applications.

Short text classification in twitter to improve information filtering

727 Citations2010Bharath Sriram, Dave Fuhry +3 more

A small set of domain-specific features extracted from the author's profile and text is proposed to use to classify short text messages to a predefined set of generic classes such as News, Events, Opinions, Deals, and Private Messages.

Topic detection and tracking: event-based information organization

682 Citations2002James Allan

This chapter defines the basic concepts of TDT and provides historical context for the concepts and provides an overview of the technical approaches that have been used and that have succeeded.

Clustering short texts using wikipedia

335 Citations2007Somnath Banerjee, Krishnan Ramanathan +1 more

A method of improving the accuracy of clustering short texts by enriching their representation with additional features from Wikipedia is proposed and empirical results indicate that this enriched representation of text items can substantially improve the clustering accuracy when compared to the conventional bag of words representation.

DiscLDA: Discriminative Learning for Dimensionality Reduction and Classification

334 Citations2008Simon Lacoste-Julien, Fei Sha +1 more

This paper presents DiscLDA, a discriminative variation on Latent Dirichlet Allocation in which a class-dependent linear transformation is introduced on the topic mixture proportions, and obtains a supervised dimensionality reduction algorithm that uncovers the latent structure in a document collection while preserving predictive power for the task of classification.

Twitter Trending Topic Classification

321 Citations2011Kathy Lee, Diana Palsetia +4 more

Experiments on a database of randomly selected 768 trending topics (over 18 classes) show that classification accuracy of up to 65% and 70% can be achieved using text-based and network-based classification modeling respectively.

International Conference on Computational LinguisticsOn-line Trend Analysis with Topic Models: #twitter Trends Detection Topic Model Online

155 Citations2012Jey Han Lau, Nigel Collier +1 more

A novel topic modelling-based methodology to track emerging events in microblogs such as Twitter that has an in-built update mechanism based on time slices and implements a dynamic vocabulary.