Streaming trend detection in Twitter
International Journal of Web Based CommunitiesPublished 1 January 2013
James Benhardus, Jugal Kalita
Citations171
SJR quartileQ3
SJR score0.26
SNIP0.24
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Methodologies of detecting and identifying trending topics from streaming data from Twitter's streaming API were outlined, and term frequency-inverse document frequency analysis identified unigrams, bigrams, and trigrams as trending topics.
Abstract
As social media continue to grow, the zeitgeist of society is increasingly found not in the headlines of traditional media institutions, but in the activity of ordinary individuals. The identification of trending topics utilises social media (such as Twitter) to provide an overview of the topics and issues that are currently popular within the online community. In this paper, we outline methodologies of detecting and identifying trending topics from streaming data. Data from Twitter's streaming API was collected and put into documents of equal duration using data collection procedures that allow for analysis over multiple timespans, including those not currently associated with Twitter-identified trending topics. Term frequency-inverse document frequency analysis and relative normalised term frequency analysis were performed on the documents to identify the trending topics. Relative normalised term frequency analysis identified unigrams, bigrams, and trigrams as trending topics, while term frequency-inverse document frequency analysis identified unigrams as trending topics. Application of these methodologies to streaming data resulted in F-measures ranging from 0.1468 to 0.7508.
Keywords
Computer SciencePhysics and Astronomy
Information Processing & ManagementTerm-weighting approaches in automatic text retrieval
9,532 Citations1988Gerard Salton, Chris Buckley
This paper summarizes the insights gained in automatic term weighting, and provides baseline single term indexing models with which other more elaborate content analysis procedures can be compared.
Earthquake shakes Twitter users
3,667 Citations2010Takeshi Sakaki, Makoto Okazaki +1 more
This paper investigates the real-time interaction of events such as earthquakes in Twitter and proposes an algorithm to monitor tweets and to detect a target event and produces a probabilistic spatiotemporal model for the target event that can find the center and the trajectory of the event location.
Proceedings of the VLDB EndowmentApproximate frequency counts over data streams
1,195 Citations2012Gurmeet Singh Manku, Rajeev Motwani
Information diffusion through blogspace
1,045 Citations2004Daniel Gruhl, R. Guha +2 more
A macroscopic characterization of topic propagation through the authors' corpus is presented, formalizing the notion of long-running "chatter" topics consisting recursively of "spike" topics generated by outside world events, or more rarely, by resonances within the community.
ACM SIGKDD Explorations NewsletterInformation diffusion through blogspace
951 Citations2004Daniel Gruhl, David Liben‐Nowell +2 more
A macroscopic characterization of topic propagation through the authors' corpus, formalizing the notion of long-running "chatter" topics consisting recursively of "spike" topics generated by outside world events, or more rarely, by resonances within the community.
FigsharePart-of-Speech Tagging for Twitter: Annotation, Features, and Experiments
830 Citations2018Kevin Gimpel, Nathan Schneider +8 more
A study of retrospective and on-line event detection
731 Citations1998Yiming Yang, Tom Pierce +1 more
This paper applied hierarchical and non-hierarchical document clustering algorithms to a corpus of 15,836 stories, focusing on the exploitation of both content and temporal information, and found the resulting cluster hierarchies highly informative for retrospective detection of previously unidentified events.
ACM SIGIR ForumOn-Line New Event Detection and Tracking
653 Citations2017James Allan, Ron Papka +1 more
Edinburgh Research Explorer (University of Edinburgh)Streaming First Story Detection with application to Twitter
586 Citations2010Saša Petrović, Miles Osborne +1 more
This work presents an algorithm based on locality-sensitive hashing which is able to overcome the limitations of traditional approaches, while maintaining competitive results in event detection on web-scale corpora.
Lexical Normalisation of Short Text Messages: Makn Sens a #twitter
455 Citations2011Bo Han, Timothy Baldwin
This paper targets out-of-vocabulary words in short text messages and proposes a method for identifying and normalising ill-formed words, which achieves state- of-the-art performance over an SMS corpus and a novel dataset based on Twitter.
Recognizing Named Entities in Tweets
372 Citations2011Xiaohua Liu, Shaodian Zhang +2 more
This work proposes to combine a K-Nearest Neighbors classifier with a linear Conditional Random Fields model under a semi-supervised learning framework to tackle the challenges of Named Entities Recognition for tweets.
Journal of Information ScienceThe automatic identification of stop words
297 Citations1992W. John Wilbur, Karl Sirotkin
It is shown how the concept of relevance may be replaced by the condition of being highly rated by a similarity measure and it becomes possible to identify the stop words in a cullectmn by automated statistical testing.
Annotating named entities in Twitter data with crowdsourcing
285 Citations2010Tim Finin, William Murnane +4 more
This paper describes the experience using both Amazon Mechanical Turk (MTurk) and Crowd-Flower to collect simple named entity annotations for Twitter status updates, and describes how to use MTurk to collect judgements on the quality of "word clouds."
Event threading within news topics
247 Citations2004Ramesh Nallapati, Ao Feng +2 more
This work attempts to capture the rich structure of events and their dependencies in a news topic through the authors' event models, and takes into account novel features such as temporal locality of stories for event recognition and time-ordering for capturing dependencies.
International Journal on Digital LibrariesA probabilistic justification for using tf×idf term weighting in information retrieval
183 Citations2000Djoerd Hiemstra
The paper shows that the new probabilistic interpretation of tf×idf term weighting might lead to better understanding of statistical ranking mechanisms, for example by explaining how they relate to coordination level ranking.
North American Chapter of the Association for Computational LinguisticsThe Edinburgh Twitter Corpus
160 Citations2010Saša Petrović, Miles Osborne +1 more
The first release of the corpus of 97 million Twitter posts is described, believing that this data will prove valuable to researchers working in social media, natural language processing, large-scale data processing, and similar areas.
Experiments in Microblog Summarization
143 Citations2010Beaux Sharifi, Mark-Anthony Hutton +1 more
The goal is to produce summaries that are similar to what a human would produce for the same collection of posts on a specific topic, and evaluate the summaries produced by the summarizing algorithms, compare them with human-produced summaries and obtain excellent results.
Computers in Biology and MedicineAn analysis of statistical term strength and its use in the indexing and retrieval of molecular biology texts
117 Citations1996W. John Wilbur, Yiming Yang
A new, information theoretical interpretation of term strength is given, some of its uses in focusing the processing of documents for information retrieval and new results obtained in document categorization are described.
Integrating web-based intelligence retrieval and decision-making from the twitter trends knowledge base
111 Citations2009Marc Cheong, Vincent Lee
The findings reveal a pattern behind trends on Twitter, enabling us to see how it 'ticks' and evolves though visualization methods, and enable us to understand the underlying characteristics behind the 'trend setters', providing a new perspective on the contributors of a trend.
Insertion, Deletion, or Substitution? Normalizing Text Messages without Pre-categorization nor Supervision
108 Citations2011Fei Liu, Fuliang Weng +2 more
This paper proposes a unified letter transformation approach that requires neither pre-categorization nor human supervision and significantly outperformed the state-of-the-art deletion-based abbreviation system and the jazzy spell checker.
Twitter analytics: Architecture, tools and analysis
42 Citations2010Rohan D.W Perera, S. Anand +2 more
A software architecture that uses a Twitter application program interface (API) to collect the tweets sent to specific users and indicates that the arrival process of new tweets to a user can be modeled as a Poisson Process while the number of re-tweets follow a geometric distribution.