A Human-annotated Dataset for Evaluating Tweet Ranking Algorithms
Published 1 January 2015
Dominic Rout, Kalina Bontcheva
Citations3
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
To facilitate the development and comparative evaluation of tweet ranking methods, a task for which re-tweets do not form a reliable gold standard, a new, openly available Twitter corpus has been created.
Abstract
Social media monitoring is now an essential part of brand management, political science, and news production. Automatic tweet ranking and content recommendation methods are required, in order to support human analysts in deriving useful insights from large-scale social media data. To facilitate the development and comparative evaluation of tweet ranking methods, a task for which re-tweets do not form a reliable gold standard, a new, openly available Twitter corpus has been created. A number of results for several popular recommendation algorithms are presented for this corpus.
Keywords
Computer SciencePhysics and Astronomy
Journal of the American Society for Information ScienceIndexing by latent semantic analysis
12,677 Citations1990Scott Deerwester, Susan Dumais +3 more
Cambridge University Press eBooksIntroduction to Information Retrieval
10,873 Citations2008Christopher D. Manning, Prabhakar Raghavan +1 more
This textbook teaches classical and web information retrieval, including web search and the related areas of text classification and text clustering from basic concepts, making it perfect for introductory courses in information retrieval for advanced undergraduates and graduate students in computer science.
ROUGE: A Package for Automatic Evaluation of Summaries
8,286 Citations2004Chin-Yew Lin
Four different RouGE measures are introduced: ROUGE-N, ROUge-L, R OUGE-W, and ROUAGE-S included in the Rouge summarization evaluation package and their evaluations.
Empirical Methods in Natural Language ProcessingTextRank: Bringing Order into Text
3,347 Citations2004Rada Mihalcea, Paul Tarau
TextRank, a graph-based ranking model for text processing, is introduced and it is shown how this model can be successfully used in natural language applications.
Information Processing & ManagementCentroid-based summarization of multiple documents
1,043 Citations2003Dragomir Radev, Hongyan Jing +2 more
A multi-document summarizer, MEAD, is presented, which generates summaries using cluster centroids produced by a topic detection and tracking system and an evaluation scheme based on sentence utility and subsumption is applied.
An Empirical Study on Learning to Rank of Tweets
257 Citations2010Yajuan Duan, Long Jiang +3 more
This paper proposes a new ranking strategy which uses not only the content relevance of a tweet, but also the account authority and tweet-specific features such as whether a URL link is included in the tweet.
Comparing Twitter Summarization Algorithms for Multiple Post Summaries
180 Citations2011David I. Inouye, Jugal Kalita
This paper compares algorithms for extractive summarization of micro log posts with two algorithms that produce summaries by selecting several posts from a given set.
User oriented tweet ranking
175 Citations2011İbrahim UYSAL, W. Bruce Croft
This paper proposes a personalized tweet ranking method, leveraging the use of retweet behavior, to bring more important tweets forward, and investigates how to determine the audience of tweets more effectively, by ranking the users based on their likelihood of retweeting the tweets.
Summarizing Microblogs Automatically
170 Citations2010Beaux Sharifi, Mark-Anthony Hutton +1 more
An algorithm is developed that takes a trending phrase or any phrase specified by a user, collects a large number of posts containing the phrase, and provides an automatically created summary of the posts related to the term.
Journal of Computer-Mediated CommunicationExplaining the Relationship between Internet Use and Interpersonal Trust: Taking into Account Motivation and Information Overload
132 Citations2008Christopher E. Beaudoin
This analysis indicates that the effects of social resource motivation for Internet use on interpersonal trust were mediated by Internet use and perceived information overload.
Edinburgh Research Explorer (University of Edinburgh)Tweet Recommendation with Graph Co-Ranking
91 Citations2012Rui Yan, Mirella Lapata +1 more
A graph-theoretic model for tweet recommendation that presents users with items they may have an interest in and can be parametrized to take into account user preferences, the popularity of tweets and their authors, and diversity.
ComputerPowers of 10: Modeling Complex Information-Seeking Systems at Multiple Scales
58 Citations2009Peter Pirolli
New models of information-seeking support systems offer two advantages: they move from prescientific conceptual frameworks about information seeking to more rigorous scientific theories and predictive models while, at the same time, expanding the kinds of things the authors study and develop.
arXiv (Cornell University)Social Media and Information Overload: Survey Results
52 Citations2013Kalina Bontcheva, Genevieve Gorrell +1 more
A UK-based online questionnaire investigating aspects of usage of user-generated media (UGM), such as Facebook, LinkedIn and Twitter, attracted 587 participants, and generally speaking, participants express satisfaction with the media, though a significant minority express a range of concerns including information overload and privacy.
Proceedings of the International AAAI Conference on Web and Social MediaRelevance Modeling for Microblog Summarization
49 Citations2021Sanda M. Harabagiu, Andrew Hickl
This paper introduces a new type of summarization task, known as microblog summarization, which aims to synthesize content from multiple microblog posts on the same topic into a human-readable prose description of fixed length using a generative model and a user behavior model.
Lecture notes in computer scienceComparing Algorithms for Microblog Summarisation
23 Citations2014Stuart Mackie, Richard McCreadie +2 more
This paper compares the performance of eleven summarisation approaches using four microblog summarisation datasets, with the aim of determining which are the most effective and therefore should be used as baselines in future research.
Reliably Evaluating Summaries of Twitter Timelines
9 Citations2013Dominic Rout, Kalina Bontcheva +1 more
The focus of the paper is on designing a user study, through which to create a gold standard for evaluating automatically generated summaries of personal timelines, which is unsatisfactory since retweets only account for certain kinds of post relevance.
IEEE Internet ComputingThanks for the Fish but I'm Drowning!
3 Citations2010Fred Douglis
Fred Douglis writes his farewell column as editor in chief and discusses how social networking sites can add to the problem of information overload.