Comparing Twitter Summarization Algorithms for Multiple Post Summaries
Published 1 October 2011
David I. Inouye, Jugal Kalita
Citations180
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper compares algorithms for extractive summarization of micro log posts with two algorithms that produce summaries by selecting several posts from a given set.
Abstract
Due to the sheer volume of text generated by a micro log site like Twitter, it is often difficult to fully understand what is being said about various topics. In an attempt to understand micro logs better, this paper compares algorithms for extractive summarization of micro log posts. We present two algorithms that produce summaries by selecting several posts from a given set. We evaluate the generated summaries by comparing them to both manually produced summaries and summaries produced by several leading traditional summarization systems. In order to shed light on the special nature of Twitter posts, we include extensive analysis of our results, some of which are unexpected.
Keywords
Computer Science
Computer Networks and ISDN SystemsThe anatomy of a large-scale hypertextual Web search engine
15,828 Citations1998Sergey Brin, Lawrence M. Page
This paper provides an in-depth description of Google, a prototype of a large-scale search engine which makes heavy use of the structure present in hypertext and looks at the problem of how to effectively deal with uncontrolled hypertext collections where anyone can publish anything they want.
k-means++: the advantages of careful seeding
6,303 Citations2007David Arthur, Sergei Vassilvitskii
By augmenting k-means with a very simple, randomized seeding technique, this work obtains an algorithm that is Θ(logk)-competitive with the optimal clustering.
Empirical Methods in Natural Language ProcessingTextRank: Bringing Order into Text
3,347 Citations2004Rada Mihalcea, Paul Tarau
TextRank, a graph-based ranking model for text processing, is introduced and it is shown how this model can be successfully used in natural language applications.
Co-clustering documents and words using bipartite spectral graph partitioning
1,696 Citations2001Inderjit S. Dhillon
A new spectral co-clustering algorithm is used that uses the second left and right singular vectors of an appropriately scaled word-document matrix to yield good bipartitionings and it can be shown that the singular vectors solve a real relaxation to the NP-complete graph bipartitionsing problem.
Journal of Artificial Intelligence ResearchLexRank: Graph-based Lexical Centrality as Salience in Text Summarization
1,650 Citations2004G. Erkan, D. R. Radev
A new approach, LexRank, for computing sentence importance based on the concept of eigenvector centrality in a graph representation of sentences is considered and the LexRank with threshold method outperforms the other degree-based techniques including continuous LexRank.
Automatic evaluation of summaries using N-gram co-occurrence statistics
1,588 Citations2003Chin-Yew Lin, Eduard Hovy
The results show that automatic evaluation using unigram co-occurrences between summary pairs correlates surprising well with human evaluations, based on various statistical metrics; while direct application of the BLEU evaluation procedure does not always give good results.
University of North Texas Digital Library (University of North Texas)TextRank: Bringing Order into Texts
633 Citations2004Mihalcea, Rada, 1974-, Tarau, Paul
University of Minnesota Digital Conservancy (University of Minnesota)Criterion Functions for Document Clustering: Experiments and Analysis
592 Citations2001Ying Zhao, George Karypis
CACTUS—clustering categorical data using summaries
470 Citations1999Venkatesh Ganti, Johannes Gehrke +1 more
This paper introduces a novel formalization of a cluster for categorical attributes by generalizing a definition of a clusters for numerical attributes and describes a very fast summarizationbased algorithm called CACTUS that discovers exactly such clusters in the data.
Columbia Academic Commons (Columbia University)MEAD - A Platform for Multidocument Multilingual Text Summarization
347 Citations2004Dragomir Radev, Timothy Allison +15 more
The functionality of MEAD is described, a comprehensive, public domain, open source, multidocument multilingual summarization environment that has been thus far downloaded by more than 500 organizations.
Information Processing & ManagementBeyond SumBasic: Task-focused summarization with sentence simplification and lexical expansion
317 Citations2007Lucy Vanderwende, Hisami Suzuki +2 more
This paper details the design of a generic extractive summarization system, which ranked first out of 22 systems in terms of overall mean Pyramid score; and in the human evaluation of summary responsiveness to the topic, the system ranked third out of 35 systems.
LexRank: Graph-based Centrality as Salience in Text Summarization
246 Citations2004Güneş Erkan, Dragomir Radev
A new approach, LexRank, for computing sentence importance based on the concept of eigenvector centrality in a graph representation of sentences is considered and the LexRank with threshold method outperforms the other degree-based techniques including continuous LexRank.
Columbia Academic Commons (Columbia University)Towards multidocument summarization by reformulation: progress and prospects
192 Citations1999Kathleen McKeown, Judith L. Klavans +3 more
The evaluation of system components shows that learning over multiple extracted linguistic features is more effective than information retrieval approaches at identifying similar text units for summarization and that it is possible to generate a fluent summary that conveys similarities among documents even when full semantic interpretations of the input text are not available.
Summarizing Microblogs Automatically
170 Citations2010Beaux Sharifi, Mark-Anthony Hutton +1 more
An algorithm is developed that takes a trending phrase or any phrase specified by a user, collects a large number of posts containing the phrase, and provides an automatically created summary of the posts related to the term.
Experiments in Single and Multi-Document Summarization Using MEAD
153 Citations2001Dragomir Radev, Sasha Blair-Goldensohn +1 more
This paper describes four experiments in text summarization about the identification by human subjects of cross-document structural relationships such as identity, paraphrase, elaboration, and fulfillment and presents numerical evaluations of all four experiments.
Experiments in Microblog Summarization
143 Citations2010Beaux Sharifi, Mark-Anthony Hutton +1 more
The goal is to produce summaries that are similar to what a human would produce for the same collection of posts on a specific topic, and evaluate the summaries produced by the summarizing algorithms, compare them with human-produced summaries and obtain excellent results.
Summarization as feature selection for text categorization
71 Citations2001Aleksander Kołcz, Vidya Prabakarmurthi +1 more
It is argued that for a large class of automatic categorization algorithms, extraction-based document categorization can be viewed as a particular form of feature selection performed on the full text of the document and its impact can be compared with state-of-the-art feature selection techniques especially devised to provide good categorization performance.
Summarizing natural language database responses
11 Citations1986Jugal Kalita, Marlene L. Jones +1 more
This paper presents details of the design and implementation of a system that produces summary responses to queries of a relational data base that is largely domain-independent, and hence can be ported relatively easily from one data base to another.
A response to the need for summary responses
7 Citations1984Jugal Kalita, M. J. Colbourn +1 more
This paper argues that natural language interfaces to databases should be able to produce summary responses as well as listing actual data, and describes a system developed to generate such summary responses.
Subtopic-based multi-document summarization
4 Citations2009Lin Dai, Jiliang Tang +1 more
This paper proposes a novel approach for multi-document summarization based on subtopic segmentation that firstly detects the subtopics in a topic, and then finds the central sentence for each subtopic.