Generating and validating abstracts of meeting conversations: a user study
Published 7 July 2010
Gabriel Murray, Giuseppe Carenini, Raymond T. Ng
Citations66
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It is found that the automatically generated summaries are ranked significantly higher than human-selected extracts on coherence and usability criteria, and more generally, users demonstrate a strong preference for abstract-style summaries over extracts.
Abstract
In this paper we present a complete system for automatically generating natural language abstracts of meeting conversations. This system is comprised of components relating to interpretation of the meeting documents according to a meeting ontology, transformation or content selection from that source representation to a summary representation, and generation of new summary text. In a formative user study, we compare this approach to gold-standard human abstracts and extracts to gauge the usefulness of the different summary types for browsing meeting conversations. We find that our automatically generated summaries are ranked significantly higher than human-selected extracts on coherence and usability criteria. More generally, users demonstrate a strong preference for abstract-style summaries over extracts. 1
Keywords
Computer Science
arXiv (Cornell University)Building Natural-Language Generation Systems
1,249 Citations1996Ehud Reiter
Lecture notes in computer scienceThe AMI Meeting Corpus: A Pre-announcement
879 Citations2006Jean Carletta, Simone Ashby +15 more
The AMI Meeting Corpus is a multi-modal data set consisting of 100 hours of meeting recordings, being created in the context of a project that is developing meeting browsing technology and will eventually be released publicly.
Computational LinguisticsSentence Fusion for Multidocument News Summarization
443 Citations2005Regina Barzilay, Kathleen McKeown
This article introduces sentence fusion, a novel text-to-text generation technique for synthesizing common information across documents that moves the summarization field from the use of purely extractive methods to the generation of abstracts that contain sentences not found in any of the input documents and can synthesize information across sources.
Statistics-Based Summarization - Step One: Sentence Compression
405 Citations2000Kevin Knight, Daniel Marcu
This paper focuses on sentence compression, a simpler version of this larger challenge, and aims to achieve two goals simultaneously: the compressions should be grammatical, and they should retain the most important pieces of information.
Artificial IntelligenceAutomatic generation of textual summaries from neonatal intensive care data
314 Citations2008François Portet, Ehud Reiter +5 more
Auto-summarization of audio-video presentations
222 Citations1999Li-wei He, Elizabeth Sanocki +2 more
A user study is reported that compares automatically generated summaries that are 20%-25% the length of full presentations to author generated summary, and finds that users learn from the computer-generated summaries, although less than from authors' summaries.
Computational LinguisticsGenerating Indicative-Informative Summaries with SumUM
140 Citations2002Horacio Saggion, Guy Lapalme
SumUM is a text summarization system that takes a raw technical text as input and produces an indicative informative summary that motivates the topics, describes entities, and defines concepts.
Natural Language EngineeringChoosing the content of textual summaries of large time-series data sets
103 Citations2006Jin Yu, Ehud Reiter +2 more
An architecture for generating short (a few sentences) summaries of large time-series data sets that integrates pattern recognition, pattern abstraction, selection of the most significant patterns, microplanning, microplanning (especially aggregation), and realisation is developed.
Summarizing spoken and written conversations
91 Citations2008Gabriel Murray, Giuseppe Carenini
By treating meetings and emails as conversations with general conversational features in common, the system can achieve competitive results with state-of-the-art systems that rely on more domain-specific features.
A global optimization framework for meeting summarization
91 Citations2009Dan Gillick, Korbinian Riedhammer +2 more
This work introduces a model for extractive meeting summarization based on the hypothesis that utterances convey bits of information, or concepts, and achieves ROUGE scores at least as good as a RouGE-based oracle derived from human summaries.
From Text to Speech Summarization
71 Citations2006Kathleen McKeown, Julia Hirschberg +2 more
It is illustrated how features derived from speech can help determine summary content within two ongoing summarization projects at Columbia University.
Constraint-based sentence compression an integer programming approach
70 Citations2006James Clarke, Mirella Lapata
This work develops an integer programming formulation and infer globally optimal compressions in the face of linguistically motivated constraints and shows that such a formulation allows for relatively simple and knowledge-lean compression models that do not require parallel corpora or large-scale resources.
Extractive vs. NLG-based abstractive summarization of evaluative text
69 Citations2008Giuseppe Carenini, Jackie Chi Kit Cheung
A novel measure of corpus controversiality of opinions contained in evaluative text is defined, and the results of a user study comparing extractive and NLG-based abstractive summarization at different levels of controversiality suggest the margin by which abstraction outperforms extraction is greater when controversiality is high.
Annotating Subjective Content in Meetings
55 Citations2008Theresa Wilson
The scheme adapts concepts from the Multi-perspective Question Answering (MPQA) Annotation Scheme, an annotation scheme for marking opinions and attributions in the news, to mark subjective content in meetings.
Resolving It, This, and That in Unrestricted Multi-Party Dialog
46 Citations2007Christoph Müller
The system handles NP-anaphoric as well as discoursedeictic anaphors, i.e. pronouns with VP antecedents, and shows that the system performs significantly better than a recency-based baseline.
ACM Transactions on Speech and Language ProcessingExtrinsic summarization evaluation
30 Citations2009Gabriel Murray, Thomas Kleinbauer +4 more
It is found that while ASR errors affect user satisfaction on an information retrieval task, users can adapt their browsing behavior to complete the task satisfactorily and consider extractive summaries to be intuitive and useful tools for browsing multimodal meeting data.
Resolving “You” in Multi-Party Dialog
25 Citations2007Surabhi Gupta, John Niekrasz +2 more
This paper presents experiments into the resolution of “you” in multi-party dialog, dividing this process into two tasks: distinguishing between generic and referential uses; and then, for referental uses, identifying the referred-to addressee(s).
Leveraging sentence weights in a concept-based optimization framework for extractive meeting summarization
25 Citations2009Shasha Xie, Benoît Favre +2 more
This work adopts an unsupervised concept-based global optimization framework for extractive meeting summarization, where a subset of sentences is selected to cover as many important concepts as possible and proposes to leverage sentence importance weights in this model.
Interpretation and Transformation for Abstracting Conversations
25 Citations2010Gabriel Murray, Giuseppe Carenini +1 more
This work addresses the challenge of automatically abstracting conversations such as face-to-face meetings and emails by focusing here on the stages of interpretation, where sentences are mapped to a conversation ontology, and transformation, where the summary content is selected.
Combining multiple information layers for the automatic generation of indicative meeting abstracts
16 Citations2007Thomas Kleinbauer, Stéphanie Becker +1 more
A summarizer is developed that uses the rich annotations in the AMI corpus, based on the freely available AMI Corpus of 100 hours of recorded meetings, to generate indicative, abstractive summaries of multi-party meetings.