Top Research Papers on Text Summarization

This paper forms the extractive summarization task as a semantic text matching problem, in which a source document and candidate summaries will be matched in a semantic space to create a semantic matching framework.

This research provides a comprehensive survey for the researchers by presenting the different aspects of ATS: approaches, methods, building blocks, techniques, datasets, evaluation methods, and future research directions.

A new approach to collecting abstract data using artificial neural networks (GANs), a class of deep learning models known for their ability to create patterns of real information, shows its promise in paving the way for advanced applications in fields.

DiscoBert extracts sub-sentential discourse units (instead of sentences) as candidates for extractive selection on a finer granularity and outperforms state-of-the-art methods by a significant margin on popular summarization benchmarks compared to other BERT-base models.

Every modern text summarization approach’s workflow and significance are reviewed with the limitations with potential recovery methods, including the feature extraction approaches, datasets, performance measurement techniques, and challenges of the ATS domain, etc.

This paper provides a broad and systematic review of research in the field of text summarization published from 2008 to 2019 and describes the techniques and methods that are often used by researchers as a comparison and means for developing methods.

A set of new metrics are proposed to quantify the entity-level factual consistency of generated summaries and it is shown that the entity hallucination problem can be alleviated by simply filtering the training data.

This article provides a comprehensive literature survey on different seq2seq models for abstractive text summarization from the viewpoint of network structures, training strategies, and summary generation algorithms.

A topic-aware extractive and abstractive summarization model named T-BERTSum, based on Bidirectional Encoder Representations from Transformers (BERTs), which achieves new state-of-the-art results while generating consistent topics compared with the most advanced method.

A systematic approach is proposed that first constructs classifiers to identify helpful reviews and then classifies the sentences in the helpful reviews into six hotel features, and the sentiment polarities of sentences are analyzed to generate the review summaries.

A novel context-aware multi-view summarization network to summarize context-enhanced visual region information from multiple views and designs an adaptive gating self-attention module to extract representations of visual regions and words.

Comparative performance assessment of large language models identified ChatGPT-4 as the best-adapted model across a diverse set of clinical text summarization tasks, and it outperformed 10 medical experts in a reader study.

This paper proposes a Detect-to-Summarize network (DSNet) framework for supervised video summarization that contains anchor-based and anchor-free counterparts, and provides a dense sampling of temporal interest proposals with multi-scale intervals that accommodate interest variations in length.

We present MLSUM, the first large-scale MultiLingual SUMmarization dataset. Obtained from online newspapers, it contains 1.5M+ article/summary pairs in five different languages – namely, French, German, Spanish, Russian, Turkish. Together with English news articles from the popular CNN/Daily mail dataset, the collected data form a large scale multilingual dataset which can enable new research directions for the text summarization community. We report cross-lingual comparative analyses based on state-of-the-art systems. These highlight existing biases which motivate the use of a multi-lingual d...

It is argued that for some highly structured and recurring events, such as sports, it is better to use more sophisticated techniques to summarize the relevant tweets, and a solution based on learning the underlying hidden state representation of the event via Hidden Markov Models is given.

This paper proposes a novel framework named Document Summarization based on Data Reconstruction (DSDR), which generates a summary which consist of those sentences that can best reconstruct the original document.

A fact-aware summarization model FASum is proposed to extract and integrate factual relations into the summary generation process via graph attention and a factual corrector model FC is designed to automatically correct factual errors from summaries generated by existing systems.

ive summarization models are commonly trained using maximum likelihood estimation, which assumes a deterministic (one-point) target distribution in which an ideal model will assign all the probability mass to the reference summary. This assumption may lead to performance degradation during inference, where the model needs to compare several system-generated (candidate) summaries that have deviated from the reference summary. To address this problem, we propose a novel training paradigm which assumes a non-deterministic distribution so that different candidate summaries are assigned probability...

Hepos, a novel efficient encoder-decoder attention with head-wise positional strides to effectively pinpoint salient information from the source is proposed, able to process ten times more tokens than existing models that use full attentions.

An overview of the existing strategies proposed for MVS is presented, including their advantages and drawbacks, and the genericsteps in MVS, such as the pre-processing of video data, feature extraction, and post-processing followed by summary generation are described.

It is found instruction tuning, not model size, is the key to the LLM’s zero-shot summarization capability and summaries are judged to be on par with human written summaries.

This paper proposes a retrieval-based neural source code summarization approach where the neural model is enhanced with the most similar code snippets retrieved from the training set, and the experimental results show that the proposed approach can improve the state-of-the-art methods.

This work proposes a post-editing corrector module that is able to correct factual errors in summaries generated by other neural summarization models and outperforms previous models on factual consistency evaluation on the CNN/DailyMail dataset.

This work explores the Transformer model that uses a self-attention mechanism and has shown to be effective in capturing long-range dependencies in source code summarization, and shows that despite the approach is simple, it outperforms the state-of-the-art techniques by a significant margin.

A reconstructive sequence-graph network (RSGN) is proposed to encode the frames and shots as sequence and graph hierarchically, where the frame-level dependencies are encoded by long short-term memory (LSTM), and the shot- level dependencies are captured by the graph convolutional network (GCN).

It is shown that not only do humans overwhelmingly prefer GPT-3 summaries, prompted using only a task description, but these also do not suffer from common dataset-specific issues such as poor factuality.

It is indicated that text-to-text pre-training in the form of T5 enables simple, end- to-end transformer based models to outperform pipelined neural architectures tailored for data-to/text generation, as well as alternatives such as BERT and GPT-2.

This paper presents a heterogeneous graph-based neural network for extractive summarization (HETERSUMGRAPH), which contains semantic nodes of different granularity levels apart from sentences that act as the intermediary between sentences and enrich the cross-sentence relations.

Experimental results show unique challenges in dialogue summarization, such as spoken terms, special discourse structures, coreferences and ellipsis, pragmatics and social common sense, which require specific representation learning technologies to better deal with.

Abstract Recent advances in large language models (LLMs) have demonstrated remarkable successes in zero- and few-shot performance on various downstream tasks, paving the way for applications in high-stakes domains. In this study, we systematically examine the capabilities and limitations of LLMs, specifically GPT-3.5 and ChatGPT, in performing zero-shot medical evidence summarization across six clinical domains. We conduct both automatic and human evaluations, covering several dimensions of summary quality. Our study demonstrates that automatic metrics often do not strongly correlate with the ...

A general and extensible guided summarization framework that can effectively take different kinds of external guidance as input is proposed and demonstrated, and how different types of guidance generate qualitatively different summaries is demonstrated, lending a degree of controllability to the learned models.

A neural abstractive multi-document summarization (MDS) model which can leverage well-known graph representations of documents, to more effectively process multiple input documents and produce abstractive summaries is developed.

SimCLS can bridge the gap between the learning objective and evaluation metrics resulting from the currently dominated sequence-to-sequence learning framework by formulating text generation as a reference-free evaluation problem assisted by contrastive learning.

This paper proposed an approach for mining product feature and opinion based on the consideration of syntactic information and semantic information and shows that the approach is more flexible and effective.

A wide variety of NLP applications, such as machine translation, summarization, and dialog, involve text generation. One major challenge for these applications is how to evaluate whether such generated texts are actually fluent, accurate, or effective. In this work, we conceptualize the evaluation of generated text as a text generation problem, modeled using pre-trained sequence-to-sequence models. The general idea is that models trained to convert the generated text to/from a reference output or the source text will achieve higher scores when the generated text is better. We operationalize th...

This paper investigates the use few-shot training with the very large GPT (Generative Pre-trained Transformer) Codex model, and finds evidence suggesting that one can significantly surpass state-of-the-art models for code-summarization, leveraging project-specific training.

This work proposes pre-training large Transformer-based encoder-decoder models on massive text corpora with a new self-supervised objective, PEGASUS, and demonstrates it achieves state-of-the-art performance on all 12 downstream datasets measured by ROUGE scores.

This survey, the first of its kind, systematically overviews the recent deep-learning-based MDS models and proposes a novel taxonomy to summarize the design strategies of neural networks and conduct a comprehensive summary of the state of the art.

A method to produce abstractive summaries of long documents that exceed several thousand words via neural abstractive summarization via transformer language model is presented, finding that transformers are ranked highly for coherence and fluency, but purely extractive methods score higher for informativeness and relevance.

It is found that the contrastive learning framework consistently produces more factual summaries than strong comparisons with post error correction, entailment-based reranking, and unlikelihood training, according to QA-based factuality evaluation.