Extractive Text Summarization
This project highlights the utilization of multiple extractive text summarization techniques, including Word Frequency, Lex, Luhn, Kl Summarizer, GPT-2 and BERT, and demonstrates the efficacy of these techniques in generating summaries and assess their quality by comparing them against summaries produced by humans using the specified scoring metrics.
Abstract
The fields of artificial intelligence (AI), machine learning (ML), and data science have grown significantly over the past ten years, opening up a wide range of opportunities in sectors as varied as healthcare, banking, and transportation. Notably, there have been substantial developments in the field of Natural Language Processing (NLP), which is a subfield of AI and ML. NLP involves the machine-based processing and understanding of human language. Among its various applications, text summarization holds prominence as it enables machines to condense lengthy texts into concise summaries. This project highlights the utilization of multiple extractive text summarization techniques, including Word Frequency, Lex, Luhn, Kl Summarizer, GPT-2 and BERT. The resultant extractive summaries are then evaluated against human-generated summaries using three distinct scoring methods: Rouge Score, BERT Score, and Mover Score. Through this project, we demonstrate the efficacy of these techniques in generating summaries and assess their quality by comparing them against summaries produced by humans using the specified scoring metrics.