Extraction of Keyphrases from Text: Evaluation of Four Algorithms
arXiv (Cornell University)Published 8 December 2002Open access
Peter D. Turney
Citations62
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
An empirical evaluation of four algorithms for automatically extracting keywords and keyphrases from documents finds NRCs Extractor yields the best match with the manually generated keyphRases.
Abstract
This report presents an empirical evaluation of four algorithms for automatically extracting keywords and keyphrases from documents. The four algorithms are compared using five different collections of documents. For each document, we have a target set of keyphrases, which were generated by hand. The target keyphrases were generated for human readers; they were not tailored for any of the four keyphrase extraction algorithms. Each of the algorithms was evaluated by the degree to which the algorithm's keyphrases matched the manually generated keyphrases. The four algorithms were (1) the AutoSummarize feature in Microsoft's Word 97, (2) an algorithm based on Eric Brill's part-of-speech tagger, (3) the Summarize feature in Verity's Search 97, and (4) NRC's Extractor algorithm. For all five document collections, NRC's Extractor yields the best match with the manually generated keyphrases.
Keywords
Computer Science
Program electronic library and information systemsAn algorithm for suffix stripping
8,136 Citations1980Martin Porter
An algorithm for suffix stripping is described, which has been implemented as a short, fast program in BCPL, and performs slightly better than a much more elaborate system with which it has been compared.
IBM Journal of Research and DevelopmentThe Automatic Creation of Literature Abstracts
3,211 Citations1958H. P. Luhn
In the exploratory research described, the complete text of an article in machine-readable form is scanned by an IBM 704 data-processing machine and analyzed in accordance with a standard program.
Journal of the ACMNew Methods in Automatic Extracting
1,469 Citations1969H. P. Edmundson
New methods of automatically extracting documents for screening purposes, i.e. the computer selection of sentences having the greatest potential for conveying to the reader the substance of the document, indicate that the three newly proposed components dominate the frequency component in the production of better extracts.
A simple rule-based part of speech tagger
1,418 Citations1992Eric Brill
This work presents a simple rule-based part of speech tagger which automatically acquires its rules and tags with accuracy comparable to stochastic taggers, demonstrating that the stochastics method is not the only viable method for part ofspeech tagging.
A trainable document summarizer
1,349 Citations1995Julian Kupiec, Jan Pedersen +1 more
The trends in the results are in agreement with those of Edmundson who used a subjectively weighted combination of features as opposed to training the feature weights using a corpus, which suggests that even shorter extracts may be useful indicative summmies.
Information Processing & ManagementConstructing literature abstracts by computer: Techniques and prospects
385 Citations1990Chris D. Paice
The question of how to achieve proper balance in auto-abstracts is a problem which has hardly been addressed hitherto; it appears that assignment of selected textual material into ‘abstract-frames’ may offer a solution.
Information Processing & ManagementAutomatic condensation of electronic publications by sentence selection
368 Citations1995Ronald Brandow, Karl Mitze +1 more
A system that performs domain-independent automatic condensation of news from a large commercial news service encompassing 41 different publications is described, with the result that the lead-based summaries outperformed the “intelligent” summaries significantly.
Evaluating and optimizing autonomous text classification systems
312 Citations1995David Lewis
This work shows how to define what constitutes good effectiveness for binary text classification systems, tune the systems to achieve the highest possible effectiveness, and estimate how the effectiveness changes as new data is processed.
ScholarlyCommons (University of Pennsylvania)A corpus-based approach to language learning
280 Citations1993Eric Brill
A learning algorithm is described that takes a small structurally annotated corpus of text and a larger unannotated corpus as input and automatically learns how to assign accurate structural descriptions to sentences not in the training corpus.
ScienceAutomatic Analysis, Theme Generation, and Summarization of Machine-Readable Texts
270 Citations1994Gerard Salton, James Allan +2 more
Methods are given for determining text themes, traversing texts selectively, and extracting summary statements that reflect text content in arbitrary subject areas in accordance with user needs.
The use of phrases and structured queries in information retrieval
265 Citations1991W. Bruce Croft, Howard R. Turtle +1 more
The results show that using phrases in this way can improve performance, and that phrases that are automatically extracted from a natural language query perform nearly as well as manually selected phrases.
Cornell University eBooksExperiments in Automatic Phrase Indexing For Document Retrieval: A Comparison of Syntactic and Non-Syntactic Methods
185 Citations1987Joel L. Fagan
A 78-year-old man with palpitations, dizziness, and presyncope was referred to the authors' institution for evaluation and a diagnosis of right ventricular dysplasia was made, and the patient underwent an electrophysiological study to evaluate hisPalpitations and Presyncope.
The identification of important concepts in highly structured technical papers
150 Citations1993Chris D. Paice, Paul A. Jones
This paper describes an approach in which the indexing and abstracting tasks are effectively combined, applicable to highly structured empirical research papers, whose content can be organised using a semantic frame.
Intelligent Data AnalysisCompound key word generation from document databases using a hierarchical clustering ART model
48 Citations1997Ana Muñoz
This paper addresses the specific problem of creating semantic term associations from a text database by using a hierarchical model made up of Fuzzy Adaptive Resonance Theory (ART) neural networks to cluster isolated words into semantic classes.
Journal of Artificial Intelligence ResearchWrap-Up: a Trainable Discourse Module for Information Extraction
44 Citations1994Stephen Soderland, W. Lehnert
Wrap-Up is a trainable IE discourse component that makes intersentential inferences and identifies logical relations among information extracted from the text, and automatically decides what classifiers are needed, and derives the feature set for each classifier automatically.
e-space (Manchester Metropolitan University)The application of linguistic processing to automatic abstract generation
43 Citations1997Frances Johnson, Chris D. Paice +2 more
The text processing which was necessary to identify anaphors so that they may be utilised in the enhancement of the sentence selection criteria suggests that the key concepts are indeed identified but the abstracts are too long.
Syntactic approaches to automatic book indexing
37 Citations1988Gerard Salton
Some approaches are given for the automatic construction of back-of-book indexes using a syntactic analysis of the available texts, followed by the identification of nominal constructions, the assignment of importance weights to the term phrases, and the choice of phrases as indexing units.
IEEE ExpertA unified approach to automatic indexing and information retrieval
30 Citations1993A. Ginsberg
WorldViews, an experimental system that unites as many aspects of information organization as possible around a simple, familiar, yet versatile knowledge representation framework, is discussed.
Journal of the American Society for Information ScienceA statistical learning approach to automatic indexing of controlled index terms
21 Citations1997Chi‐Hong Leung, Wing‐Kay Kan
Experimental results have shown that the statistical learning approach and the learning feedback technique are practical means to automatic indexing of controlled index terms.
A production rule system for message summarization
14 Citations1997Elaine Marsh, Henry Hamburger +1 more
Extraction of index words from manuals
13 Citations1997Hiroshi Nakagawa
An automatic extraction method of index words from manuals, which optimizes Rijsbergen's E calculated from the recall and precision is described and experimentally evaluated.
Development of a document summarization system for effective information services
13 Citations1997Dong-Hyun Jang, Sung Hyun Myaeng
A system that constructs a summary by extracting sentences that are likely to represent the main theme of a document by using a probabilistic model that takes into account lexical and statistical information obtained from a document corpus.