KP-Miner: A Simple System for Effective Keyphrase Extraction

Published 1 November 2006

Samhaa R. El-Beltagy

DOI10.1109/innovations.2006.301948

Citations24

TL;DR

A simple, yet effective algorithm (KP-Miner) for achieving automatic key phrase extraction is presented and the results are compared to two existing sophisticated machine learning based automatic keyphrase extraction systems.

Abstract

Automatic key phrase extraction has many important applications including but not limited to summarization, cataloging/indexing, feature extraction for clustering and classification, and data mining. This paper presents a simple, yet effective algorithm (KP-Miner) for achieving this task. The result of an experiment carried out to investigate the effectiveness of this algorithm is also presented. In this experiment the devised algorithm is applied to six different datasets consisting of 481 documents. The results are then compared to two existing sophisticated machine learning based automatic keyphrase extraction systems. The results of this experiment show that the devised algorithm is comparable to both systems

Keywords

Computer Science

Information Processing & ManagementTerm-weighting approaches in automatic text retrieval

9,532 Citations1988Gerard Salton, Chris Buckley

This paper summarizes the insights gained in automatic term weighting, and provides baseline single term indexing models with which other more elaborate content analysis procedures can be compared.

Program electronic library and information systemsAn algorithm for suffix stripping

8,136 Citations1980Martin Porter

An algorithm for suffix stripping is described, which has been implemented as a short, fast program in BCPL, and performs slightly better than a much more elaborate system with which it has been compared.

KEA

863 Citations1999Ian H. Witten, Gordon W. Paynter +3 more

This paper uses a large test corpus to evaluate Kea’s effectiveness in terms of how many author-assigned keyphrases are correctly identified, and describes the system, which is simple, robust, and publicly available.

View PDF

Information RetrievalLearning Algorithms for Keyphrase Extraction

837 Citations2000Peter D. Turney

The experimental results support the claim that a custom-designed algorithm (GenEx), incorporating specialized procedural domain knowledge, can generate better keyphrases than a general-purpose algorithm (C4.5).

arXiv (Cornell University)Learning to Extract Keyphrases from Text

280 Citations2002Peter D. Turney

The experimental results support the claim that a specialized learning algorithm (GenEx) can generate better keyphrases than a general-purpose learning algorithms (C4.5) and the non-learning algorithms that are used in commercial software (Word 97 and Search 97).

View PDF

International Joint Conference on Artificial IntelligenceAutomatic hypertext keyphrase detection

39 Citations2005Daniel Kelleher, Saturnino Luz

It is found that hyperlink information can improve the effectiveness of automatic keyphrase extraction by 50% and shows promise for the applicability of these techniques to more far-reaching tasks.

A practical system of keyphrase extraction for web pages

29 Citations2005Mo Chen, Jian-Tao Sun +2 more

A regression model was first trained based on a set of human-labeled documents then used to extract keyphrases from new pages automatically, and a method was put forward in this paper in order to evaluate the similarity of phrases.