A Feature Weight Adjustment Algorithm for Document Categorization

Published 1 January 2000

Shrikanth Shankar, George Karypis

Citations27

TL;DR

This paper presents a fast iterative feature weight adjustment algorithm for the linear-complexity centroid based classi cation algorithm that substantially outperforms Rocchio and Widrow-Ho and is competitive with SVM.

Abstract

In recent years we have seen a tremendous growth in the volume of text documents available on the Internet, digital libraries, news sources, and company-wide intra-nets. Automatic text categorization, which is the task of assigning text documents to pre-specified classes (topics or themes) of documents, is an important task that can help both in organizing as well as in finding information on these huge resources. In this paper we present a fast iterative feature weight adjustment algorithm for the linear-complexity centroid based classification algorithm. Our algorithm uses a measure of the discriminating power of each term to gradually adjust the weights of all features concurrently. We experimentally evaluate our algorithm on the Reuters-21578 and OHSUMED document collection and compare it against a variety of other categorization algorithms. Our experiments show that feature weight adjustment improves the performance of the centroid-based classifier by 2%-5%, substantially outperforms Rocchio and Widrow-Hoff and is competitive with SVM.

Keywords

Computer Science

Choice Reviews OnlineGenetic algorithms in search, optimization, and machine learning

49,278 Citations1989

This book brings together the computer techniques, mathematical tools, and research results that will enable both students and practitioners to apply genetic algorithms to problems in many fields.

Lecture notes in computer scienceText categorization with Support Vector Machines: Learning with many relevant features

7,925 Citations1998Thorsten Joachims

SVMs achieve substantial improvements over the currently best performing methods and behave robustly over a variety of di-erent learning tasks, eliminating the need for manual parameter tuning.

Adaptive Signal Processing

5,121 Citations1991

This volume discusses three recently developed general methodologies for designing signal predictors under nonclassical operating conditions, namely the robust predictor, the high-speed Levinson modeling, and the approximate conditional mean nonlinear predictor.

A Comparative Study on Feature Selection in Text Categorization

4,766 Citations1997Yiming Yang, Jan Pedersen

DF thresholding, the simplest method with the lowest cost in computation, can be reliably used instead of IG or CHI when the computation of these measures are too expensive, and strong correlations between the DF, IG and CHI values of a term are found.

A re-examination of text categorization methods

2,660 Citations1999Yiming Yang, Xin Liu

The results show that SVM, kNN and LLSF signi cantly outperform NNet and NB when the number of positive training instances per category are small, and that all the methods perform comparably when the categories are over 300 instances.

View PDF

TechnometricsMachine Learning, Neural and Statistical Classification

2,188 Citations1995Bill Fulkerson, D. Michie +2 more

Computer systems that learn: classification and prediction methods from statistics, neural nets, machine learning, and expert systems

906 Citations1991Sholom M. Weiss, Casimir A. Kulikowski

This book discusses how to Estimate the True Performance of a Learning System, and the Importance of Unbiased Error Rate Estimation, and Machine Learning: Easily Understood Decision Rules.

Distributional clustering of words for text classification

683 Citations1998Lee D. Baker, Andrew Kachites McCallum

This paper describes the application of Distributional Clustering to document classi(cid:12)cation and shows that it can reduce the feature dimensionality by three orders of magnitude and lose only 2% accuracy, better than Latent Semantic In-dexing, class-based clustering, feature selection by mutual information, or Markov-blanket-based feature selection.

Lecture notes in computer scienceText Categorization Using Weight Adjusted k-Nearest Neighbor Classification

353 Citations2001Eui-Hong Han, George Karypis +1 more

A Weight Adjusted k-Nearest Neighbor (WAKNN) classification that learns feature weights based on a greedy hill climbing technique and two performance optimizations of WAKNN that improve the computational performance by a few orders of magnitude, but do not compromise on the classification quality.

View PDF

Using a generalized instance set for automatic text categorization

188 Citations1998Wai Lam, Chao Yang Ho

This work proposes a new technique known as the generalized instance set (GIS) algorithm by unifying the strengths of k-NN and linear classifiers and adapting to characteristics of text categorization problems.