A Probabilistic Approach to Feature Selection for Multi-class Text Categorization

Lecture notes in computer sciencePublished 1 January 2007

Ke Wu, Bao‐Liang Lu, Masao Uchiyama, Hitoshi Isahara

DOI10.1007/978-3-540-72383-7_153

Source URL

Citations9

SJR quartileQ2

SJR score0.35

SNIP0.55

TL;DR

Experiments show that the proposed probabilistic approach to feature selection for multi-class text categorization can yield better performance than information gain and i¾?-square, which are two well-known feature selection methods.

Abstract

In this paper, we propose a probabilistic approach to feature selection for multi-class text categorization. Specifically, we regard document class and occurrence of each feature as events, calculate the probability of occurrence of each feature by the theorem on the total probability and utilize the values as a ranking criterion. Experiments on Reuters-2000 collection show that the proposed method can yield better performance than information gain and χ-square, which are two well-known feature selection methods.

Keywords

Computer Science

ACM Transactions on Intelligent Systems and TechnologyLIBSVM

41,340 Citations2011Chih-Chung Chang, Chih‐Jen Lin

Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.

The Nature of Statistical Learning Theory

39,279 Citations1995Vladimir Vapnik

The Nature of Statistical Learning Theory

10,566 Citations2000Vladimir Vapnik

Machine LearningGene Selection for Cancer Classification using Support Vector Machines

9,839 Citations2002Isabelle Guyon, Jason Weston +2 more

This paper proposes a new method of gene selection utilizing Support Vector Machine methods based on Recursive Feature Elimination (RFE), and demonstrates experimentally that the genes selected yield better classification performance and are biologically relevant to cancer.

View PDF

ACM Computing SurveysMachine learning in automated text categorization

7,899 Citations2002Fabrizio Sebastiani

This survey discusses the main approaches to text categorization that fall within the machine learning paradigm and discusses in detail issues pertaining to three different problems, namely, document representation, classifier construction, and classifier evaluation.

View PDF

A Comparative Study on Feature Selection in Text Categorization

4,766 Citations1997Yiming Yang, Jan Pedersen

DF thresholding, the simplest method with the lowest cost in computation, can be reliably used instead of IG or CHI when the computation of these measures are too expensive, and strong correlations between the DF, IG and CHI values of a term are found.

In Defense of One-Vs-All Classification

1,387 Citations2004Ryan Rifkin, Aldebaro Klautau

It is argued that a simple "one-vs-all" scheme is as accurate as any other approach, assuming that the underlying binary classifiers are well-tuned regularized classifiers such as support vector machines.

Pattern RecognitionHierarchical classification and feature reduction for fast face detection with support vector machines

181 Citations2003Bernd Heisele, T. Serre +2 more

Experiments with a face detection system show that combining feature reduction with hierarchical classification leads to a speed-up by a factor of 335 with similar classification performance.

WIT transactions on information and communication technologiesFeature Selection Using Support Vector Machines

72 Citations2002Janez Brank, Marko Grobelnik +2 more

Experimental results indicate that, at the same level of vector sparsity, feature selection based on SVM normals yields better classification performance than odds ratio or information gainbased feature selection when linear SVM classifiers are used.

Fast recognition of multi-view faces with feature selection

31 Citations2005Zhigang Fan, Bao‐Liang Lu

This work integrates the two tasks, feature selection and multiclass classification, into a single consistent framework and effectively realize the goal of discriminative feature selection, and shows that the classification process can be made faster without degrading the generalization performance.