Efficient Feature Subset Selection for Support Vector Machines

Published 1 January 2001

Matthias Heiler, Daniel Cremers, Christoph Schnörr

Citations8

TL;DR

This paper presents a method for selecting high-quality feature sets and provides evidence that it selects high- quality feature sets at a fraction of the costs of classical methods.

Abstract

Support vector machines can be regarded as algorithms for compressing information about class membership into a few support vectors with clear geometric interpretation. It is tempting to use this compressed information to select the most relevant input features. In this paper we present a method for doing so and provide evidence that it selects high-quality feature sets at a fraction of the costs of classical methods.

Keywords

Computer Science

ACM Transactions on Intelligent Systems and TechnologyLIBSVM

41,340 Citations2011Chih-Chung Chang, Chih‐Jen Lin

Issues such as solving SVM optimization problems theoretical convergence multiclass classification probability estimates and parameter selection are discussed in detail.

The Nature of Statistical Learning Theory

39,279 Citations1995Vladimir Vapnik

Elements of Information Theory

37,533 Citations2001Thomas M. Cover, Joy A. Thomas

Artificial IntelligenceWrappers for feature subset selection

8,925 Citations1997Ron Kohavi, George H. John

The wrapper method searches for an optimal feature subset tailored to a particular algorithm and a domain and compares the wrapper approach to induction without feature subset selection and to Relief, a filter approach tofeature subset selection.

Stochastic modelling and applied probabilityA Probabilistic Theory of Pattern Recognition

3,277 Citations1996Luc Devroye, László Györfi +1 more

The Bayes Error and Vapnik-Chervonenkis theory are applied as guide for empirical classifier selection on the basis of explicit specification and explicit enforcement of the maximum likelihood principle.

Elsevier eBooksIrrelevant Features and the Subset Selection Problem

2,374 Citations1994George H. John, Ron Kohavi +1 more

A method for feature subset selection using cross-validation that is applicable to any induction algorithm is described, and experiments conducted with ID3 and C4.5 on artificial and real datasets are discussed.

Pattern Recognition LettersAdaptive floating search methods in feature selection

309 Citations1999Petr Somol, Pavel Pudil +2 more

A new suboptimal search strategy for feature selection that represents a more sophisticated version of “classical” floating search algorithms and facilitates finding a solution even closer to the optimal one.

Knowledge Discovery and Data MiningFeature subset selection using the wrapper method: overfltting and dynamic search space topology

262 Citations1995Ron Kohavi, Dan Sommerfield

Initial positive changes to health behaviours were observed post BC diagnosis, except healthy body weight, but maintenance of such changes over the long-term was poor, and BCSs may benefit from additional advice and support to make healthy lifestyle choices throughout survivorship.

IEEE Transactions on Systems Man and CyberneticsOn the Possible Orderings in the Measurement Selection Problem

256 Citations1977Thomas M. Cover, Jan Van Campenhout

It is shown that for any ordering on the probability of error as a function of the subset of measurements (subject to an obvious set monotonicity condition), there exists a multivariate normal two-hypothesis problem N(K) versus N(¿¿,K) that exhibits this ordering.

UCL Discovery (University College London)Image representations for object detection using kernel classifiers

37 Citations2000E Evgeniou, Massimiliano Pontil +2 more

This paper presents experimental comparisons of various image representations for object detection using kernel classifiers, and presents a feature selection method using SVMs, and shows experimental results.