Invariant Feature Extraction and Classification in Kernel Spaces

Published 29 November 1999

Sebastian Mika, Gunnar Rätsch, Jason Weston, Bernhard Schölkopf, Alex Smola, Klaus‐Robert Müller

Citations174

TL;DR

Employing a unified framework in terms of a nonlinear variant of the Rayleigh coefficient, this work proposes non-linear generalizations of Fisher's discriminant and oriented PCA using Support Vector kernel functions.

Abstract

We incorporate prior knowledge to construct nonlinear algorithms for invariant feature extraction and discrimination. Employing a unified framework in terms of a nonlinear variant of the Rayleigh coefficient, we propose non-linear generalizations of Fisher&apos;s discriminant and oriented PCA using Support Vector kernel functions. Extensive simulations show the utility of our approach.

Keywords

Computer ScienceMathematics

Journal of Computer and System SciencesA Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting

20,320 Citations1997Yoav Freund, Robert E. Schapire

Choice Reviews OnlineNeural networks for pattern recognition

18,687 Citations1994

This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition, and is designed as a text, with over 100 exercises, to benefit anyone involved in the fields of neural computation and pattern recognition.

A training algorithm for optimal margin classifiers

11,594 Citations1992Bernhard E. Boser, Isabelle Guyon +1 more

A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented, applicable to a wide variety of the classification functions, including Perceptrons, polynomials, and Radial Basis Functions.

Elsevier eBooksIntroduction to Statistical Pattern Recognition

11,121 Citations1990Keinosuke Fukunaga

A system to determine the sex of fish from their length, as accurately as possible, using a set of labelled data, together with the class label (male or female), as a training set.

Neural ComputationNonlinear Component Analysis as a Kernel Eigenvalue Problem

8,078 Citations1998Bernhard Schölkopf, Alexander J. Smola +1 more

A new method for performing a nonlinear form of principal component analysis by the use of integral operator kernel functions is proposed and experimental results on polynomial feature extraction for pattern recognition are presented.

International Conference on Neural Information ProcessingAdvances in kernel methods: support vector learning

5,815 Citations1999Bernhard Schölkopf, Christopher J. C. Burges +1 more

Support vector machines for dynamic reconstruction of a chaotic system, Klaus-Robert Muller et al pairwise classification and support vector machines, Ulrich Kressel.

Neural ComputationFast Learning in Networks of Locally-Tuned Processing Units

4,219 Citations1989John Moody, Christian J. Darken

This work proposes a network architecture which uses a single internal layer of locally-tuned processing units to learn both classification tasks and real-valued function approximations (Moody and Darken 1988).

Fisher discriminant analysis with kernels

2,661 Citations2003Mika Sirén, Gunnar Rätsch +3 more

A non-linear classification technique based on Fisher's discriminant which allows the efficient computation of Fisher discriminant in feature space and large scale simulations demonstrate the competitiveness of this approach.

Journal of the American Statistical AssociationRegularized Discriminant Analysis

2,238 Citations1989Jerome H. Friedman

Alternatives to the usual maximum likelihood estimates for the covariance matrices are proposed, characterized by two parameters, the values of which are customized to individual situations by jointly minimizing a sample-based estimate of future misclassification risk.

Medical Entomology and ZoologyPrincipal Component Neural Networks: Theory and Applications

888 Citations1996Konstantinos Diamantaras, Sun‐Yuan Kung

A review of Linear Algebra, Principal Component Analysis, and VLSI Implementation.