Kernel PCA and De-Noising in Feature Spaces

Published 1 December 1998

Sebastian Mika, Bernhard Schölkopf, Alex Smola, Klaus‐Robert Müller, Matthias Scholz, Gunnar Rätsch

Citations828

TL;DR

This work presents ideas for finding approximate pre-images, focusing on Gaussian kernels, and shows experimental results using these pre- images in data reconstruction and de-noising on toy examples as well as on real world data.

Abstract

Kernel PCA as a nonlinear feature extractor has proven powerful as a preprocessing step for classification algorithms. But it can also be considered as a natural generalization of linear principal component analysis. This gives rise to the question how to use nonlinear features for data compression, reconstruction, and de-noising, applications common in linear PCA. This is a nontrivial task, as the results provided by kernel PCA live in some high dimensional feature space and need not have pre-images in input space. This work presents ideas for finding approximate pre-images, focusing on Gaussian kernels, and shows experimental results using these pre-images in data reconstruction and de-noising on toy examples as well as on real world data.

Keywords

Computer ScienceEngineering

A training algorithm for optimal margin classifiers

11,594 Citations1992Bernhard E. Boser, Isabelle Guyon +1 more

A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented, applicable to a wide variety of the classification functions, including Perceptrons, polynomials, and Radial Basis Functions.

IEEE Transactions on Signal ProcessingMatching pursuits with time-frequency dictionaries

9,115 Citations1993Stéphane Mallat, Zhifeng Zhang

The authors introduce an algorithm, called matching pursuit, that decomposes any signal into a linear expansion of waveforms that are selected from a redundant dictionary of functions, chosen in order to best match the signal structures.

Neural ComputationNonlinear Component Analysis as a Kernel Eigenvalue Problem

8,078 Citations1998Bernhard Schölkopf, Alexander J. Smola +1 more

A new method for performing a nonlinear form of principal component analysis by the use of integral operator kernel functions is proposed and experimental results on polynomial feature extraction for pattern recognition are presented.

Medical Entomology and ZoologyPrincipal Component Neural Networks: Theory and Applications

888 Citations1996Konstantinos Diamantaras, Sun‐Yuan Kung

A review of Linear Algebra, Principal Component Analysis, and VLSI Implementation.

International Conference on Machine LearningSimplified support vector decision rules

429 Citations1996Christopher J. C. Burges

The results show that the method can decrease the computational complexity of the decision rule by a factor of ten with no loss in generalization perfor mance making the SVM test speed com petitive with that of other methods.

Informatik aktuellFast Approximation of Support Vector Kernel Expansions, and an Interpretation of Clustering as Approximation in Feature Spaces

79 Citations1998Bernhard Schölkopf, Phil Knirsch +2 more

This work considers the problem of reducing the computational complexity of evaluating these expansions by approximating them using fewer terms and points out a connection between clustering and approximation in reproducing kernel Hilbert spaces generated by a particular class of kernels.