Mercer kernel-based clustering in feature space

IEEE Transactions on Neural NetworksPublished 1 May 2002

Mark Girolami

DOI10.1109/tnn.2002.1000150

Citations895

TL;DR

It is shown that the eigenvectors of a kernel matrix which defines the implicit mapping provides a means to estimate the number of clusters inherent within the data and a computationally simple iterative procedure is presented for the subsequent feature space partitioning of the data.

Abstract

The article presents a method for both the unsupervised partitioning of a sample of data and the estimation of the possible number of inherent clusters which generate the data. This work exploits the notion that performing a nonlinear data transformation into some high dimensional feature space increases the probability of the linear separability of the patterns within the transformed space and therefore simplifies the associated data structure. It is shown that the eigenvectors of a kernel matrix which defines the implicit mapping provides a means to estimate the number of clusters inherent within the data and a computationally simple iterative procedure is presented for the subsequent feature space partitioning of the data.

Keywords

Computer Science

TechnometricsStatistical Learning Theory

26,913 Citations1999Yuhai Wu, Vladimir Vapnik

Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.

Neural ComputationNonlinear Component Analysis as a Kernel Eigenvalue Problem

8,078 Citations1998Bernhard Schölkopf, Alexander J. Smola +1 more

A new method for performing a nonlinear form of principal component analysis by the use of integral operator kernel functions is proposed and experimental results on polynomial feature extraction for pattern recognition are presented.

IEEE Transactions on ComputersA Projection Pursuit Algorithm for Exploratory Data Analysis

1,635 Citations1974Jerome H. Friedman, John W. Tukey

View PDF

OpenGrey (Institut de l'Information Scientifique et Technique)Nonlinear Discriminant Analysis Using Kernel Functions

231 Citations1999Volker Röth, Volker Steinhage

The presented algorithm allows a simple formulation of the EM-algorithm in terms of kernel functions which leads to a unique concept for unsupervised mixture analysis, supervised discriminant analysis and semi-supervised discriminantAnalysis with partially unlabelled observations in feature spaces.

IEEE Transactions on Pattern Analysis and Machine IntelligenceICA mixture models for unsupervised classification of non-Gaussian classes and automatic context switching in blind signal separation

222 Citations2000Te-Won Lee, Michael S. Lewicki +1 more

The algorithm estimates the density of each class and is able to model class distributions with non-Gaussian structure and can improve classification accuracy compared with standard Gaussian mixture models.

Feature grouping by 'relocalisation' of eigenvectors of the proximity matrix

91 Citations1990G. Scott, H. C. Longuet–Higgins

This work describes a widely applicable method of grouping or clustering image features (such as points, lines, corners, flow vectors and the like) by taking as input a "proximity matrix" H a square, symmetric matrix of dimension N.

Pattern RecognitionMaximum certainty data partitioning

71 Citations2000Stephen Roberts, Richard Everson +1 more

It is shown that minimisation of partition entropy can be used to estimate the number and structure of probable data generators and the notion of cluster analysis in information-theoretic terms is re-considered.

BiometricsAn Algorithm for Euclidean Sum of Squares Classification

67 Citations1977A. D. Gordon, J. Henderson

The problem is reformulated itn non-linear programming terms, and a new algorithm for seeking the minimum sum of squared distances about the g centroids is described, and an efficient hybrid algorithmi is introduced.

Neural ComputationAn Expectation-Maximization Approach to Nonlinear Component Analysis

62 Citations2001Roman Rosipal, Mark Girolami

This work proposes an expectation-maximization approach for performing kernel principal component analysis and shows this to be a computationally efficient method, especially when the number of data points is large.

Stochastic Algorithms for Exploratory Data Analysis: Data Clustering and Data Visualization

14 Citations1998Joachim M. Buhmann

Iterative, EM-type algorithms for data clustering and data visualization are derived on the basis of the maximum entropy principle and allow the data analyst to detect structure in vectorial or relational data.