Local distance preservation in the GP-LVM through back constraints
Published 1 January 2006
Neil D. Lawrence, Joaquin Quiñonero-Candela
Citations221
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper provides an overview of dimensionality reduction techniques, placing the emphasis on the kind of distance relation preserved, and shows how the GP-LVM can be generalized, through back constraints, to additionally preserve local distances.
Abstract
S.513-520
Keywords
Computer Science
ScienceNonlinear Dimensionality Reduction by Locally Linear Embedding
15,035 Citations2000Sam T. Roweis, Lawrence K. Saul
Locally linear embedding (LLE) is introduced, an unsupervised learning algorithm that computes low-dimensional, neighborhood-preserving embeddings of high-dimensional inputs that learns the global structure of nonlinear manifolds.
ScienceA Global Geometric Framework for Nonlinear Dimensionality Reduction
13,740 Citations2000Joshua B. Tenenbaum, Vin de Silva +1 more
An approach to solving dimensionality reduction problems that uses easily measured local metric information to learn the underlying global geometry of a data set and efficiently computes a globally optimal solution, and is guaranteed to converge asymptotically to the true structure.
Neural ComputationNonlinear Component Analysis as a Kernel Eigenvalue Problem
8,078 Citations1998Bernhard Schölkopf, Alexander J. Smola +1 more
A new method for performing a nonlinear form of principal component analysis by the use of integral operator kernel functions is proposed and experimental results on polynomial feature extraction for pattern recognition are presented.
Journal of the Royal Statistical Society Series B (Statistical Methodology)Probabilistic Principal Component Analysis
3,730 Citations1999Michael E. Tipping, Chris Bishop
It is demonstrated how the principal axes of a set of observed data vectors may be determined through maximum likelihood estimation of parameters in a latent variable model that is closely related to factor analysis.
IEEE Transactions on ComputersA Nonlinear Mapping for Data Structure Analysis
3,381 Citations1969John W. Sammon
An algorithm for the analysis of multivariate data is presented along with some experimental results that is based upon a point mapping of N L-dimensional vectors from the L-space to a lower-dimensional space such that the inherent data "structure" is approximately preserved.
Neural ComputationMixtures of Probabilistic Principal Component Analyzers
1,903 Citations1999Michael E. Tipping, Chris Bishop
PCA is formulated within a maximum likelihood framework, based on a specific form of gaussian latent variable model, which leads to a well-defined mixture model for probabilistic principal component analyzers, whose parameters can be determined using an expectation-maximization algorithm.
Stochastic Neighbor Embedding
1,461 Citations2002Geoffrey E. Hinton, Sam T. Roweis
This probabilistic framework makes it easy to represent each object by a mixture of widely separated low-dimensional images, which allows ambiguous objects, like the document count vector for the word "bank", to have versions close to the images of both "river" and "finance" without forcing the image of outdoor concepts to be located close to those of corporate concepts.
Neural ComputationGTM: The Generative Topographic Mapping
1,389 Citations1998Chris Bishop, Markus Svensén +1 more
A form of nonlinear latent variable model called the generative topographic mapping, for which the parameters of the model can be determined using the expectation-maximization algorithm, is introduced.
ScienceThe "Wake-Sleep" Algorithm for Unsupervised Neural Networks
1,117 Citations1995Geoffrey E. Hinton, Peter Dayan +2 more
An unsupervised learning algorithm for a multilayer network of stochastic neurons is described, where bottom-up "recognition" connections convert the input into representations in successive hidden layers, and top-down "generative" connections reconstruct the representation in one layer from the representations in the layer above.
IEEE Transactions on Pattern Analysis and Machine IntelligenceGaussian Process Dynamical Models for Human Motion
1,038 Citations2007Jonathan M. Wang, David J. Fleet +1 more
This work marginalize out the model parameters in closed form by using Gaussian process priors for both the dynamical and the observation mappings, which results in a nonparametric model for dynamical systems that accounts for uncertainty in the model.
Probabilistic Non-linear Principal Component Analysis with Gaussian Process Latent Variable Models
948 Citations2005Neil D. Lawrence
ACM Transactions on GraphicsStyle-based inverse kinematics
647 Citations2004Keith Grochow, Steven Martin +2 more
Learning a kernel matrix for nonlinear dimensionality reduction
521 Citations2004Kilian Q. Weinberger, Fei Sha +1 more
This work investigates how to learn a kernel matrix for high dimensional data that lies on or near a low dimensional manifold and shows how to discover a mapping that "unfolds" the underlying manifold from which the data was sampled.
Gaussian Process Dynamical Models
432 Citations2005Jianguo Wang, Aaron Hertzmann +1 more
This paper marginalize out the model parameters in closed-form, using Gaussian Process (GP) priors for both the dynamics and the observation mappings, resulting in a nonparametric model for dynamical systems that accounts for uncertainty in the model.
Priors for people tracking from small training sets
254 Citations2005Raquel Urtasun, David J. Fleet +2 more
It is shown that the SGPLVM sufficiently constrains the problem such that tracking can be accomplished with straightforward deterministic optimization.
Learning Shared Latent Structure for Image Synthesis and Robotic Imitation
181 Citations2005Aaron P. Shon, Keith Grochow +2 more
An algorithm that uses Gaussian process regression to learn common hidden structure shared between corresponding sets of heterogenous observations can learn the nonlinear mapping between corresponding views of objects, filling in missing data as needed to synthesize novel views.
Nuclear Instruments and Methods in Physics Research Section A Accelerators Spectrometers Detectors and Associated EquipmentBayesian neural networks and density networks
178 Citations1995David Mackay
This paper reviews the Bayesian approach to learning in neural networks, then introduces a new adaptive model, the density network, a neural network for which target outputs are provided, but the inputs are unspecified.
Neural Computing and ApplicationsFeed-forward neural networks and topographic mappings for exploratory data analysis
122 Citations1996David Lowe, Michael E. Tipping
A feed-forward neural network is utilised to effect a topographic, structure-preserving, dimension-reducing transformation of the data, with an additional facility to incorporate different degrees of associated subjective information.
The Vocal Joystick
23 Citations2006Jeffrey A. Bilmes, James Malkin +9 more
This paper describes the engine system design, engine optimization, and user-interface improvements, and outline some of the signal processing and pattern recognition modules that were successful and present new results comparing the vocal joystick with a state-of-the-art eye tracking pointing device.
Large margin non-linear embedding
10 Citations2005Alexander Zien, Joaquin Quiñonero Candela
This work proposes reversing the process of classification methods: for fixed decision boundaries, the authors "learn" the location of the data, and achieves this by combining an entropy-based embedding method with anropy-based version of semi-supervised logistic regression.
MIT Press eBooksAdvances in Neural Information Processing Systems 18: Proceedings of the 2005 Conference
10 Citations2006Y Weiss, Bernhard Schölkopf +1 more