Kernel independent component analysis
2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03).Published 24 January 2004
Francis Bach, Michael I. Jordan
Citations569
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A class of algorithms for independent component analysis which use contrast functions based on canonical correlations in a reproducing kernel Hilbert space is presented, showing that these algorithms outperform many of the presently known algorithms.
Abstract
We present a class of algorithms for independent component analysis (ICA) which use contrast functions based on canonical correlations in a reproducing kernel Hilbert space. On the one hand, we show that our contrast functions are related to mutual information and have desirable mathematical properties as measures of statistical dependence. On the other hand, building on recent developments in kernel methods, we show that these criteria can be computed efficiently. Minimizing these criteria leads to flexible and robust algorithms for ICA. We illustrate with simulations involving a wide variety of source distributions, showing that our algorithms outperform many of the presently known algorithms.
Keywords
ChemistryComputer Science
TechnometricsStatistical Learning Theory
26,913 Citations1999Yuhai Wu, Vladimir Vapnik
Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
Journal of the Operational Research SocietyIntroduction to Algorithms
16,942 Citations1991V. J. Rayward‐Smith, Thomas H. Cormen +2 more
Neural ComputationAn Information-Maximization Approach to Blind Separation and Blind Deconvolution
9,194 Citations1995Anthony J. Bell, Terrence J. Sejnowski
It is suggested that information maximization provides a unifying framework for problems in "blind" signal processing and dependencies of information transfer on time delays are derived.
Neural ComputationNonlinear Component Analysis as a Kernel Eigenvalue Problem
8,078 Citations1998Bernhard Schölkopf, Alexander J. Smola +1 more
A new method for performing a nonlinear form of principal component analysis by the use of integral operator kernel functions is proposed and experimental results on polynomial feature extraction for pattern recognition are presented.
Wiley StatsRef: Statistics Reference OnlineGeneralized Additive Models
3,175 Citations2014Trevor Hastie, Robert Tibshirani
Neural ComputationNatural Gradient Works Efficiently in Learning
2,700 Citations1998Шун-ичи Амари
The dynamical behavior of natural gradient online learning is analyzed and is proved to be Fisher efficient, implying that it has asymptotically the same performance as the optimal batch estimation of parameters.
SIAM Journal on Matrix Analysis and ApplicationsThe Geometry of Algorithms with Orthogonality Constraints
2,678 Citations1998Alan Edelman, T. A. Arias +1 more
The theory proposed here provides a taxonomy for numerical linear algebra algorithms that provide a top level mathematical view of previously unrelated algorithms and developers of new algorithms and perturbation theories will benefit from the theory.
Signal ProcessingBlind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture
2,522 Citations1991Christian Jutten, Jeanny Hérault
Neural ComputationIndependent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources
1,978 Citations1999Te-Won Lee, Mark Girolami +1 more
An extension of the infomax algorithm of Bell and Sejnowski (1995) is presented that is able blindly to separate mixed signals with sub- and supergaussian source distributions and is effective at separating artifacts such as eye blinks and line noise from weaker electrical signals that arise from sources in the brain.
A New Learning Algorithm for Blind Signal Separation
1,888 Citations1995Шун-ичи Амари, Andrzej Cichocki +1 more
A new on-line learning algorithm which minimizes a statistical dependency among outputs is derived for blind separation of mixed signals and has an equivariant property and is easily implemented on a neural network like model.
Journal of the American Statistical AssociationEstimating Optimal Transformations for Multiple Regression and Correlation
1,586 Citations1985Leo Breiman, Jerome H. Friedman
Using the Nyström Method to Speed Up Kernel Machines
1,522 Citations2000Christopher K. I. Williams, Matthias Seeger
It is shown that an approximation to the eigendecomposition of the Gram matrix can be computed by the Nystrom method (which is used for the numerical solution of eigenproblems) and the computational complexity of a predictor using this approximation is O(m2n).
Neural ComputationRegularization Theory and Neural Networks Architectures
1,355 Citations1995Federico Girosi, Michael Jones +1 more
This paper shows that regularization networks encompass a much broader range of approximation schemes, including many of the popular general additive models and some of the neural networks, and introduces new classes of smoothness functionals that lead to different classes of basis functions.
Neural ComputationHigh-Order Contrasts for Independent Component Analysis
1,221 Citations1999J.-F. Cardoso
This article considers high-order measures of independence for the independent component analysis problem and discusses the class of Jacobi algorithms for their optimization and compares the proposed approaches with gradient-based techniques from the algorithmic point of view and also on a set of biomedical data.
Journal of the American Statistical AssociationEfficient and Adaptive Estimation for Semiparametric Models.
1,128 Citations1994Anton Schίck, Peter J. Bickel +3 more
Efficient svm training using low-rank kernel representations
595 Citations2002Shai Fine, Katya Scheinberg
This work shows that for a low rank kernel matrix it is possible to design a better interior point method (IPM) in terms of storage requirements as well as computational complexity and derives an upper bound on the change in the objective function value based on the approximation error and the number of active constraints (support vectors).
2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03).Kernel independent component analysis
569 Citations2004Francis Bach, Michael I. Jordan
A class of algorithms for independent component analysis which use contrast functions based on canonical correlations in a reproducing kernel Hilbert space is presented, showing that these algorithms outperform many of the presently known algorithms.
Neural ComputationIndependent Factor Analysis
508 Citations1999Hagai Attias
An expectation-maximization (EM) algorithm is presented, which performs unsupervised learning of an associated probabilistic model of the mixing situation and is shown to be superior to ICA since it can learn arbitrary source densities from the data.
IEEE Transactions on Signal ProcessingBlind separation of mixture of independent sources through a quasi-maximum likelihood approach
464 Citations1997Dinh Tuan Pham, Philippe Garat
Two methods for separating mixture of independent sources without any precise knowledge of their probability distribution are proposed by considering a maximum likelihood (ML) solution corresponding to some given distributions of the sources and relaxing this assumption afterward.
arXiv (Cornell University)A kernel method for canonical correlation analysis
459 Citations2006Shotaro Akaho
The effectiveness of applying kernel method to canonical correlation analysis is investigated, which shows an efficient approach to improve such a linear method.
Multidimensional independent component analysis
407 Citations2002J.-F. Cardoso
This paper starts from the ICA or blind source separation (BSS) model and shows that it can be uniquely identified provided it is properly parameterized in terms of one-dimensional subspaces and generalized to multidimensional components.
Journal of the Royal Statistical Society Series B (Statistical Methodology)Canonical Correlation Analysis When the Data are Curves
324 Citations1993S. E. Leurgans, R. A. Moyeed +1 more
Transactions of the American Mathematical SocietyAsymptotic behavior of the eigenvalues of certain integral equations
117 Citations1963Harold Widom
Lecture notes in computer scienceNonlinear Feature Extraction Using Generalized Canonical Correlation Analysis
96 Citations2001Thomas Melzer, Michael J. Reiter +1 more
A new non-linear feature extraction technique based on Canonical Correlation Analysis (CCA), which is especially well suited for relating two sets of measurements, with applications in regression and object recognition.
IEEE Transactions on Neural NetworksEfficient source adaptivity in independent component analysis
73 Citations2001Nikos Vlassis, Yoichi Motomura
It is shown with a real and a synthetic experiment that this method can provide more accurate solutions than state-of-the-art methods when optimization is carried out in the vicinity of the global minimum of the contrast function.
SIAM Journal on OptimizationModified Cholesky Factorizations in Interior-Point Algorithms for Linear Programming
55 Citations1999Stephen J. Wright
A modified Cholesky algorithm typical of those used in most interior-point codes for linear programming is investigated, yielding good interior- point steps even when the coefficient matrix of the main linear system to be solved for the step components is ill conditioned.
Neural ComputationA Constrained EM Algorithm for Independent Component Analysis
47 Citations2001Max Welling, Markus Weber
A novel way of performing independent component analysis using a constrained version of the expectation-maximization (EM) algorithm and the simpler soft-switching approach is introduced, which uses only one parameter to decide on the sub- or supergaussian nature of the sources.
Kernel Feature Spaces and Nonlinear Blind Souce Separation
23 Citations2001Stefan Harmeling, Andreas Ziehe +2 more
Learning Canonical Correlations
22 Citations1997Magnus Borga, Hans Knutsson +1 more
It is shown that canonical correlations can be used in computer vision to find feature detectors by giving examples of the desired features and the algorithm successively finds all the canonical correlations beginning with the largest one.
Tree-dependent component analysis
19 Citations2002Francis Bach, Michael I. Jordan
This paper presents a generalization of independent component analysis (ICA), where instead of looking for a linear transform that makes the data components independent, it is shown that the optimal transform is found by minimizing a contrast function based on mutual information.