The nonlinear PCA criterion in blind source separation: Relations with other approaches
NeurocomputingPublished 1 November 1998
Juha Karhunen, P. Pajunen, Erkki Oja
Citations100
SJR quartileQ1
SJR score1.47
SNIP1.94
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The paper shows that nonlinear PCA is a versatile starting point for deriving different kinds of algorithms for blind signal processing problems, leading to computationally efficient and fast converging algorithms.
Abstract
We present new results on the nonlinear principal component analysis (PCA) criterion in blind source separation (BSS). We derive the criterion in a form that allows easy comparisons with other BSS and independent component analysis (ICA) contrast functions like cumulants, Bussgang criteria, and information theoretic contrasts. This clarifies how the nonlinearity should be chosen optimally. We also discuss the connections of the nonlinear PCA learning rule with the Bell-Sejnowski algorithm and the adaptive EASI algorithm. Furthermore, we show that a nonlinear PCA criterion can be minimized using least-squares approaches, leading to computationally efficient and fast converging algorithms. The paper shows that nonlinear PCA is a versatile starting point for deriving different kinds of algorithms for blind signal processing problems.
Keywords
ChemistryComputer Science
Neural ComputationAn Information-Maximization Approach to Blind Separation and Blind Deconvolution
9,194 Citations1995Anthony J. Bell, Terrence J. Sejnowski
It is suggested that information maximization provides a unifying framework for problems in "blind" signal processing and dependencies of information transfer on time delays are derived.
Signal ProcessingBlind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture
2,522 Citations1991Christian Jutten, Jeanny Hérault
IEEE Transactions on Signal ProcessingEquivariant adaptive source separation
1,346 Citations1996J.-F. Cardoso, B.H. Laheld
A class of adaptive algorithms for source separation that implements an adaptive version of equivariant estimation and is henceforth called EASI, which yields algorithms with a simple structure for both real and complex mixtures.
IEEE Transactions on Signal ProcessingProjection approximation subspace tracking
1,257 Citations1995Bin Yang
A novel interpretation of the signal subspace as the solution of a projection like unconstrained minimization problem is presented, and it is shown that recursive least squares techniques can be applied to solve this problem by making an appropriate projection approximation.
IEEE Signal Processing LettersInfomax and maximum likelihood for blind source separation
676 Citations1997J.-F. Cardoso
This article shows that the infomax (information-maximization) principle is equivalent to the maximum likelihood.
IEEE Transactions on Neural NetworksA class of neural networks for independent component analysis
403 Citations1997Juha Karhunen, Erkki Oja +3 more
This paper proposes neural structures related to multilayer feedforward networks for performing complete independent component analysis (ICA) and modify the previous nonlinear PCA type algorithms so that their separation capabilities are greatly improved.
Neural NetworksRepresentation and separation of signals using nonlinear PCA type learning
371 Citations1994Juha Karhunen, J. Joutsensalo
A class of nonlinear PCA (principal component analysis) type learning algorithms is derived by minimizing a general statistical signal representation error and several known algorithms emerge as special cases of these optimization approaches that provide useful information on the properties of the algorithms.
Neural NetworksLeast mean square error reconstruction principle for self-organizing neural-nets
344 Citations1993Lei Xu
It is proved that for one layer with n"1 linear units, the LMSER rule lets their weights converge to rotations of the data's first n"2 principal components, corresponding to the global minimum in the Mean Square Error (MSE) landscape, which has many saddles but no local minimum.
Neural ComputationAdaptive Online Learning Algorithms for Blind Separation: Maximum Entropy and Minimum Mutual Information
313 Citations1997Howard H. Yang, Шун-ичи Амари
It is shown by computer simulation that the convergence of the stochastic descent algorithms is improved by using the natural gradient and the adaptively estimated cumulants.
Neural NetworksGeneralizations of principal component analysis, optimization problems, and neural networks
268 Citations1995Juha Karhunen, J. Joutsensalo
This framework study more closely generalizations of the problems of variance maximization and mean-square error minimization and derive gradient-type neural learning algorithms both for symmetric and hierarchic PCA-type networks.
IEEE Transactions on Circuits and Systems I Fundamental Theory and ApplicationsRobust neural networks with on-line learning for blind identification and blind separation of sources
259 Citations1996Andrzej Cichocki, R. Unbehauen
Two unsupervised, self-normalizing, adaptive learning algorithms are developed for robust blind identification and/or blind separation of independent source signals from a linear mixture of them and are suitable for real-time implementations.
NeurocomputingThe nonlinear PCA learning rule in independent component analysis
219 Citations1997Erkki Oja
It has been verified experimentally that when nonlinear Principal Component Analysis (PCA) learning rules are used for the weights of a neural layer, the neurons have signal separation capabilities and can be used for image and speech signal separation.
International Journal of Adaptive Control and Signal ProcessingHIGH-ORDER CONTRASTS FOR SELF-ADAPTIVE SOURCE SEPARATION
126 Citations1996Éric Moreau, Odile Macchi
The resulting adaptive one-stage and two-stage systems achieve satisfactory separation performance independently of the statistics of sources and of the kind of linear mixture.
Neural approaches to independent component analysis and source separation.
110 Citations1996Juha Karhunen
Various neural approaches have recently been proposed for blind source separation and ICA and the respective learning algorithms are reviewed, and some extensions of the basic ICA model are discussed.
Applications of neural blind separation to signal and image processing
87 Citations2002Juha Karhunen, Aapo Hyvärinen +3 more
Neural blind separation techniques developed in the laboratory are applied to the extraction of features from natural images and to the separation of medical EEG signals, which yields features that describe the underlying data better than for example classical principal component analysis.
International Journal of Neural SystemsStochastic ICA Contrast Maximisation Using Oja's Nonlinear PCA Algorithm
29 Citations1997Mark Girolami, Colin Fyfe
It is shown that Oja's Nonlinear PCA algorithm performs a general stochastic online adaptive ICA, an important extension of linear Principal Component Analysis, which has been applied to the classical statistical signal-processing problem of Blind Separation of Sources.
Blind source separation using least-squares type adaptive algorithms
27 Citations2002Juha Karhunen, P. Pajunen
Adaptive least-squares type algorithms are introduced for blind source separation based on minimizing a criterion used in context with nonlinear PCA (principal component analysis) networks, based on a close relationship to Oja's PCA learning rule.
Proceedings of International Conference on Neural Networks (ICNN'97)Blind source separation and tracking using nonlinear PCA criterion: a least-squares approach
13 Citations2002Juha Karhunen, P. Pajunen
This paper introduces computationally efficient least-squares type algorithms for the basic blind source separation problem, which can still be regarded neural, and they have a close relationship to the previous algorithms.
Fast converging methods for multichannel blind equalization or separation of multipath mixtures
7 Citations2002R. H. Lambert, C.L. Nikias