Blind source separation using block-coordinate relative Newton method
Signal ProcessingPublished 9 June 2004
Alexander M. Bronstein, Michael M. Bronstein, Michael Zibulevsky
Citations9
SJR quartileQ1
SJR score0.96
SNIP1.51
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A block-coordinate version of the relative Newton method, recently proposed for quasi-maximum likelihood blind source separation, that converges in near constant number of iterations (order of 10) independently of the problem size.
Abstract
Presented here is a block-coordinate version of the relative Newton method, recently proposed for quasi-maximum likelihood blind source separation. Special structure of the Hessian matrix allows performing block-coordinate Newton descent efficiently. Simulations show that typically our method converges in near constant number of iterations (order of 10) independently of the problem size.
Keywords
Computer ScienceEngineering
Neural ComputationAn Information-Maximization Approach to Blind Separation and Blind Deconvolution
9,194 Citations1995Anthony J. Bell, Terrence J. Sejnowski
It is suggested that information maximization provides a unifying framework for problems in "blind" signal processing and dependencies of information transfer on time delays are derived.
SIAM Journal on Scientific ComputingAtomic Decomposition by Basis Pursuit
6,899 Citations1998Scott Shaobing Chen, David L. Donoho +1 more
Basis Pursuit (BP) is a principle for decomposing a signal into an "optimal" superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions.
IEEE Transactions on Neural NetworksFast and robust fixed-point algorithms for independent component analysis
6,275 Citations1999Aapo Hyvärinen
Using maximum entropy approximations of differential entropy, a family of new contrast (objective) functions for ICA enable both the estimation of the whole decomposition by minimizing mutual information, and estimation of individual independent components as projection pursuit directions.
Vision ResearchSparse coding with an overcomplete basis set: A strategy employed by V1?
3,720 Citations1997Bruno A. Olshausen, David J. Field
These deviations from linearity provide a potential explanation for the weak forms of non-linearity observed in the response properties of cortical simple cells, and they further make predictions about the expected interactions among units in response to naturalistic stimuli.
A New Learning Algorithm for Blind Signal Separation
1,888 Citations1995Шун-ичи Амари, Andrzej Cichocki +1 more
A new on-line learning algorithm which minimizes a statistical dependency among outputs is derived for blind separation of mixed signals and has an equivariant property and is easily implemented on a neural network like model.
Proceedings of the IEEEBlind signal separation: statistical principles
1,852 Citations1998J.-F. Cardoso
The objectives of this paper are to review some of the approaches that have been developed to address blind signal separation and independent component analysis, to illustrate how they stem from basic principles, and to show how they relate to each other.
IEEE Transactions on Signal ProcessingEquivariant adaptive source separation
1,346 Citations1996J.-F. Cardoso, B.H. Laheld
A class of adaptive algorithms for source separation that implements an adaptive version of equivariant estimation and is henceforth called EASI, which yields algorithms with a simple structure for both real and complex mixtures.
Neural ComputationHigh-Order Contrasts for Independent Component Analysis
1,221 Citations1999J.-F. Cardoso
This article considers high-order measures of independence for the independent component analysis problem and discusses the class of Jacobi algorithms for their optimization and compares the proposed approaches with gradient-based techniques from the algorithmic point of view and also on a set of biomedical data.
Neural ComputationBlind Source Separation by Sparse Decomposition in a Signal Dictionary
754 Citations2001Michael Zibulevsky, Barak A. Pearlmutter
Signal ProcessingUnderdetermined blind source separation using sparse representations
710 Citations2001Pau Bofill, Michael Zibulevsky
This work estimates both the number of sources and the mixing matrix by the maxima of a potential function along the circle of unit length, and obtains the minimal l1 norm representation of each data point by a linear combination of the pair of basis vectors that enclose it.
IEEE Transactions on Signal ProcessingBlind separation of instantaneous mixtures of nonstationary sources
500 Citations2001Dinh-Tuan Pham, J.-F. Cardoso
Novel approaches in this direction based on the principles of maximum likelihood and minimum mutual information are developed, contrasting separating methods for non-Gaussian and nonstationary models and emphasizing that what makes the algorithms work is not the nonstationarity itself but rather the property that each realization of the source signals has a time-varying envelope.
IEEE Transactions on Signal ProcessingBlind separation of mixture of independent sources through a quasi-maximum likelihood approach
464 Citations1997Dinh Tuan Pham, Philippe Garat
Two methods for separating mixture of independent sources without any precise knowledge of their probability distribution are proposed by considering a maximum likelihood (ML) solution corresponding to some given distributions of the sources and relaxing this assumption afterward.
Journal of the Optical Society of America AProbabilistic framework for the adaptation and comparison of image codes
312 Citations1999Michael S. Lewicki, Bruno A. Olshausen
The learned bases are shown to have better coding efficiency than traditional Fourier and wavelet bases and to provide a Bayesian solution to the problems of image denoising and filling in of missing pixels.
SIAM Journal on Matrix Analysis and ApplicationsJoint Approximate Diagonalization of Positive Definite Hermitian Matrices
256 Citations2001Dinh Tuan Pham
This paper provides an iterative algorithm to jointly approximately diagonalize K Hermitian positive definite matrices and calculates the matrix B which minimizes the criterionsum_{k=1}^K n_k, nk being positive numbers, which is a measure of the deviation from diagonality of the matrices BCkB*.
Independent Component Analysis: Principles and Practice
242 Citations2001Stephen Roberts, Richard Everson
This chapter discusses ICA: model order selection and dynamic source models William Penny, Stephen Roberts and Richard Everson, and its applications in data classification, visualisation and image processing.
Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE<title>Blind source separation by sparse decomposition</title>
203 Citations2000Michael Zibulevsky, Barak A. Pearlmutter
This work exploits the property of the sources to have a sparse representation in a corresponding signal dictionary, which provides faster and more robust computations, when there are an equal number of sources and mixtures.
Electronics LettersRobust learning algorithmfor blind separation of signals
194 Citations1994Andrzej Cichocki, R. Unbehauen +1 more
A novel, efficient, self-normalising, unsupervised adaptive learning algorithm for the on-line (real-time) separation of statistically independent unknown source signals from a linear mixture of them.
Optimization methods & softwareGlobally convergent block-coordinate techniques for unconstrained optimization
123 Citations1999Luigi Grippof, Marco Sciandrone
New classes of globally convergent block-coordinate techniques for the unconstrained minimization of a continuously differentiable function and line-search-based schemes that may also include partial global inimizations with respect to some component are defined.
Joint diagonalization of correlation matrices by using gradient methods with application to blind signal separation
99 Citations2003M. Joho, H Mathis
The paper addresses the blind signal separation problem for the case where the source signals are non-stationary and/or non-white, and the sensors are possibly noisy by presenting cost functions for jointly diagonalizing several correlation matrices.
Blind Source Separation via Multinode Sparse Representation
84 Citations2001Michael Zibulevsky, Pavel Kisilev +2 more
The property of multiscale transforms, such as wavelet or wavelet packets, is used to decompose signals into sets of local features with various degrees of sparsity, for selecting the best (most sparse) subsets of features for further separation.
BLIND SOURCE SEPARATION WITH RELATIVE NEWTON METHOD
44 Citations2003Michael Zibulevsky
A relative optimization framework for the quasimaximum likelihood blind source separation and relative Newton method as its particular instance and demonstrates the efficiency of the presented approach on example of sparse sources.
Separation of reflections via sparse ICA
21 Citations2004Michael M. Bronstein, Alex Bronstein +2 more
Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIESeparation of reflections via sparse ICA
20 Citations2003Alexander M. Bronstein, Michael M. Bronstein +2 more
The sparse ICA (SPICA) approach is extended and applied to the separation of the desired image from the superimposed images, without having any a priory knowledge about its structure and/or statistics.
Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIEBlind separation of sparse sources with relative Newton method
14 Citations2003Michael Zibulevsky
A relative optimization framework for the quasi-maximum likelihood blind source separation and relative Newton method as its particular instance and Sequential optimization with the gradual reduction of the smoothing parameter leads to the super-efficient separation.
2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03).Separation of semireflective layers using sparse ICA
5 Citations2003Alex Bronstein, Michael M. Bronstein +2 more
The sparse ICA (SPICA) approach is extended to BSS and applied to the separation of the desired image from the superimposed images, without having any a priori knowledge about its structure and/or statistics.
BLOCK-COORDINATE RELATIVE NEWTON METHOD FOR BLIND SOURCE SEPARATION
4 Citations2003Alexander M. Bronstein, Michael M. Bronstein +1 more
This work generalizes the relative Newton method recently proposed for quasi-maximum likelihood blind source separation by performing block-coordinate Newton descent, which significantly boosts the algorithm performance and reduces its computational complexity.