Bayesian and regularization methods for hyperparameter estimation in image restoration
IEEE Transactions on Image ProcessingPublished 1 January 1999
Rafael Molina, Aggelos K. Katsaggelos, Javier Mateos
Citations203
SJR quartileQ1
SJR score2.50
SNIP3.41
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper derives expressions for the iterative evaluation of the two hyperparameters applying the evidence and maximum a posteriori (MAP) analysis within the hierarchical Bayesian paradigm.
Abstract
In this paper, we propose the application of the hierarchical Bayesian paradigm to the image restoration problem. We derive expressions for the iterative evaluation of the two hyperparameters applying the evidence and maximum a posteriori (MAP) analysis within the hierarchical Bayesian paradigm. We show analytically that the analysis provided by the evidence approach is more realistic and appropriate than the MAP approach for the image restoration problem. We furthermore study the relationship between the evidence and an iterative approach resulting from the set theoretic regularization approach for estimating the two hyperparameters, or their ratio, defined as the regularization parameter. Finally the proposed algorithms are tested experimentally.
Keywords
Computer ScienceMathematics
Journal of the Royal Statistical Society Series B (Statistical Methodology)Maximum Likelihood from Incomplete Data Via the <i>EM</i> Algorithm
49,657 Citations1977A. P. Dempster, N. M. Laird +1 more
Springer series in statisticsStatistical Decision Theory and Bayesian Analysis
7,725 Citations1985James O. Berger
Neural ComputationBayesian Interpolation
4,377 Citations1992David Mackay
The Bayesian approach to regularization and model-comparison is demonstrated by studying the inference problem of interpolating noisy data by examining the posterior probability distribution of regularizing constants and noise levels.
Lecture notes in statisticsBayesian Learning for Neural Networks
4,375 Citations1996Radford M. Neal
Bayesian Learning for Neural Networks shows that Bayesian methods allow complex neural network models to be used without fear of the "overfitting" that can occur with traditional neural network learning methods.
Machine LearningA Bayesian Method for the Induction of Probabilistic Networks from Data
3,525 Citations1992Gregory F. Cooper, Edward H. Herskovits
This paper presents a Bayesian method for constructing probabilistic networks from databases, focusing on constructing Bayesian belief networks, and extends the basic method to handle missing data and hidden variables.
The Annals of StatisticsOn the Convergence Properties of the EM Algorithm
3,252 Citations1983Changbao Wu
Neural ComputationA Practical Bayesian Framework for Backpropagation Networks
2,934 Citations1992David Mackay
A quantitative and practical Bayesian framework is described for learning of mappings in feedforward networks that automatically embodies "Occam's razor," penalizing overflexible and overcomplex models.
Machine LearningA Bayesian method for the induction of probabilistic networks from data
2,344 Citations1992Gregory F. Cooper, Edward H. Herskovits
This paper presents a Bayesian method for constructing probabilistic networks from databases, focusing on constructing Bayesian belief networks, and extends the basic method to handle missing data and hidden variables.
Pattern RecognitionDigital Image Restoration
640 Citations1979
A multichannel blind restoration technique for linearly degraded images without the explicit knowledge of either the Point Spread Function (PSF) or the original image is proposed.
IEEE Transactions on Acoustics Speech and Signal ProcessingImage reconstruction and restoration: overview of common estimation structures and problems
637 Citations1989G. Demoment
The problem of image reconstruction and restoration is first formulated, and some of the current regularization approaches used to solve the problem are described, and a Bayesian interpretation of the regularization techniques is given.
Elsevier eBooksTheory Refinement on Bayesian Networks
634 Citations1991Wray Buntine
Algorithms for refinement of Bayesian networks are presented to illustrate what is meant by "partial theory", "alternative theory representation", etc, and are an incremental variant of batch learning algorithms from the literature so can work well in batch and incremental mode.
NetworksSequential updating of conditional probabilities on directed graphical structures
533 Citations1990David J. Spiegelhalter, Steffen L. Lauritzen
It is shown how one can introduce imprecision into such probabilities as a data base of cases accumulates and how to take advantage of a range of well-established statistical techniques.
IEEE Transactions on Image ProcessingMethods for choosing the regularization parameter and estimating the noise variance in image restoration and their relation
486 Citations1992N.P. Galatsanos, Aggelos K. Katsaggelos
An error analysis based on an objective mean-square-error (MSE) criterion is used to motivate regularization and two approaches for choosing the regularization parameter and estimating the noise variance are proposed.
Journal of the Royal Statistical Society Series B (Statistical Methodology)On the Convergence of the <i>EM</i> Algorithm
336 Citations1983Russell A. Boyles
IEEE Transactions on Signal ProcessingA regularized iterative image restoration algorithm
285 Citations1991Aggelos K. Katsaggelos, J. Biemond +2 more
The adaptively restored images have better quality than the nonadaptively restored ones based on visual observations and on an objective criterion of merit which accounts for the noise masking property of the visual system.
Developments in Maximum Entropy Data Analysis
256 Citations1989Stephen F. Gull
The derivation of “Classic” MaxEnt is completed, showing that it leads to a natural choice for the regularising parameter α, that supersedes the traditional practice of setting x2=N, and the new condition is that the dimensionless measure of structure -2αS should be equal to the number of good singular values contained in the data.
Optical EngineeringIterative Image Restoration Algorithms
252 Citations1989Aggelos K. Katsaggelos
This tutorial paper discusses the use of successive-approximation-based iterative restoration algorithms for the removal of linear blurs and noise from images and regularization is introduced as a means for preventing the excessive noise magnification that is typically associated with ill-posed inverse problems such as the deblurring problem.
Iterative Identification and Restoration of Images
172 Citations1991Reginald L. Lagendijk, Aggelos K. Katsaggelos +1 more
The blur identification problem is formulated as a constrained maximum-likelihood problem, which directly incorporate a priori known relations between the blur (and image model) coefficients, such as symmetry properties, into the identification procedure.
IEEE Transactions on Pattern Analysis and Machine IntelligenceOn the hierarchical Bayesian approach to image restoration: applications to astronomical images
95 Citations1994Rafael Molina
The author applies the hierarchical Bayesian approach to image restoration problems and compares it with other approaches in handling the estimation of the hyperparameters.
Optical EngineeringImage identification and restoration based on the expectation-maximization algorithm
92 Citations1990Aggelos K. Katsaggelos
The problem of identifying the image and blur parameters and restoring a noisy blurred image is addressed and two algorithms for identification/restoration, based on two different choices of complete data, are derived and compared.
Hyperparameters: Optimize, or Integrate Out?
89 Citations1996David Mackay
Two approximate methods for computational implementation of Bayesian hierarchical models which include unknown hyperparameters such as regularization constants are examined, and comparisons are made with the ideal hierarchical Bayesian solution.
On the Convergence of the Em Algorithm
82 Citations2005Alfred O. Hero
This paper obtains a derivation of region of convergence and asymptotic convergence rates for a specified complete data space by representing the E step in a Taylor series with remainder by the EM algorithm.
IEEE Transactions on Image ProcessingOn some Bayesian/regularization methods for image restoration
72 Citations1995G. E. Archer, D. M. Titterington
Methods are reviewed for choosing regularized restorations in image processing and a method developed by Galatsanos and Katsaggelos is given a Bayesian interpretation and is compared with other Bayesian and non-Bayesian alternatives.
Journal of Applied StatisticsUsing spatial models as priors in astronomical image analysis
70 Citations1989Rafael Molina, B. D. Ripley
Journal of Applied StatisticsUsing spatial models as priors in astronomical image analysis
67 Citations1993Rafael Molina, B. D. Ripley
IEEE Transactions on Signal ProcessingSimultaneous iterative image restoration and evaluation of the regularization parameter
54 Citations1992Moon Gi Kang, Aggelos K. Katsaggelos
A nonlinear regularized iterative image restoration algorithm is proposed, according to which only the noise variance is assumed to be known in advance, and experimental results are shown.
Alpha, Evidence, and the Entropic Prior
23 Citations1993Charlie E. M. Strauss, David H. Wolpert +1 more
The correct entropic prior is computed by marginalization of alpha, and the approximations used to restore the famous “Susie” image may have questionable aspects.
Journal of Visual Communication and Image RepresentationIterative evaluation of the regularization parameter in regularized image restoration
22 Citations1992Aggelos K. Katsaggelos, Moon Gi Kang
A nonlinear regularized iterative image restoration algorithm is proposed, according to which no prior knowledge about the noise variance is assumed, and experimental results are shown.
Circuits Systems and Signal ProcessingAn iterative method for restoring noisy blurred images
21 Citations1984Aggelos K. Katsaggelos, J. Biemond +2 more
The Astronomical JournalBayesian deconvolution in optical astronomy
20 Citations1992Rafael Molina, A. del Olmo +2 more
The Astronomical JournalBayesian deconvolution with prior knowledge of object location - Applications to ground-based planetary images
17 Citations1992Rafael Molina, B. D. Ripley +3 more
RePEc: Research Papers in EconomicsOn the Use of Evidence in Neural Networks
10 Citations1993David H. Wolpert
It turns out that the evidence procedure's MAP estimate for neural nets is, in toto, approximation error, and the exact result neither has to be re-calculated for every new data set, nor requires the running of computer code.
Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE<title>Hierarchical Bayesian approach to image restoration and the iterative evaluation of the regularization parameter</title>
6 Citations1994Rafael Molina, Aggelos K. Katsaggelos
This work relates the hierarchical Bayesian approach to image restoration to an iterative approach for estimating hyperparameters in a deterministic way.
Some Estimation and Restoration Techniques for Statistical Image Analysis
1 Citations1994Graeme Archer
Algorithms for estimation within hierarchical and empirical Bayesian models are developed, and results with non-Bayesian methods are compared, to produce restorations of degraded scenes which are the most probable given the parameter estimates and the data.