New Method for Parameter Estimation in Probabilistic Models: Minimum Probability Flow
Physical Review LettersPublished 21 November 2011
Jascha Sohl‐Dickstein, Peter Battaglino, Michael R. DeWeese
Citations111
SJR quartileQ1
SJR score2.86
SNIP2.41
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
Fitting probabilistic models to data is often difficult, due to the general intractability of the partition function. We propose a new parameter fitting method, minimum probability flow (MPF), which is applicable to any parametric model. We demonstrate parameter estimation using MPF in two cases: a continuous state space model, and an Ising spin glass. In the latter case, MPF outperforms current techniques by at least an order of magnitude in convergence time with lower error in the recovered coupling parameters.
Keywords
Computer ScienceNeurosciencePhysics and Astronomy
Elements of Information Theory
37,533 Citations2001Thomas M. Cover, Joy A. Thomas
Cambridge University Press eBooksConvex Optimization
31,266 Citations2004Stephen Boyd, Lieven Vandenberghe
Neural ComputationAn Information-Maximization Approach to Blind Separation and Blind Deconvolution
9,194 Citations1995Anthony J. Bell, Terrence J. Sejnowski
It is suggested that information maximization provides a unifying framework for problems in "blind" signal processing and dependencies of information transfer on time delays are derived.
Cognitive ScienceA Learning Algorithm for Boltzmann Machines*
3,324 Citations1985David H. Ackley, Geoffrey E. Hinton +1 more
Handbook of Markov Chain Monte Carlo
3,006 Citations2011Steve Brooks, Andrew Gelman +2 more
This review discusses theoretical and practical aspects of Hamiltonian Monte Carlo, and presents some of its variations, including using windows of states for deciding on acceptance or rejection, computing trajectories using fast approximations, tempering during the course of a trajectory to handle isolated modes, and short-cut methods that prevent useless trajectories from taking much computation time.
Physical Review LettersNonuniversal critical dynamics in Monte Carlo simulations
2,867 Citations1987Robert H. Swendsen, Jian‐Sheng Wang
A new approach to Monte Carlo simulations is presented, giving a highly efficient method of simulation for large systems near criticality, despite the fact that the algorithm violates dynamic universality at second-order phase transitions.
NatureWeak pairwise correlations imply strongly correlated network states in a neural population
1,853 Citations2006Elad Schneidman, Michael J. Berry +2 more
It is shown, in the vertebrate retina, that weak correlations between pairs of neurons coexist with strongly collective behaviour in the responses of ten or more neurons, and it is found that this collective behaviour is described quantitatively by models that capture the observed pairwise correlations but assume no higher-order interactions.
Journal of the Royal Statistical Society Series D (The Statistician)Statistical Analysis of Non-Lattice Data
1,575 Citations1975Julian Besag
Philosophical magazineSolution of 'Solvable model of a spin glass'
1,253 Citations1977D. J. Thouless, Philip W. Anderson +1 more
NeuronTheoretical Neuroscience
1,076 Citations2002Dmitri B. Chklovskii
A recently published book by Peter Dayan and L.F. Abbott, Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems, attempts to cover the whole field of theoretical neuroscience.
Estimation of Non-Normalized Statistical Models by Score Matching
723 Citations2005Aapo Hyvärinen
While the estimation of the gradient of log-density function is, in principle, a very difficult non-parametric problem, it is proved a surprising result that gives a simple formula that simplifies to a sample average of a sum of some derivatives of the log- density given by the model.
Statistics and ComputingBayesian parameter estimation via variational methods
610 Citations2000Tommi Jaakkola, Michael I. Jordan
It is shown that an accurate variational transformation can be used to obtain a closed form approximation to the posterior distribution of the parameters thereby yielding an approximate posterior predictive model.
Journal of NeuroscienceThe Structure of Multi-Neuron Firing Patterns in Primate Retina
495 Citations2006Jonathon Shlens, Greg D. Field +6 more
Large-scale multi-electrode recordings were used to measure electrical activity in nearly complete, regularly spaced mosaics of several hundred ON and OFF parasol retinal ganglion cells in macaque monkey retina, and pairwise and adjacent interactions accurately accounted for the structure and prevalence of multi-neuron firing patterns.
Mathematical BiosciencesRecent developments in parameter estimation and structure identification of biochemical and genomic systems
397 Citations2009I-Chun Chou, Eberhard O. Voit
The article presented here reviews the field of inverse modeling within BST and proposes an operational 'work-flow' that guides the user through the estimation process, identifies possibly problematic steps, and suggests corresponding solutions based on the specific characteristics of the various available algorithms.
Journal of NeuroscienceA Maximum Entropy Model Applied to Spatial and Temporal Correlations from Cortical Networks<i>In Vitro</i>
308 Citations2008Aonan Tang, David Jackson +12 more
Although a second-order maximum entropy model successfully predicts correlated states in cortical networks, it should be extended to account for temporal correlations observed between states, and a significant relationship between strong pairwise temporal correlations and observed sequence length is found.
Computational Statistics & Data AnalysisSome extensions of score matching
178 Citations2006Aapo Hyvärinen
It is shown how to estimate non-normalized models defined in the non-negative real domain, i.e. R"+^n", and it is shown that the score matching estimator can be obtained in closed form for some exponential families.
Physical Review LettersPrediction of Spatiotemporal Patterns of Neural Activity from Pairwise Correlations
146 Citations2009Olivier Marre, Sami El‐Boustani +2 more
A model-based analysis to predict the occurrence of population patterns in distributed spiking activity using a maximum entropy principle with a Markovian assumption is designed and a model that accounts for both spatial and temporal pairwise correlations among neurons is obtained.
Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topicsMean-field theory of Boltzmann machine learning
146 Citations1998Toshiyuki Tanaka
A mean-field theory for Boltzmann machine learning is presented, derived by employing Thouless-Anderson-Palmer free energy formalism to a full extent and a theoretical foundation for an effective trick of using ``diagonal weights,'' introduced by Kappen and Rodr\'{\i}guez is given.
Failures of the One-Step Learning Algorithm
21 Citations2001David Mackay
Examples of models E(x; w) and Markov chains T for which the true likelihood is unimodal in the parameters are given, but the one-step algorithm does not necessarily converge to the maximum likelihood parameters.