Bounds on approximate steepest descent for likelihood maximization in exponential families

An approximate steepest descent strategy is described, converging in families of regular exponential densities to maximum likelihood estimates of density functions, obtained by an application of the principle of minimum relative entropy subject to empirical constraints.

Abstract

An approximate steepest descent strategy is described, converging in families of regular exponential densities to maximum likelihood estimates of density functions. These density estimates are also obtained by an application of the principle of minimum relative entropy subject to empirical constraints. We prove tight bounds on the increase of the log-likelihood at each iteration of our strategy for families of exponential densities whose log-densities are spanned by a set of bounded basis functions.>