Sufficient statistics in the optimum control of stochastic systems

Only certain formal properties of the loss function will be required, they will be kept to a minimum, and their significance will be discussed as they are introduced.

Abstract

The search for a sufficient statistic is primarily a problem of data reduction. When a great deal of data is available, it must somehow be summarized in such a way that no valuable information is lost. Furthermore, the criterion that no information should be lost must usually be stated without reference to the purpose to which the data is to be put. For the control problem, this criterion roughly stated requires that it be possible to find an optimum control function which depends on the data only through the sufficient statistic. In order to be useful this property must hold for a large class of loss functions. All the results of this paper will be proved for very broad classes of loss functions. Only certain formal properties of the loss function will be required, they will be kept to a minimum, and their significance will be discussed as they are introduced. In Section 2, the estimation problem is discussed. This problem is completely independent of the loss function. An “equivalent statistic” is defined and a method is provided for computing it sequentially. This problem is of interest in itself but is presented here primarily because the results are required later. In Section 3, an “informative statistic” is defined. This definition refers to the loss function and is roughly equivalent to the criterion stated above for a sufficient statistic. The term “sufficient statistic” will be used imprecisely to indicate either or both the properties, “equivalent” and “informative.” In Section 3, two classes of loss functions, the second more restrictive than the first, are considered. In Theorems 3 and 4 an informative statistic is provided for the two cases. The proofs of the theorems are constructive and provide the equations necessary to actually compute the optimum control as a function of the informative statistic. The method is essentially that of dynamic programming.