SYNCHRONIZING TO THE ENVIRONMENT: INFORMATION-THEORETIC CONSTRAINTS ON AGENT LEARNING

It is shown that the total uncertainty experienced by the agent during the process is closely related to the transient information, a new quantity that captures the manner in which the environment's entropy growth curve converges to its asymptotic form.

Abstract

Using an information-theoretic framework, we examine how an intelligent agent, given an accurate model of its environment, synchronizes to the environment — i.e., comes to know in which state the environment is. We show that the total uncertainty experienced by the agent during the process is closely related to the transient information, a new quantity that captures the manner in which the environment's entropy growth curve converges to its asymptotic form. We also discuss how an agent's estimates of its environment's structural properties are related to its estimate of the environment entropy rate. If structural properties are ignored, the missed regularities are converted to apparent randomness. Conversely, using representations that assume too much memory results in false predictability.