Networks adjusting networks
This paper describes extensions of previousàdaptive critics' which have been one-dimensional, acyclic, and suited only for feed-forward controllers, and an idea is described for approximating recurrent back propagation with a 3-network method which is local in time.
Abstract
This paper describes extensions of previousàdaptive critics' which have been one-dimensional, acyclic, and suited only for feed-forward controllers. The extensions address the following issues: 1. Feed-forward adaptive critics for fully recurrent probabilistic control nets. 2. Recurrent adaptive critics. 3. Vector-valued adaptive critics based on a system identiication component. Furthermore an idea is described for approximating recurrent back propagation with a 3-network method which is local in time. In one experiment a linear adaptive critic adjusts a recurrent network such that it solves a non-linear task (a `delayed XOR'-problem). In another experiment a four-dimensional adaptive critic quickly learns to solve a complicated pole balancing task.