Rule-injection hints as a means of improving network performance and learning time

The potential of the hint, whether applied to back-propagation learning or to more general types of pattern associators is to reduce training time and improve generalization performance.

Abstract

Neural networks can be given "hints" by increasing the number of parameters learned to include parameters related to the original relationship. The effect of this hint, whether applied to back-propagation learning or to more general types of pattern associators is to reduce training time and improve generalization performance. A detailed vector field analysis of a hinted back-propagation network solving the XOR problem, shows that the hint is capable of eliminating pathological local minima. A set-theory/functional entropy analysis shows that the hint can be applied to any learning mechanism that has an internal ("hidden") layer of processing. These analyses and tests conducted on a variety of problems using different types of networks demonstrate the potential of the hint as a method of controlling training in order to predictably train systems to effectively model data.