Pedagogical pattern selection strategies

This paper presents and compares several alternative pattern selection strategies that adapt to the training process, and results show that convergence time and learning accuracy can be improved, but only by strategies of the deterministic type.

Abstract

The problem of the pattern selection strategy for neural network training has not yet received much attention. In back propagation training all patterns are usually presented equally often in random order. This paper presents and compares several alternative pattern selection strategies that adapt to the training process. They favor the selection of patterns producing high error values to the disadvantage of the patterns already mastered by the network. The strategies presented are of two types, random and deterministic. In a random strategy, a pattern is selected randomly with some variable probability depending on the state of the training process. In contrast, the deterministic strategies follow predefined, global schemes that increase the presentation frequency of certain patterns by forced repetition. Simulation results for two test problems show that convergence time and learning accuracy can be improved, but only by strategies of the deterministic type.