Developing Practical Neural Network Applications Using Back‐Propagation

The heuristics and techniques used to overcome back-propagation problems, particularly lack of generalization, are identified and outlined, along with areas of potential improvements to the paradigm.

Abstract

Abstract: In the past few years, neural networks have emerged as a problem‐solving technique with capabilities suited to many civil engineering problems. Among the various neural network paradigms available, back‐propagation is by far the most utilized for its relatively simple mathematical proofs and good generalization capabilities. Despite its capabilities, back‐propagation suffers from several problems that hinder the development of practical neural network applications. These include slow training, ill‐defined knowledge representation and problem structuring, and nonguided design of an optimal network configuration for adequate generalization. This paper represents an effort to guide the process of developing practical neural network applications using back‐propagation. The paper starts with a brief description of back‐propagation mathematics. Some of the heuristics and techniques used to overcome back‐propagation problems, particularly lack of generalization, are identified and outlined, along with areas of potential improvements to the paradigm. An application development methodology is proposed utilizing the identified heuristics and techniques. The methodology provides a structured framework for designing and implementing practical neural network applications with less effort.