Cascade Correlation: An Incremental Tool for Function Approximation

Published 1 January 1993

Gian Paolo Drago, Sandro Ridella

DOI10.1007/978-1-4471-2063-6_209

Citations9

TL;DR

The capability of the Cascade Correlation algorithm of finding, through an incremental procedure, a sum of weighted hyperbolic tangents, which approximate every function of practical interest to any desired degree of accuracy is shown.

Abstract

In this paper we show the capability of the Cascade Correlation (CC) algorithm of finding, through an incremental procedure, a sum of weighted hyperbolic tangents, which approximate every function of practical interest to any desired degree of accuracy. The incremental algorithm works only on one-layer perceptron and it is, then, a way of solving the credit assignment problem. We show that the integrated squared error has a speed of convergence of order O(1/nh), where nh is the number of hidden neurons: numerical results, through computer simulation, agree with the theory. Our analysis shows that CC represents an efficient implementation of the Projection Pursuit algorithm.

Keywords

Computer SciencePhysics and Astronomy

IEEE Transactions on Information TheoryUniversal approximation bounds for superpositions of a sigmoidal function

2,849 Citations1993Andrew R. Barron

The approximation rate and the parsimony of the parameterization of the networks are shown to be advantageous in high-dimensional settings and the integrated squared approximation error cannot be made smaller than order 1/n/sup 2/d/ uniformly for functions satisfying the same smoothness assumption.

Research Showcase @ Carnegie Mellon University (Carnegie Mellon University)The Cascade-Correlation learning architecture

2,401 Citations2018Scott E. Fahlman, Christian Lebière

The Cascade-Correlation architecture has several advantages over existing algorithms: it learns very quickly, the network determines its own size and topology, it retains the structures it has built even if the training set changes, and it requires no back-propagation of error signals through the connections of the network.

View PDF

Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights

1,384 Citations1990Dat Nguyen, Bernard Widrow

The authors describe how a two-layer neural network can approximate any nonlinear function by forming a union of piecewise linear segments and a method is given for picking initial weights for the network to decrease training time.

The Annals of StatisticsProjection Pursuit

1,144 Citations1985Peter J. Huber

Proceedings of the IEEEConstructive approximations for neural networks by sigmoidal functions

145 Citations1990Lee Jones

G. Cybenko (1989) has demonstrated the existence of uniform approximations to any continuous f provided that sigma is continuous, relying on the Hahn-Branch theorem and the dual characterization of C(I/sup n/).

IEEE Transactions on Neural NetworksStatistically controlled activation weight initialization (SCAWI)

130 Citations1992G. Drago, Sandro Ridella

An optimum weight initialization which strongly improves the performance of the back propagation (BP) algorithm is suggested and an optimum range for R is shown to exist in order to minimize the time needed to reach the minimum of the cost function.