The Recurrent Cascade-Correlation architecture

Research Showcase @ Carnegie Mellon University (Carnegie Mellon University)Published 30 June 2018Open access

Scott E. Fahlman

Citations146

TL;DR

Recurrent Cascade-Correlation is a recurrent version of the Cascade- Correlation learning architecture of Fahlman and Lebiere that can learn from examples to map a sequence of inputs into a desired sequence of outputs.

Abstract

Abstract: "Recurrent Cascade-Correlation (RCC) is a recurrent version of the Cascade-Correlation learning architecture of Fahlman and Lebiere [Fahlman, 1990]. RCC can learn from examples to map a sequence of inputs into a desired sequence of outputs. New hidden units with recurrent connections are added to the network one at a time, as they are needed during training. In effect, the network builds up a finite-state machine tailored specifically for the current problem. RCC retains the advantages of Cascade-Correlation: fast learning, good generalization, automatic construction of a near-minimal multi-layered network, and the ability to learn complex behaviors through a sequence of simple lessons.The power of RCC is demonstrated on two tasks: learning a finite- state grammar from examples of legal strings, and learning to recognize characters in Morse code."

Keywords

Computer Science

Cognitive ScienceFinding Structure in Time

10,841 Citations1990Jeffrey L. Elman

A proposal along these lines first described by Jordan (1986) which involves the use of recurrent links in order to provide networks with a dynamic memory and suggests a method for representing lexical categories and the type/token distinction is developed.

Neural ComputationA Learning Algorithm for Continually Running Fully Recurrent Neural Networks

4,451 Citations1989Ronald J. Williams, David Zipser

The exact form of a gradient-following learning algorithm for completely recurrent networks running in continually sampled time is derived and used as the basis for practical algorithms for temporal supervised learning tasks.

Research Showcase @ Carnegie Mellon University (Carnegie Mellon University)The Cascade-Correlation learning architecture

2,401 Citations2018Scott E. Fahlman, Christian Lebière

The Cascade-Correlation architecture has several advantages over existing algorithms: it learns very quickly, the network determines its own size and topology, it retains the structures it has built even if the training set changes, and it requires no back-propagation of error signals through the connections of the network.

View PDF

Neural ComputationFinite State Automata and Simple Recurrent Networks

462 Citations1989Axel Cleeremans, David Servan‐Schreiber +1 more

A network architecture introduced by Elman (1988) for predicting successive elements of a sequence and shows that long distance sequential contingencies can be encoded by the network even if only subtle statistical properties of embedded strings depend on the early information.

View PDF

International Conference on Acoustics, Speech, and Signal ProcessingConsonant recognition by modular construction of large phonemic time-delay neural networks

153 Citations2003Alexander Waibel, Hirozumi Sawai +1 more

It is shown that neural networks for speech recognition can be constructed in a modular fashion by exploiting the hidden structure of previously trained phonetic subcategory networks to avoid the excessive learning times that would be necessary to train larger networks and allow for incremental learning.

Elsevier eBooksConsonant Recognition by Modular Construction of Large Phonemic Time-Delay Neural Networks

59 Citations1990Alex Waibel, Hidefumi Sawai +1 more

Connectionist Language Users

42 Citations1992Robert B. Allen

These studies introduce the use of connectionist state machines as intelligent agents in which the emphasis is on perception of and interaction with their environment through language; thus the agents are termed connectionist language users (clues).

International Journal of Neural SystemsLEARNING SEQUENTIAL STRUCTURE WITH THE REAL-TIME RECURRENT LEARNING ALGORITHM

40 Citations1989Anthony Smith, David Zipser

A more powerful recurrent learning procedure, called real-time recurrent learning2,6 (RTRL), is applied to some of the same problems studied by Servan-Schreiber, Cleeremans, and McClelland and revealed that the internal representations developed by RTRL networks revealed that they learn a rich set of internal states that represent more about the past than is required by the underlying grammar.