CMNN: Cooperative Modular Neural Networks for pattern recognition

Pattern Recognition LettersPublished 1 November 1997

G. Auda, Mohamed S. Kamel

DOI10.1016/s0167-8655(97)00108-6

Source URL

Citations51

SJR quartileQ1

SJR score1.00

SNIP1.43

TL;DR

Compared to previous modular designs, a significant enhancement in the efficiency of the modular structure is noticed and CMNN's trade-off is a slight increase in the required computational work as compared to some models.

Abstract

A new modular neural network model, the Cooperative Modular Neural Network (CMNN), is introduced. The main idea is to decompose the classification task by dividing the input space into several homogeneous regions (in terms of degree of overlap). A hierarchical task-decomposition algorithm is developed for this task. The structure is modularized and a separate module is trained to draw the classification decision-boundaries within specific regions. Moreover, the task-decomposition algorithm assigns some additional “specialized modules" to high overlap regions detected in the input space. CMNN is, then, experimentally compared to both the non-modular alternative, and several famous modular models, viz., Decoupled; Other-output; ART-BP; Hierarchical; Multiple-experts; Ensembles (two versions); and Merge-glue models. For a comprehensive comparison with non-modular alternatives, ten famous benchmark classification applications are used, viz., Iris; Numerals recognition; Arabic handwritten letters; Two-dimension 20-class; Heart infection; 2-class; Glass; Cleveland heart diseases; Vowels; and Hepatitis databases. Compared to the non-modular alternative, CMNN reduces the benchmark percentage of classification error significantly, or otherwise, performs as good as the non-modular network (in extreme cases of low and high overlap). Compared to previous modular designs, a significant enhancement in the efficiency of the modular structure is noticed. CMNN's trade-off, though, is a slight increase in the required computational work as compared to some models.

Keywords

Computer Science

Neural ComputationAdaptive Mixtures of Local Experts

4,988 Citations1991Robert A. Jacobs, Michael I. Jordan +2 more

A new supervised learning procedure for systems composed of many separate networks, each of which learns to handle a subset of the complete set of training cases, which is demonstrated to be able to be solved by a very simple expert network.

Neural NetworksDemocracy in neural nets: Voting schemes for classification

309 Citations1994Roberto Battiti, A.M. Colla

This paper found that the use of a small number of nets with a sufficiently large uncorrelation in their mistakes reaches a combined performance that is significantly higher than the best obtainable from the individual nets, with a negligible effort after starting from a pool of networks produced in the development phase of an application.

Comparing Voting Systems

280 Citations1987Hannu Nurmi

IEEE International Conference on Neural NetworksMultiple networks for function learning

93 Citations2002Ethem Alpaydın

In the case of learning a mapping, it is proposed to build several possible models instead of one, train them all independently on the same task and take a vote over their responses, where the result of voting is better than the results of all of the networks that participated in the voting process.

Hierarchical clustering with ART neural networks

65 Citations2002G. Bártfai

The concept of a modular neural network structure, which is capable of clustering input patterns through unsupervised learning, and representing a self-consistent hierarchy of clusters at several levels of specificity, is introduced.

Concurrency Practice and ExperienceConstruction of a large‐scale neural network: Simulation of handwritten Japanese character recognition on NCUBE

34 Citations1990Kazuki Joe, Yoshihiro Mori +1 more

This paper describes how new learning methods may make it possible for a large-scale, hierarchical neural network to recognize most Japanese handwritten characters, and describes the implementation of this network on a hypercube concurrent computer.

Cooperation of neural nets for robust classification

22 Citations1990M. de Bollivier, Patrick Gallinari +1 more

The authors propose multimodule architectures based on the cooperation of two or more neural net techniques as a solution to the problem of building robust neural network classification algorithms that perform well on a wide variety of problems.

A new neural network structure with cooperative modules

17 Citations2002G. Auda, Mohamed S. Kamel +1 more

A new neural network structure is proposed in order to solve complex (large) classification problems using an unsupervised network that gave higher accuracy and faster learning speed than the backpropagation scheme.

Modular connectionist structure for 100-word recognition

9 Citations2002Heidi Hackbarth, Juval Mantel

Under the criterion of high recognition rates along with very short reaction time, modular subnet assemblies provide for successful recognition of 100 words and are also recommended for speaker-independent classification.