Parallelism, hierarchy, scaling in time-delay neural networks for spotting Japanese phonemes CV-syllables

Published 1 January 1989

Sawai, Waibei, Haffner, Miyatake, Shikano

DOI10.1109/ijcnn.1989.118682

Citations33

TL;DR

To extend the performance of TDNNs (time-delay neural networks) to all phoneme recognition and word/continuous speech recognition, the authors present several techniques and constructed a TDNN which can discriminate a single CV syllable or phoneme.

Abstract

To extend the performance of TDNNs (time-delay neural networks) to all phoneme recognition and word/continuous speech recognition, the authors present several techniques. First, they show that it is possible to scale up the TDNN to a large phonemic TDNN aimed at discriminating all phonemes without loss of recognition performance and without excessive training tokens. Second, the authors propose fast backpropagation learning methods which make it possible to train a large phonemic TDNN within 1.5 hours. Finally, they show several methods for spotting Japanese CV syllables/phonemes in input speech based on TDNNs: they constructed a TDNN which can discriminate a single CV syllable or phoneme. Syllable and phoneme spotting experiments show excellent results, with syllable and phoneme spotting rates of better than 96.7% and 92% correct, respectively.>

Keywords

Computer Science

The MIT Press eBooksParallel Distributed Processing

15,227 Citations1986David E. Rumelhart, James L. McClelland +1 more

IEEE ASSP MagazineAn introduction to computing with neural nets

7,858 Citations1987Richard P. Lippmann

IEEE Transactions on Acoustics Speech and Signal ProcessingPhoneme recognition using time-delay neural networks

2,625 Citations1989Alexander Waibel, Toshiyuki Hanazawa +3 more

The authors present a time-delay neural network (TDNN) approach to phoneme recognition which is characterized by two important properties: using a three-layer arrangement of simple computing units, a hierarchy can be constructed that allows for the formation of arbitrary nonlinear decision surfaces, which the TDNN learns automatically using error backpropagation.

ScholarlyCommons (University of Pennsylvania)Learning Algorithms for Connectionist Networks: Applied Gradient Methods of Nonlinear Optimization

239 Citations1988Raymond L. Watrous

It is shown that in plateau regions of relatively constant gradient, the momentum term acts to increase the step size by a factor of 1/1-μ, where μ is the momentumTerm, and in valley regions with steep sides,The momentum constant acts to focus the search direction toward the local minimum by averaging oscillations in the gradient.

View PDF

IEEE Transactions on Acoustics Speech and Signal ProcessingModularity and scaling in large phonemic neural networks

155 Citations1989Alexander Waibel, Hirozumi Sawai +1 more

The authors train several small time-delay neural networks aimed at all phonemic subcategories and report excellent fine phonemic discrimination performance for all cases and propose several technique that make it possible to grow larger nets in an incremental and modular fashion without loss in recognition performance and without the need for excessive training time or additional data.

Phoneme Recognition: Neural Networks vs. Hidden Markov Models

129 Citations1988Alex Waibel, Toshiyuki Hanazawa +3 more

View PDF

Phoneme recognition: neural networks vs. hidden Markov models vs. hidden Markov models

66 Citations2003Alexander Waibel, Toshiyuki Hanazawa +3 more

A time-delay neural network for phoneme recognition that was able to invent without human interference meaningful linguistic abstractions in time and frequency such as formant tracking and segmentation and does not rely on precise alignment or segmentation of the input.

The Journal of the Acoustical Society of AmericaUsing an adaptive network to recognize demisyllables in continuous speech

5 Citations1988Candace Kamm, Thomas K. Landauer +1 more

A nonlinear, multilayer associative network was trained on a speech recognition task using continuous speech and a minimum‐error‐rate figure of merit was used to evaluate the effect of the size of the training corpus on t...

View PDF