Multilingual MLP features for low-resource LVCSR systems

Published 1 March 2012

Samuel Thomas, Sriram Ganapathy, Hynek Heřmanský

DOI10.1109/icassp.2012.6288862

Citations109

TL;DR

A new approach to training multilayer perceptrons (MLPs) for large vocabulary continuous speech recognition (LVCSR) in new languages which have only few hours of annotated in-domain training data (for example, 1 hour of data).

Abstract

We introduce a new approach to training multilayer perceptrons (MLPs) for large vocabulary continuous speech recognition (LVCSR) in new languages which have only few hours of annotated in-domain training data (for example, 1 hour of data). In our approach, large amounts of annotated out-of-domain data from multiple languages are used to train multilingual MLP systems without dealing with the different phoneme sets for these languages. Features extracted from these MLP systems are used to train LVCSR systems in the low-resource language similar to the Tandem approach. In our experiments, the proposed features provide a relative improvement of about 30% in an low-resource LVCSR setting with only one hour of training data.

Keywords

Computer Science

The Journal of the Acoustical Society of AmericaPerceptual linear predictive (PLP) analysis of speech

2,539 Citations1990Hynek Heřmanský

A new technique for the analysis of speech, the perceptual linear predictive (PLP) technique, which uses three concepts from the psychophysics of hearing to derive an estimate of the auditory spectrum, and yields a low-dimensional representation of speech.

SWITCHBOARD: telephone speech corpus for research and development

2,140 Citations1992J. Godfrey, E. Holliman +1 more

Tandem connectionist feature extraction for conventional HMM systems

693 Citations2002Hynek Heřmanský, Dan Ellis +1 more

A large improvement in word recognition performance is shown by combining neural-net discriminative feature processing with Gaussian-mixture distribution modeling.

Probabilistic and Bottle-Neck Features for LVCSR of Meetings

341 Citations2007František Grézl, Martin Karafiát +2 more

This work is exploring the possibility of obtaining the features directly from neural net without the necessity of converting output probabilities to features suitable for subsequent GMM-HMM system.

Multilingual acoustic modeling for speech recognition based on subspace Gaussian Mixture Models

160 Citations2010Lukáš Burget, Petr Schwarz +11 more

This work reports experiments on a different approach to multilingual speech recognition, in which the phone sets are entirely distinct but the model has parameters not tied to specific states that are shared across languages.

View PDF

A study on multilingual acoustic modeling for large vocabulary ASR

116 Citations2009Hui Lin, Li Deng +4 more

The results demonstrate that when the context coverage is poor in language-specific training, one tenth of the adaptation data can be used to achieve equivalent performance in cross-lingual speech recognition.

Cross-lingual and multi-stream posterior features for low resource LVCSR systems

71 Citations2010Samuel Thomas, Sriram Ganapathy +1 more

This work proposes to train low resource LVCSR system with additional sources of information like annotated data from other languages (German and Spanish) and various acoustic feature streams (short-term and modulation features) and multilayer perceptrons (MLPs) on these sources of Information.

Learning long-term temporal features in LVCSR using neural networks

68 Citations2004Barry Chen, Qifeng Zhu +1 more

Experiments show that combining the best two-stage approach with conventional short-term features significantly reduces word error rates on the 2001 NIST Hub-5 conversational telephone speech (CTS) evaluation set with models trained using the Switchboard Corpus.

Combination of Acoustic Classifiers Based on Dempster-Shafer Theory of Evidence

48 Citations2007Fabio Valente, Hynek Heřmanský

This paper investigates combination of neural net based classifiers using Dempster-Shafer theory of evidence under some assumptions, which resembles a product of errors rule observed in human speech perception.

View PDF

Training and adapting MLP features for Arabic speech recognition

35 Citations2009J. Park, Frank Diehl +3 more

Various schemes for applying Multi-Layer Perceptrons features to state-of-the-art Arabic speech recognition are described: the use of MLP-features for short-vowel modelling in graphemic systems; rapid discriminative model training by standard PLP feature lattice re-use; and MLP feature adaptation using Linear Input Networks (LIN).