Towards End-To-End Speech Recognition with Recurrent Neural Networks
Published 21 June 2014
Alex Graves, Navdeep Jaitly
Citations1,852
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A speech recognition system that directly transcribes audio data with text, without requiring an intermediate phonetic representation is presented, based on a combination of the deep bidirectional LSTM recurrent neural network architecture and the Connectionist Temporal Classification objective function.
Abstract
This paper presents a speech recognition sys-tem that directly transcribes audio data with text, without requiring an intermediate phonetic repre-sentation. The system is based on a combination of the deep bidirectional LSTM recurrent neural network architecture and the Connectionist Tem-poral Classification objective function. A mod-ification to the objective function is introduced that trains the network to minimise the expec-tation of an arbitrary transcription loss function. This allows a direct optimisation of the word er-ror rate, even in the absence of a lexicon or lan-guage model. The system achieves a word error rate of 27.3 % on the Wall Street Journal corpus with no prior linguistic information, 21.9 % with only a lexicon of allowed words, and 8.2 % with a trigram language model. Combining the network with a baseline system further reduces the error rate to 6.7%. 1.
Keywords
Computer Science
Neural ComputationLong Short-Term Memory
98,079 Citations1997Sepp Hochreiter, Jürgen Schmidhuber
A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
ScienceReducing the Dimensionality of Data with Neural Networks
20,953 Citations2006Geoffrey E. Hinton, Ruslan Salakhutdinov
IEEE Transactions on Signal ProcessingBidirectional recurrent neural networks
10,022 Citations1997Mike Schuster, Kuldip K. Paliwal
It is shown how the proposed bidirectional structure can be easily modified to allow efficient estimation of the conditional posterior probability of complete symbol sequences without making any explicit assumption about the shape of the distribution.
Speech recognition with deep recurrent neural networks
8,837 Citations2013Alex Graves, Abdelrahman Mohamed +1 more
This paper investigates deep recurrent neural networks, which combine the multiple levels of representation that have proved so effective in deep networks with the flexible use of long range context that empowers RNNs.
Neural NetworksFramewise phoneme classification with bidirectional LSTM and other neural network architectures
5,463 Citations2005Alex Graves, Jürgen Schmidhuber
The main findings are that bidirectional networks outperform unidirectional ones, and Long Short Term Memory (LSTM) is much faster and also more accurate than both standard Recurrent Neural Nets (RNNs) and time-windowed Multilayer Perceptrons (MLPs).
IEEE Transactions on Acoustics Speech and Signal ProcessingComparison of parametric representations for monosyllabic word recognition in continuously spoken sentences
5,322 Citations1980S. Davis, P. Mermelstein
Several parametric representations of the acoustic signal were compared with regard to word recognition performance in a syllable-oriented continuous speech recognition system and the superior performance of the mel-frequency cepstrum coefficients may be attributed to the fact that they better represent the perceptually relevant aspects of the short-term speech spectrum.
Connectionist temporal classification
5,299 Citations2006Alex Graves, Santiago Fernández +2 more
Kaldi Speech Recognition Toolkit
4,893 Citations2024Daniel Povey
The design of Kaldi is described, a free, open-source toolkit for speech recognition research that provides a speech recognition system based on finite-state automata together with detailed documentation and a comprehensive set of scripts for building complete recognition systems.
Studies in computational intelligenceSupervised Sequence Labelling with Recurrent Neural Networks
3,084 Citations2012Alex Graves
A new type of output layer that allows recurrent networks to be trained directly for sequence labelling tasks where the alignment between the inputs and the labels is unknown, and an extension of the long short-term memory network architecture to multidimensional data, such as images and video sequences.
Deep Neural Networks for Acoustic Modeling in Speech Recognition
1,899 Citations2012Geoffrey E. Hinton, Li Deng +9 more
This paper provides an overview of this progress and repres nts the shared views of four research groups who have had recent successes in using deep neural networks for a coustic modeling in speech recognition.
Kluwer Academic Publishers eBooksConnectionist Speech Recognition: A Hybrid Approach
1,136 Citations1993Hervé Bourlard, Nelson Morgan
Neural NetworksReinforcement learning of motor skills with policy gradients
849 Citations2008Jan Peters, Stefan Schaal
This paper examines learning of complex motor skills with human-like limbs, and combines the idea of modular motor control by means of motor primitives as a suitable way to generate parameterized control policies for reinforcement learning with the theory of stochastic policy gradient learning.
Maximum mutual information estimation of hidden Markov model parameters for speech recognition
821 Citations2005L.R. Bahl, Peter Brown +2 more
A method for estimating the parameters of hidden Markov models of speech is described and recognition results are presented comparing this method with maximum likelihood estimation.
A committee of neural networks for traffic sign classification
375 Citations2011Dan Cireşan, Ueli Meier +2 more
This work describes the approach that won the preliminary phase of the German traffic sign recognition benchmark with a better-than-human recognition rate, and obtains an even better recognition rate by further training the nets.
Framewise phoneme classification with bidirectional lstm and other neural network architectures
353 Citations2005Alex Graves, Jürgen Schmidhuber
Applied Physics Letters10.1162/153244303768966139
319 Citations2000
This work finds that LSTM augmented by "peephole connections" from its internal cells to its multiplicative gates can learn the fine distinction between sequences of spikes spaced either 50 or 49 time steps apart without the help of any short training exemplars.
IEEE Transactions on Speech and Audio ProcessingA frequency warping approach to speaker normalization
286 Citations1998L. Lee, Richard C. Rose
An efficient means for estimating a linear frequency Warping factor and a simple mechanism for implementing frequency warping by modifying the filterbank in mel-frequency cepstrum feature analysis are presented.
Application of pretrained deep neural networks to large vocabulary speech recognition
249 Citations2012Navdeep Jaitly, Patrick Nguyen +2 more
This paper reports results of a DBN-pretrained context-dependent ANN/HMM system trained on two datasets that are much larger than any reported previously, and outperforms the best Gaussian Mixture Model Hidden Markov Model baseline.
Learning a better representation of speech soundwaves using restricted boltzmann machines
229 Citations2011Navdeep Jaitly, Geoffrey E. Hinton
A novel approach for modeling speech sound waves using a Restricted Boltzmann machine (RBM) with a novel type of hidden variable is presented and initial results demonstrate phoneme recognition performance better than the current state-of-the-art for methods based on Mel cepstrum coefficients.
Open vocabulary speech recognition with flat hybrid models
144 Citations2005M. Bisani, Hermann Ney
It is demonstrated that, by using a simple flat hybrid model, a well-optimized state-ofthe-art speech recognition system over a wide range of out-of-vocabulary rates can be significantly improved.
From speech to letters - using a novel neural network architecture for grapheme based ASR
45 Citations2009Florian Eyben, Martin Wöllmer +2 more
This work investigates a novel ASR approach using Bidirectional Long Short-Term Memory Recurrent Neural Networks and Connectionist Temporal Classification, which is capable of transcribing graphemes directly and yields results highly competitive with phoneme transcription.