Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition
Lecture notes in computer sciencePublished 1 January 2005
Alex Graves, Santiago Fernández, Jürgen Schmidhuber
Citations848
SJR quartileQ2
SJR score0.35
SNIP0.55
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
In this paper, two experiments on the TIMIT speech corpus with bidirectional and unidirectional Long Short Term Memory networks are carried out and it is found that a hybrid BLSTM-HMM system improves on an equivalent traditional HMM system.
Abstract
In this paper, we carry out two experiments on the TIMIT speech corpus with bidirectional and unidirectional Long Short Term Memory (LSTM) networks. In the first experiment (framewise phoneme classification) we find that bidirectional LSTM outperforms both unidirectional LSTM and conventional Recurrent Neural Networks (RNNs). In the second (phoneme recognition) we find that a hybrid BLSTM-HMM system improves on an equivalent traditional HMM system, as well as unidirectional LSTM-HMM.
Keywords
Computer Science
Neural ComputationLong Short-Term Memory
98,079 Citations1997Sepp Hochreiter, Jürgen Schmidhuber
A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
IEEE Transactions on Signal ProcessingBidirectional recurrent neural networks
10,022 Citations1997Mike Schuster, Kuldip K. Paliwal
It is shown how the proposed bidirectional structure can be easily modified to allow efficient estimation of the conditional posterior probability of complete symbol sequences without making any explicit assumption about the shape of the distribution.
Neural NetworksFramewise phoneme classification with bidirectional LSTM and other neural network architectures
5,463 Citations2005Alex Graves, Jürgen Schmidhuber
The main findings are that bidirectional networks outperform unidirectional ones, and Long Short Term Memory (LSTM) is much faster and also more accurate than both standard Recurrent Neural Nets (RNNs) and time-windowed Multilayer Perceptrons (MLPs).
Gradient Flow in Recurrent Nets: The Difficulty of Learning LongTerm Dependencies
1,261 Citations2009
This chapter contains sections titled: Introduction Exponential Error Decay Dilemma: Avoiding Aradient Decay Prevents Long-Term Latching Remedies and Conclusion.
Kluwer Academic Publishers eBooksConnectionist Speech Recognition: A Hybrid Approach
1,136 Citations1993Hervé Bourlard, Nelson Morgan
Connectionist Speech Recognition
578 Citations1994Hervé Bourlard, Nelson Morgan
Speech Reference EPFL-CONF-82487 describes the “politics of language” in the developing world and some of the challenges faced by speech interpreters and interpreters in the rapidly changing environment.
BioinformaticsExploiting the past and the future in protein secondary structure prediction
497 Citations1999Pierre Baldi, Søren Brunak +3 more
A family of novel architectures which can learn to make predictions based on variable ranges of dependencies are introduced, extending recurrent neural networks and introducing non-causal bidirectional dynamics to capture both upstream and downstream information.
IEEE Transactions on Neural NetworksAn application of recurrent nets to phone probability estimation
445 Citations1994A.J. Robinson
Recognition results are presented for the DARPA TIMIT and Resource Management tasks, and it is concluded that recurrent nets are competitive with traditional means for performing phone probability estimation.
Applied Physics Letters10.1162/153244303768966139
319 Citations2000
This work finds that LSTM augmented by "peephole connections" from its internal cells to its multiplicative gates can learn the fine distinction between sequences of spikes spaced either 50 or 49 time steps apart without the help of any short training exemplars.
REMAP: Recursive Estimation and Maximization of A Posteriori Probabilities - Application to Transition-Based Connectionist Speech Recognition
41 Citations1995Yochai Konig, Hervé Bourlard +1 more
REMAP is introduced, an approach for the training and estimation of posterior probabilities using a recursive algorithm that is reminiscent of the EM-based Forward-Backward (Liporace 1982) algorithm for the estimation of sequence likelihoods.
Conference of the International Speech Communication AssociationRemap: recursive estimation and maximization of a posteriori probabilities in transition-based speech recognition
21 Citations1996Yochai Konig, Nelson Morgan
Experiments on the implementation of recurrent neural networks for speech phone recognition
15 Citations2002Ruxin Chen, Leah H. Jamieson
An extensive set of experiments that explore training methods and criteria for recurrent neural networks (RNNs) used for speech phone recognition and proposes a new criterion function that allows direct minimization of the frame error rate.