Encoding Source Language with Convolutional Neural Network for Machine Translation
arXiv (Cornell University)Published 6 March 2015Open access
Fandong Meng, Zhengdong Lu, Mingxuan Wang, Hang Li, Wenbin Jiang, Qun Liu
Citations52
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
The recently proposed neural network joint model (NNJM) (Devlin et al., 2014) augments the n-gram target language model with a heuristically chosen source context window, achieving state-of-the-art performance in SMT. In this paper, we give a more systematic treatment by summarizing the relevant source information through a convolutional architecture guided by the target information. With different guiding signals during decoding, our specifically designed convolution+gating architectures can pinpoint the parts of a source sentence that are relevant to predicting a target word, and fuse them with the context of entire source sentence to form a unified representation. This representation, together with target language words, are fed to a deep neural network (DNN) to form a stronger NNJM. Experiments on two NIST Chinese-English translation tasks show that the proposed model can achieve significant improvements over the previous NNJM by up to +1.08 BLEU points on average
Keywords
Computer Science
arXiv (Cornell University)Sequence to Sequence Learning with Neural Networks
13,362 Citations2014Ilya Sutskever, Oriol Vinyals +1 more
Moses
4,868 Citations2007Philipp Koehn, Richard Zens +12 more
An open-source toolkit for statistical machine translation whose novel contributions are support for linguistically motivated factors, confusion network decoding, and efficient data formats for translation models and language models.
Computational LinguisticsA Systematic Comparison of Various Statistical Alignment Models
3,928 Citations2003Franz Josef Och, Hermann Ney
An important result is that refined alignment models with a first-order dependence and a fertility model yield significantly better results than simple heuristic models.
A Convolutional Neural Network for Modelling Sentences
3,561 Citations2014Nal Kalchbrenner, Edward Grefenstette +1 more
A convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) is described that is adopted for the semantic modelling of sentences and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations.
arXiv (Cornell University)Sequence to Sequence Learning with Neural Networks
3,526 Citations2014Ilya Sutskever, Oriol Vinyals +1 more
This paper presents a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure, and finds that reversing the order of the words in all source sentences improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier.
arXiv (Cornell University)Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
3,282 Citations2014Kyunghyun Cho, Bart van Merriënboer +5 more
Statistical phrase-based translation
3,268 Citations2003Philipp Koehn, Franz Josef Och +1 more
The empirical results suggest that the highest levels of performance can be obtained through relatively simple means: heuristic learning of phrase translations from word-based alignments and lexical weighting of phrase translation.
Minimum error rate training in statistical machine translation
2,766 Citations2003Franz Josef Och
It is shown that significantly better results can often be obtained if the final evaluation criterion is taken directly into account as part of the training procedure.
Journal of Machine Learning ResearchA neural probabilistic language model
2,663 Citations2003BengioYoshua, DucharmeRéjean +2 more
This work reports on experiments using neural networks for the probability function, showing on two text corpora that the proposed approach very significantly improves on a state-of-the-art trigram model.
RWTH Publications (RWTH Aachen)Moses: Open Source Toolkit for Statistical Machine Translation
1,451 Citations2007Philipp Koehn, Hieu Hoang +12 more
Recurrent Continuous Translation Models
1,345 Citations2013Nal Kalchbrenner, Phil Blunsom
A class of probabilistic continuous translation models called Recurrent Continuous Translation Models that are purely based on continuous representations for words, phrases and sentences and do not rely on alignments or phrasal translation units are introduced.
Computational LinguisticsHierarchical Phrase-Based Translation
1,164 Citations2007David Chiang
A statistical machine translation model that uses hierarchical phrasesphrases that contain subphrasing that is formally a synchronous context-free grammar but is learned from a parallel text without any syntactic annotations is presented.
Discriminative training and maximum entropy models for statistical machine translation
1,044 Citations2001Franz Josef Och, Hermann Ney
A framework for statistical machine translation of natural languages based on direct maximum entropy models, which contains the widely used source-channel approach as a special case and shows that a baseline statistical machinetranslation system is significantly improved using this approach.
arXiv (Cornell University)Convolutional Neural Network Architectures for Matching Natural Language Sentences
970 Citations2015Baotian Hu, Zhengdong Lu +2 more
Convolutional neural network models for matching two sentences are proposed, by adapting the convolutional strategy in vision and speech and nicely represent the hierarchical structures of sentences with their layer-by-layer composition and pooling.
Clause restructuring for statistical machine translation
619 Citations2005Michael Collins, Philipp Koehn +1 more
The reordering approach is applied as a pre-processing step in both the training and decoding phases of a phrase-based statistical MT system, showing an improvement from 25.2% Bleu score for a baseline system to 26.8% Blee score for the system with reordering.
Fast and Robust Neural Network Joint Models for Statistical Machine Translation
489 Citations2014Jacob Devlin, Rabih Zbib +4 more
A novel formulation for a neural network joint model (NNJM), which augments the NNLM with a source context window, which is purely lexicalized and can be integrated into any MT decoder.
What's in a Translation Rule
465 Citations2004Michel Galley, Mark Hopkins +2 more
The theory is used to introduce a linear algorithm that can be used to derive from word-aligned, parallel corpora the minimal set of syntactically motivated transformation rules that explain human translation data.
Forest Rescoring: Faster Decoding with Integrated Language Models
290 Citations2007Liang Huang, David Chiang
This work develops faster approaches for efficient decoding based on k-best parsing algorithms and demonstrates their effectiveness on both phrase-based and syntax-based MT systems.
Joint Language and Translation Modeling with Recurrent Neural Networks
259 Citations2013Michael Auli, Michel Galley +2 more
This work presents a joint language and translation model based on a recurrent neural network which predicts target words based on an unbounded history of both source and target words which shows competitive accuracy compared to the traditional channel model features.
A New String-to-Dependency Machine Translation Algorithm with a Target Dependency Language Model
254 Citations2008Libin Shen, Jinxi Xu +1 more
A novel string-todependency algorithm for statistical machine translation that employs a target dependency language model during decoding to exploit long distance word relations, which are unavailable with a traditional n-gram language model.
A novel dependency-to-string model for statistical machine translation
56 Citations2011Jun Xie, Haitao Mi +1 more
A source dependency structure based model that requires no heuristics or separate ordering models of the previous works to control the word order of translations and performs well on long distance reordering.
Translation with Source Constituency and Dependency Trees
16 Citations2013Fandong Meng, Jun Xie +3 more
A novel translation model, which simultaneously exploits the constituency and dependency trees on the source side, to combine the advantages of two types of trees, and significantly outperforms the state-of-theart hierarchical phrase-based model.