A Systematic Comparison of Training Criteria for Statistical Machine Translation
RWTH Publications (RWTH Aachen)Published 1 June 2007
Richard Zens, Saša Hasan, Hermann Ney
Citations31
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
It is shown that, not only from a theoretical point of view but also in terms of translation quality, the minimum Bayes risk decision rule is preferable.
Abstract
We address the problem of training the free parameters of a statistical machine translation system. We show significant improvements over a state-of-the-art minimum error rate training baseline on a large Chinese-English translation task. We present novel training criteria based on maximum likelihood estimation and expected loss computation. Additionally, we compare the maximum a-posteriori decision rule and the minimum Bayes risk decision rule. We show that, not only from a theoretical point of view but also in terms of translation quality, the minimum Bayes risk decision rule is preferable. 1
Keywords
Computer Science
Numerical recipes in C
15,253 Citations1994William H. Press, Saul A. Teukolsky +2 more
The Diskette v 2.06, 3.5''[1.44M] for IBM PC, PS/2 and compatibles [DOS] Reference Record created on 2004-09-07, modified on 2016-08-08.
METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments
3,723 Citations2005Satanjeev Banerjee, Alon Lavie
METEOR is described, an automatic metric for machine translation evaluation that is based on a generalized concept of unigram matching between the machineproduced translation and human-produced reference translations and can be easily extended to include more advanced matching strategies.
Minimum error rate training in statistical machine translation
2,766 Citations2003Franz Josef Och
It is shown that significantly better results can often be obtained if the final evaluation criterion is taken directly into account as part of the training procedure.
A statistical approach to machine translation
1,699 Citations1990Peter F. Brown, John Cocke +6 more
The application of the statistical approach to translation from French to English and preliminary results are described and the results are given.
Automatic evaluation of machine translation quality using n-gram co-occurrence statistics
1,605 Citations2002George R. Doddington
NIST commissioned NIST to develop an MT evaluation facility based on the IBM work, which is now available from NIST and serves as the primary evaluation measure for TIDES MT research.
Discriminative training and maximum entropy models for statistical machine translation
1,044 Citations2001Franz Josef Och, Hermann Ney
A framework for statistical machine translation of natural languages based on direct maximum entropy models, which contains the widely used source-channel approach as a special case and shows that a baseline statistical machinetranslation system is significantly improved using this approach.
Lecture notes in computer sciencePharaoh: A Beam Search Decoder for Phrase-Based Statistical Machine Translation Models
634 Citations2004Philipp Koehn
Pharaoh, a freely available decoder for phrase-based statistical machine translation models is described, which is the implement at ion of an efficient dynamic programming search algorithm with lattice generation and XML markup for external components.
Re-evaluating the Role of Bleu in Machine Translation Research
621 Citations2006Chris Callison-Burch, Miles Osborne +1 more
It is shown that an improved Bleu score is neither necessary nor sufficient for achieving an actual improvement in translation quality, and two significant counterexamples to Bleu’s correlation with human judgments of quality are given.
Munich Personal RePEc Archive (Ludwig Maximilian University of Munich)Minimum Bayes-risk decoding for statistical machine translation
330 Citations2004Shankar Kumar, Bill Byrne
The results show that MBR decoding can be used to tune statistical MT performance for specific loss functions, and a hierarchy of loss functions that incorporate different levels of linguistic information from word strings, word-to-word alignments from an MT system, and syntactic structure from parse-trees of source and target language sentences.
Evaluation of Machine Translation and its Evaluation
253 Citations2006Joseph Turian, Luke Shea +1 more
The unigram-based F-measure has significantly higher correlation with human judgments than recently proposed alternatives and has an intuitive graphical interpretation, which can facilitate insight into how MT systems might be improved.
Discriminative Reranking for Machine Translation
207 Citations2004Libin Shen, Anoop Sarkar +1 more
This paper describes the application of discriminative reranking techniques to the problem of machine translation and introduces two novel perceptroninspired reranking algorithms that improve on the quality of machinetranslation over the baseline system based on evaluation using the BLEU metric.
Minimum risk annealing for training log-linear models
142 Citations2006David A. Smith, Jason Eisner
This work proposes training log-linear combinations of models for dependency parsing and for machine translation, and describes techniques for optimizing nonlinear functions such as precision or the BLEU metric.
A discriminative global training algorithm for statistical MT
71 Citations2006Christoph Tillmann, Tong Zhang
This paper presents a novel training algorithm for a linearly-scored block sequence translation model, which employs less domain specific knowledge and is both simpler and more extensible than previous approaches.
<i>N</i>-gram posterior probabilities for statistical machine translation
67 Citations2006Richard Zens, Hermann Ney
A sentence length model based on posterior probabilities is introduced and significant improvements on the Chinese-English NIST task and n-gram posterior probabilities are introduced.
The RWTH Statistical Machine Translation System for the IWSLT 2006 Evaluation
51 Citations2006Arne Mauser, Richard Zens +3 more
The method for dealing with punctuation restoration, in order to overcome the difficulties of spoken language translation, is described.
Minimum Bayes Risk decoding for BLEU
23 Citations2007Nicola Ehling, Richard Zens +1 more
It is shown that MBR decoding on N-best lists leads to an improvement of translation quality and the approach aims to minimize the expected loss of translation errors with regard to the BLEU score.
Training and evaluating error minimization rules for statistical machine translation
20 Citations2005Ashish Venugopal, Andreas Zollmann +1 more
A novel training approach for non-tractable decision rules is presented, allowing us to compare and evaluate these and other decision rules on a large scale translation task, taking advantage of the high dimensional parameter space available to the phrase based Pharaoh decoder.
Stochastic modelling
6 Citations2001Hermann Ney
An overview of the stochastic modelling approach to machine translation, starting with the Bayes decision rule as in pattern classification and speech recognition, and showing how the resulting system architecture can be structured into three parts: the language model probability, the string translation model probability and the search procedure that generates the word sequence in the target language.