Monte Carlo techniques for phrase-based translation
Machine TranslationPublished 1 June 2010
Abhishek Arun, Barry Haddow, Philipp Koehn, Adam Lopez, Chris Dyer, Phil Blunsom
Citations13
SJR quartileQ2
SJR score0.34
SNIP1.92
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A novel Gibbs sampler is defined for sampling translations given a source sentence and it is shown that it effectively explores this posterior distribution of sampling defined by a translation model.
Abstract
Recent advances in statistical machine translation have used approximate beam search for NP-complete inference within probabilistic translation models. We present an alternative approach of sampling from the posterior distribution defined by a translation model. We define a novel Gibbs sampler for sampling translations given a source sentence and show that it effectively explores this posterior distribution. In doing so we overcome the limitations of heuristic beam search and obtain theoretically sound solutions to inference problems such as finding the maximum probability translation and minimum risk training and decoding.
Keywords
Computer Science
IEEE Transactions on Pattern Analysis and Machine IntelligenceStochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
17,980 Citations1984Stuart Geman, Donald Geman
The analogy between images and statistical mechanics systems is made and the analogous operation under the posterior distribution yields the maximum a posteriori (MAP) estimate of the image given the degraded observations, creating a highly parallel ``relaxation'' algorithm for MAP estimation.
Mathematical ProgrammingOn the limited memory BFGS method for large scale optimization
8,529 Citations1989Dong C. Liu, Jorge Nocedal
The numerical tests indicate that the L-BFGS method is faster than the method of Buckley and LeNir, and is better able to use additional storage to accelerate convergence, and the convergence properties are studied to prove global convergence on uniformly convex problems.
Journal of the American Statistical AssociationThe Monte Carlo Method
6,075 Citations1949N. Metropolis, Stanislaw M. Ulam
BiometrikaReversible jump Markov chain Monte Carlo computation and Bayesian model determination
5,929 Citations1995Peter J. Green
A new framework for the construction of reversible Markov chain samplers that jump between parameter subspaces of differing dimensionality is proposed, which is flexible and entirely constructive, and should have wide applicability in model determination problems.
Statistical phrase-based translation
3,268 Citations2003Philipp Koehn, Franz Josef Och +1 more
The empirical results suggest that the highest levels of performance can be obtained through relatively simple means: heuristic learning of phrase translations from word-based alignments and lexical weighting of phrase translation.
Minimum error rate training in statistical machine translation
2,766 Citations2003Franz Josef Och
It is shown that significantly better results can often be obtained if the final evaluation criterion is taken directly into account as part of the training procedure.
Edinburgh Research Explorer (University of Edinburgh)Factored Translation Models
509 Citations2007Philipp Koehn, Hieu Hoang
In a number of experiments, it is shown that factored translation models lead to better translation performance, both in terms of automatic scores, as well as more grammatical coherence.
A phrase-based, joint probability model for statistical machine translation
475 Citations2002Daniel Marcu, William Wong
A joint probability model for statistical machine translation is presented, which automatically learns word and phrase equivalents from bilingual corpora, which is more accurate than translations produced using IBM Model 4.
Munich Personal RePEc Archive (Ludwig Maximilian University of Munich)Minimum Bayes-risk decoding for statistical machine translation
330 Citations2004Shankar Kumar, Bill Byrne
The results show that MBR decoding can be used to tune statistical MT performance for specific loss functions, and a hierarchy of loss functions that incorporate different levels of linguistic information from word strings, word-to-word alignments from an MT system, and syntactic structure from parse-trees of source and target language sentences.
Findings of the 2013 Workshop on Statistical Machine Translation
298 Citations2013Chris Callison-Burch, Philipp Koehn +6 more
Edinburgh Research Explorer (University of Edinburgh)Bayesian Inference for PCFGs via Markov Chain Monte Carlo
280 Citations2007Mark Johnson, Thomas L. Griffiths +1 more
Fast decoding and optimal decoding for machine translation
234 Citations2001Ulrich Germann, Michael Jahr +3 more
This paper compares the speed and output quality of a traditional stack-based decoding algorithm with two new decoders: a fast greedy decoder and a slow but optimal decoder that treats decoding as an integer-programming optimization problem.
Empirical Methods in Natural Language ProcessingImproving Translation Quality by Discarding Most of the Phrasetable
201 Citations2007Howard Johnson, Joel Martin +2 more
It is possible to reduce the bulk of phrasetables for Statistical Machine Translation using a technique based on the significance testing of phrase pair co-occurrence in the parallel corpus and an improvement in BLEU is obtained at the same time although the effect is less pronounced if state-of-the-art phrasetic smoothing is employed.
Local gain adaptation in stochastic gradient descent
144 Citations1999Nicol N. Schraudolph
The limitations of this approach are discussed, and an alternative is developed by extending Sutton''s work on linear systems to the general, nonlinear case, and the resulting online algorithms are computationally little more expensive than other acceleration techniques, and do not assume statistical independence between successive training patterns.
Minimum risk annealing for training log-linear models
142 Citations2006David A. Smith, Jason Eisner
This work proposes training log-linear combinations of models for dependency parsing and for machine translation, and describes techniques for optimizing nonlinear functions such as precision or the BLEU metric.
Solving the problem of cascading errors
115 Citations2006Jenny Rose Finkel, Christopher D. Manning +1 more
A novel architecture is presented, which models these pipelines as Bayesian networks, with each low level task corresponding to a variable in the network, and then it is performed approximate inference to find the best labeling.
Edinburgh Research Explorer (University of Edinburgh)A Discriminative Latent Variable Model for Statistical Machine Translation
115 Citations2008Phil Blunsom, Trevor Cohn +1 more
A translation model is presented which models derivations as a latent variable, in both training and decoding, and is fully discriminative and globally optimised, and results show that accounting for multiple derivations does indeed improve performance.
Sampling alignment structure under a Bayesian translation model
96 Citations2008John DeNero, Alexandre Bouchard‐Côté +1 more
This work describes the first tractable Gibbs sampling procedure for estimating phrase pair frequencies under a probabilistic model of phrase alignment and proposes and evaluates two nonparametric priors that successfully avoid the degenerate behavior noted in previous work.
Lecture notes in computer scienceComputational Complexity of Problems on Probabilistic Grammars and Transducers
79 Citations2000Francisco Casacuberta, Colin de la Higuera
Variational decoding for statistical machine translation
53 Citations2009Zhifei Li, Jason Eisner +1 more
A variational approximation, which considers all the derivations but still allows tractable decoding, is developed, which is similar to minimum-risk decoding for BLEU and improves the state of the art.
RWTH Publications (RWTH Aachen)A Systematic Comparison of Training Criteria for Statistical Machine Translation
31 Citations2007Richard Zens, Saša Hasan +1 more
It is shown that, not only from a theoretical point of view but also in terms of translation quality, the minimum Bayes risk decision rule is preferable.
Monte carlo inference and maximization for phrase-based translation
25 Citations2009Abhishek Arun, Chris Dyer +4 more
A novel Gibbs sampler is defined for sampling translations given a source sentence and it is shown that it effectively explores this posterior distribution of sampling defined by a translation model.
Local Search with Very Large-Scale Neighborhoods for Optimal Permutations in Machine Translation
22 Citations2006Jason Eisner, Roy W. Tromble
A novel decoding procedure for statistical machine translation and other ordering tasks based on a family of Very Large-Scale Neighborhoods, some of which have previously been applied to other NP-hard permutation problems are introduced.