RAG Picking Helps: Retrieval Augmented Generation for Machine Translation
It is shown that retrieval-augmented fine-tuning of NMT mod-012 els under the RAGMT framework results in an average improvement of 2.03 BLEU scores over simple fine-tuning approaches on English 015 to German domain-specific translation.
Abstract
We introduce RAGMT, a retrieval augmented 001 generation (RAG)-based multi-task framework 002 for Machine Translation (MT) using non-003 parametric knowledge sources. To the best 004 of our knowledge, we are the first to adapt 005 the RAG framework for MT to support end-006 to-end training and use knowledge graphs as 007 the non-parametric source. We also propose 008 the use of new auxiliary training objectives that 009 improve the performance of RAG for domain-010 specific MT. Our experiments demonstrate that 011 retrieval-augmented fine-tuning of NMT mod-012 els under the RAGMT framework results in 013 an average improvement of 2.03 BLEU scores 014 over simple fine-tuning approaches on English 015 to German domain-specific translation. We also 016 demonstrate the efficacy of RAGMT with using 017 in-domain versus domain-agnostic knowledge 018 graphs and careful ablations over the model 019 components. Qualitatively, RAGMT is eas-020 ily interpretable and appears to demonstrate 021 “copy-over-translation" behaviour over named 022 entities. 023