A Document-Level Neural Machine Translation Model with Dynamic Caching Guided by Theme-Rheme Information
Published 1 January 2020Open access
Yiqi Tong, Jiangbin Zheng, Hongkang Zhu, Yidong Chen, Xiaodong Shi
Citations9
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A novel cache-based document-level NMT model which conducts dynamic caching guided by theme-rheme information is proposed, which is the first to introduce theme- rheme theory into the field of machine translation.
Abstract
Research on document-level Neural Machine Translation (NMT) models has attracted increasing attention in recent years. Although the proposed works have proved that the inter-sentence information is helpful for improving the performance of the NMT models, what information should be regarded as context remains ambiguous. To solve this problem, we proposed a novel cache-based document-level NMT model which conducts dynamic caching guided by theme-rheme information. The experiments on NIST evaluation sets demonstrate that our proposed model achieves substantial improvements over the state-of-the-art baseline NMT models. As far as we know, we are the first to introduce theme-rheme theory into the field of machine translation.
Keywords
Computer Science
UvA-DARE (University of Amsterdam)Adam: A Method for Stochastic Optimization
84,783 Citations2014Diederik P. Kingma, Jimmy Ba
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics)MizAR 60 for Mizar 50
76,311 Citations2023Jakubův, Jan, Chvalovský, Karel +7 more
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics)
50,318 Citations2021Mandi, Jayanta, Canoy, Rocsildes +2 more
A simple numeric simulation of DNA-co-polymerized hydrogel shape change and a genetic algorithm that generates and selects large batches of material designs that compete with one another to evolve and converge on optimal objective-matching designs are constructed.
arXiv (Cornell University)Neural Machine Translation by Jointly Learning to Align and Translate
14,565 Citations2014Dzmitry Bahdanau
arXiv (Cornell University)Sequence to Sequence Learning with Neural Networks
13,362 Citations2014Ilya Sutskever, Oriol Vinyals +1 more
Language<b>Introduction to functional grammar</b> . By M. A. K. Halliday London & Baltimore: Edward Arnold, 1985. Pp. xxxv, 387.
9,924 Citations1989Christian M.I.M. Matthiessen
An Introduction to Functional Grammar
8,212 Citations2014Μ. Α. Κ. Halliday, Christian M.I.M. Matthiessen +2 more
An Introduction to Functional Grammar (Halliday的代表作)
Hierarchical Attention Networks for Document Classification
4,829 Citations2016Zichao Yang, Diyi Yang +4 more
Experiments conducted on six large scale text classification tasks demonstrate that the proposed architecture outperform previous methods by a substantial margin.
Statistical Significance Tests for Machine Translation Evaluation.
1,465 Citations2004Philipp Koehn
Zenodo (CERN European Organization for Nuclear Research)Neuromodulatory Control Networks (NCNs): A Biologically Inspired Architecture for Dynamic LLM Processing
1,395 Citations2025Morgan, Michael Christian
The Penn Discourse TreeBank 2.0.
1,130 Citations2008Rashmi Prasad, Nikhil Dinesh +5 more
The second version of the Penn Discourse Treebank, PDTB-2.0, is presented, describing its lexically-grounded annotations of discourse relations and their two abstract object arguments over the 1 million word Wall Street Journal corpus.
IEEE Transactions on Pattern Analysis and Machine IntelligenceA cache-based natural language model for speech recognition
503 Citations1990Roland Kühn, Renato De Mori
A novel kind of language model which reflects short-term patterns of word use by means of a cache component (analogous to cache memory in hardware terminology) is presented and contains a 3g-gram component of the traditional type.
The Penn Discourse Treebank
328 Citations2004Eleni Miltsakaki, Rashmi Prasad +2 more
A preliminary analysis of inter-annotator agreement is presented – both the level of agreement and the types of inter -annotator variation.
Context-Aware Neural Machine Translation Learns Anaphora Resolution
302 Citations2018Elena Voita, Pavel Serdyukov +2 more
A context-aware neural machine translation model designed in such way that the flow of information from the extended context to the translation model can be controlled and analyzed is introduced.
Document-Level Neural Machine Translation with Hierarchical Attention Networks
276 Citations2018Lesly Miculicich, Dhananjay Ram +2 more
Experiments show that hierarchical attention significantly improves the BLEU score over a strong NMT baseline with the state-of-the-art in context-aware methods, and that both the encoder and decoder benefit from context in complementary ways.
Improving the Transformer Translation Model with Document-Level Context
257 Citations2018Jiacheng Zhang, Huanbo Luan +5 more
This work extends the Transformer model with a new context encoder to represent document-level context, which is then incorporated into the original encoder and decoder, and introduces a two-step training method to take full advantage of abundant sentence-level parallel corpora and limited document- level parallel Corpora.
NCRF++: An Open-source Neural Sequence Labeling Toolkit
186 Citations2018Jie Yang, Yue Zhang
NCRF++ is designed for quick implementation of different neural sequence labeling models with a CRF inference layer, and provides users with an inference for building the custom model structure through configuration file with flexible neural feature design and utilization.
Transactions of the Association for Computational LinguisticsLearning to Remember Translation History with a Continuous Cache
179 Citations2018Zhaopeng Tu, Yang Liu +2 more
This work proposes to augment NMT models with a very light-weight cache-like memory network, which stores recent hidden representations as translation history and the probability distribution over generated words is updated online depending on the translation history retrieved from the memory.
Hierarchical Recurrent Neural Network for Document Modeling
165 Citations2015Rui Lin, Shujie Liu +4 more
A novel hierarchical recurrent neural network language model (HRNNLM) for document modeling that integrates it as the sentence history information into the word level RNN to predict the word sequence with cross-sentence contextual information.
arXiv (Cornell University)Does Neural Machine Translation Benefit from Larger Context?
128 Citations2017Sébastien Jean, Stanislas Lauly +2 more
A neural machine translation architecture that models the surrounding text in addition to the source sentence leads to better performance, both in terms of general translation quality and pronoun prediction, when trained on small corpora, although this improvement largely disappears when trained with a larger corpus.
Transactions of the Association for Computational LinguisticsContext Gates for Neural Machine Translation
108 Citations2017Zhaopeng Tu, Yang Liu +3 more
Context gates are proposed which dynamically control the ratios at which source and target contexts contribute to the generation of target words and can enhance both the adequacy and fluency of NMT with more careful control of the information flow from contexts.
Cache-based Document-level Statistical Machine Translation
97 Citations2011Zhengxian Gong, Min Zhang +1 more
This paper presents three kinds of caches to store relevant document-level information: a dynamic cache, which stores bilingual phrase pairs from the best translation hypotheses of previous sentences in the test document; a static cache,which stores relevantilingual phrase pairs extracted from similar bilingual document pairs in the training parallel corpus; and a topic cache,Which stores the target-side topic words related with the test documents in the source-side.
DiscoursDiscourse in Statistical Machine Translation
76 Citations2012Christian Hardmeier
This thesis addresses the technical and linguistic aspects of discourse-level processing in phrase-based statistical machine translation (SMT) with a focus on connected texts.
Towards Making the Most of Context in Neural Machine Translation
73 Citations2020Zaixiang Zheng, Xiang Yue +3 more
A new document-level NMT framework is proposed that deliberately models the local context of each sentence with the awareness of the global context of the document in both source and target languages, to be able to deal with documents containing any number of sentences, including single sentences.
Proceedings of the AAAI Conference on Artificial IntelligenceVariational Recurrent Neural Machine Translation
72 Citations2018Jinsong Su, Shan Wu +4 more
A novel variational recurrent neural machine translation model that introduces a series of latent random variables to model the translation procedure of a sentence in a generative way, instead of a single latent variable.
Implicitation of Discourse Connectives in (Machine) Translation
53 Citations2013Thomas Meyer, Bonnie Webber
A corpus analysis and a method for semi-automatic detection of discourse connectives in machine translation cases that are not translated into comparable forms in the target language.
On Integrating Discourse in Machine Translation
25 Citations2017Karin Smith
In order to take Machine Translation to another level, it will need to judge output not based on a single reference translation, but based on notions of fluency and of adequacy – ideally with reference to the source text.
arXiv (Cornell University)Fusing Recency into Neural Machine Translation with an Inter-Sentence\n Gate Model
17 Citations2018Shaohui Kuang, Deyi Xiong
arXiv (Cornell University)A Survey on Document-level Neural Machine Translation: Methods and Evaluation
16 Citations2019Sameen Maruf, Fahimeh Saleh +1 more
The aim of this paper is to highlight the major works that have been undertaken in the space of document-level machine translation before and after the neural revolution so that researchers can recognise where they started from and which direction they are heading in.
arXiv (Cornell University)Towards Making the Most of Context in Neural Machine Translation
7 Citations2020Zaixiang Zheng, Xiang Yue +3 more
Capturing document context inside sentence-level neural machine translation models with self-training
6 Citations2021Elman Mansimov, Gábor Melis +1 more
This work proposes an approach that doesn’t require training a specialized model on parallel document-level corpora and is applied to a trained sentence-level NMT model at decoding time and demonstrates that it has higher BLEU score and higher human preference than the baseline.
Big Data AnalyticsBuilding a Chinese discourse topic corpus with a micro-topic scheme based on theme-rheme theory
6 Citations2017Xuefeng Xi, Guodong Zhou
A micro-topic scheme (MTS) representation is proposed based on functional grammar theory, with elementary discourse topic unit (EDTU) as the node and referent of theme-rheme as link to represent the discourse topic structure in the Chinese language according to theme- rheme theory.
ACM Transactions on Asian and Low-Resource Language Information ProcessingA Survey of Discourse Representations for Chinese Discourse Annotation
5 Citations2019Xiaomian Kang, Chengqing Zong +1 more
This survey provides a systematic analysis of existing discourse representation theories to evaluate whether they are suitable for annotation of Chinese text, and concludes that none of the existing theories are quite suitable for scalable Chinese discourse annotation because they are not both expressive and practical.