Context-Aware Learning for Neural Machine Translation

Interest in larger-context neural machine translation, including document-level and multi-modal translation, has been growing. Multiple works have proposed new network architectures or evaluation schemes, but potentially helpful context is still sometimes ignored by larger-context translation models. In this paper, we propose a novel learning algorithm that explicitly encourages a neural translation model to take into account additional context using a multilevel pair-wise ranking loss. We evaluate the proposed learning algorithm with a transformer-based larger-context translation system on document-level translation. By comparing performance using actual and random contexts, we show that a model trained with the proposed algorithm is more sensitive to the additional context.

• Publication year

  2019

• Venue

  arXiv.org

• Publication date

  2019-03-12

• Fields of study

  Computer Science

• Identifiers
  arXiv 1903.04715
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Adversarial Evaluation of Multimodal Machine Translation

  2018influential reference
• Learning to Discriminate Noises for Incorporating External Information in Neural Machine Translation

  2018cited by this paper
• Achieving Human Parity on Automatic Chinese to English News Translation

  2018cited by this paper
• Extreme Adaptation for Personalized Neural Machine Translation

  2018cited by this paper
• Improving the Transformer Translation Model with Document-Level Context

  2018cited by this paper
• OpenSubtitles2018: Statistical Rescoring of Sentence Alignments in Large, Noisy Parallel Corpora

  2018cited by this paper
• Context-Aware Neural Machine Translation Learns Anaphora Resolution

  2018influential reference
• The MeMAD Submission to the WMT18 Multimodal Translation Task

  2018cited by this paper
• Document-Level Neural Machine Translation with Hierarchical Attention Networks

  2018cited by this paper
• Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation

  2018cited by this paper
• Attention is All you Need

  2017cited by this paper
• Does Neural Machine Translation Benefit from Larger Context?

  2017cited by this paper
• Document Context Neural Machine Translation with Memory Networks

  2017cited by this paper
• LIUM-CVC Submissions for WMT17 Multimodal Translation Task

  2017cited by this paper
• Neural Machine Translation with Extended Context

  2017cited by this paper
• Convolutional Sequence to Sequence Learning

  2017cited by this paper
• OSU Multimodal Machine Translation System Report

  2017cited by this paper
• Evaluating Discourse Phenomena in Neural Machine Translation

  2017cited by this paper
• Doubly-Attentive Decoder for Multi-modal Neural Machine Translation

  2017cited by this paper
• Learning to Remember Translation History with a Continuous Cache

  2017cited by this paper
• Stronger Baselines for Trustable Results in Neural Machine Translation

  2017cited by this paper
• Exploiting Cross-Sentence Context for Neural Machine Translation

  2017cited by this paper
• Attention Strategies for Multi-Source Sequence-to-Sequence Learning

  2017cited by this paper
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

  2016cited by this paper
• Larger-Context Language Modelling with Recurrent Neural Network

  2015cited by this paper
• Neural Machine Translation of Rare Words with Subword Units

  2015cited by this paper
• Neural Machine Translation by Jointly Learning to Align and Translate

  2014cited by this paper
• Natural Language Processing (Almost) from Scratch

  2011cited by this paper

• Detecting incidental correlation in multimodal learning via latent variable modeling

  2023cites this paper
• HanoiT: Enhancing Context-aware Translation via Selective Context

  2023cites this paper
• Beyond General Purpose Machine Translation: The Need for Context-specific Empirical Research to Design for Appropriate User Trust

  2022cites this paper
• TANDO: A Corpus for Document-level Machine Translation

  2022cites this paper
• A Survey on Document-level Neural Machine Translation

  2021cites this paper
• Measuring and Increasing Context Usage in Context-Aware Machine Translation

  2021cites this paper
• Enhancing Lexical Translation Consistency for Document-Level Neural Machine Translation

  2021cites this paper
• Dynamic Context Selection for Document-level Neural Machine Translation via Reinforcement Learning

  2020cites this paper
• SJTU-NICT’s Supervised and Unsupervised Neural Machine Translation Systems for the WMT20 News Translation Task

  2020cites this paper
• Rethinking Document-level Neural Machine Translation

  2020cites this paper
• Using Context in Neural Machine Translation Training Objectives

  2020cites this paper
• Structurally Comparative Hinge Loss for Dependency-Based Neural Text Representation

  2020cites this paper
• Corpora for Document-Level Neural Machine Translation

  2020cites this paper
• Capturing Longer Context for Document-level Neural Machine Translation: A Multi-resolutional Approach

  2020cites this paper
• Exploring contextual information in neural machine translation

  2019cites this paper
• A Survey on Document-level Machine Translation: Methods and Evaluation

  2019cites this paper
• Context-Aware Neural Machine Translation Decoding

  2019cites this paper
• Analysing Coreference in Transformer Outputs

  2019cites this paper