Cost Weighting for Neural Machine Translation Domain Adaptation

In this paper, we propose a new domain adaptation technique for neural machine translation called cost weighting, which is appropriate for adaptation scenarios in which a small in-domain data set and a large general-domain data set are available. Cost weighting incorporates a domain classifier into the neural machine translation training algorithm, using features derived from the encoder representation in order to distinguish in-domain from out-of-domain data. Classifier probabilities are used to weight sentences according to their domain similarity when updating the parameters of the neural translation model. We compare cost weighting to two traditional domain adaptation techniques developed for statistical machine translation: data selection and sub-corpus weighting. Experiments on two large-data tasks show that both the traditional techniques and our novel proposal lead to significant gains, with cost weighting outperforming the traditional methods.

• Publication year

  2017

• Venue

  NMT@ACL

• Publication date

  2017-08-01

• Fields of study

  Computer Science

• Identifiers
  DOI 10.18653/v1/W17-3205
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• An Empirical Comparison of Domain Adaptation Methods for Neural Machine Translation

  2017cited by this paper
• An Empirical Comparison of Simple Domain Adaptation Methods for Neural Machine Translation

  2017cited by this paper
• Semi-supervised Convolutional Networks for Translation Adaptation with Tiny Amount of In-domain Data

  2016influential reference
• Edinburgh Neural Machine Translation Systems for WMT 16

  2016influential reference
• Fast Domain Adaptation for Neural Machine Translation

  2016cited by this paper
• Stanford Neural Machine Translation Systems for Spoken Language Domains

  2015cited by this paper
• Improving Neural Machine Translation Models with Monolingual Data

  2015influential reference
• Neural Machine Translation by Jointly Learning to Align and Translate

  2014cited by this paper
• Sequence to Sequence Learning with Neural Networks

  2014cited by this paper
• Incremental Topic-Based Translation Model Adaptation for Conversational Spoken Language Translation

  2013cited by this paper
• Vector Space Model for Adaptation in Statistical Machine Translation

  2013cited by this paper
• SenseSpotting: Never let your parallel data tie you to an old domain

  2013cited by this paper
• Adaptation Data Selection using Neural Language Models: Experiments in Machine Translation

  2013cited by this paper
• Topic Models for Dynamic Translation Model Adaptation

  2012cited by this paper
• Perplexity Minimization for Translation Model Domain Adaptation in Statistical Machine Translation

  2012cited by this paper
• Sparse lexicalised features and topic adaptation for SMT

  2012cited by this paper
• Domain Adaptation via Pseudo In-Domain Data Selection

  2011cited by this paper
• Cache-based Document-level Statistical Machine Translation

  2011cited by this paper
• Importance Weight Aware Gradient Updates

  2010cited by this paper
• Intelligent Selection of Language Model Training Data

  2010cited by this paper
• Discriminative Instance Weighting for Domain Adaptation in Statistical Machine Translation

  2010cited by this paper
• Context Adaptation in Statistical Machine Translation Using Models with Exponentially Decaying Cache

  2010cited by this paper
• Domain Adaptation for Statistical Machine Translation with Monolingual Resources

  2009cited by this paper
• Discriminative Corpus Weight Estimation for Machine Translation

  2009cited by this paper
• Investigations on large-scale lightly-supervised training for statistical machine translation.

  2008cited by this paper
• Bilingual-LSA Based LM Adaptation for Spoken Language Translation

  2007cited by this paper
• Word-Level Confidence Estimation for Machine Translation

  2007cited by this paper
• Mixture-Model Adaptation for SMT

  2007cited by this paper
• Statistical Significance Tests for Machine Translation Evaluation

  2004cited by this paper
• Bleu: a Method for Automatic Evaluation of Machine Translation

  2002cited by this paper

• Zero-Shot Prompting for LLM-Based Machine Translation Using In-Domain Target Sentences

  2025cites this paper
• WDSRL: Multi-Domain Neural Machine Translation With Word-Level Domain-Sensitive Representation Learning

  2024cites this paper
• A Reinforcement Learning Approach to Improve Low-Resource Machine Translation Leveraging Domain Monolingual Data

  2024cites this paper
• Simple and Scalable Nearest Neighbor Machine Translation

  2023cites this paper
• An Ensemble Strategy with Gradient Conflict for Multi-Domain Neural Machine Translation

  2023cites this paper
• Domain Adaptation: Challenges, Methods, Datasets, and Applications

  2023cites this paper
• Effective domain awareness and adaptation approach via mask substructure for multi-domain neural machine translation

  2023cites this paper
• Neural Machine Translation Transfer Model Based on Mutual Domain Guidance

  2022cites this paper
• Neural Translation System of Meta-Domain Transfer Based on Self-Ensemble and Self-Distillation

  2022cites this paper
• Optum’s Submission to WMT22 Biomedical Translation Tasks

  2022cites this paper
• Deep Transfer Network of Heterogeneous Domain Feature in Machine Translation

  2022cites this paper
• “Hi, how can I help you?” Improving Machine Translation of Conversational Content in a Business Context

  2022cites this paper
• A Comparison of Sentence-Weighting Techniques for NMT

  2021cites this paper
• Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey

  2021influential citation
• Domain-Aware Self-Attention for Multi-Domain Neural Machine Translation

  2021cites this paper
• Finding Sparse Structures for Domain Specific Neural Machine Translation

  2021cites this paper
• Supervised Adaptation of Sequence-to-Sequence Speech Recognition Systems using Batch-Weighting

  2020cites this paper
• Neural Machine Translation

  2020influential citation
• Distill, Adapt, Distill: Training Small, In-Domain Models for Neural Machine Translation

  2020cites this paper
• The Roles of Language Models and Hierarchical Models in Neural Sequence-to-Sequence Prediction

  2020cites this paper
• Addressing Zero-Resource Domains Using Document-Level Context in Neural Machine Translation

  2020cites this paper
• Approaching Neural Chinese Word Segmentation as a Low-Resource Machine Translation Task

  2020cites this paper
• A Survey of Domain Adaptation for Machine Translation

  2020influential citation
• A Novel Sentence-Level Agreement Architecture for Neural Machine Translation

  2020cites this paper
• Iterative Domain-Repaired Back-Translation

  2020cites this paper
• Domain Divergences: A Survey and Empirical Analysis

  2020cites this paper
• Machine Translation of Open Educational Resources: Evaluating Translation Quality and the Transition to Neural Machine Translation

  2020cites this paper
• Factorized Transformer for Multi-Domain Neural Machine Translation

  2020cites this paper
• Finding Sparse Structure for Domain Specific Neural Machine Translation

  2020cites this paper
• Findings of the WMT 2020 Shared Task on Machine Translation Robustness

  2020cites this paper
• Morphological Segmentation of Polysynthetic Languages for Neural Machine Translation: The Case of Inuktitut

  2020cites this paper
• The Translation Problem

  2020cites this paper
• Uses of Machine Translation

  2020cites this paper
• Neural Translation Models

  2020cites this paper
• Beyond Parallel Corpora

  2020cites this paper
• Reinforcement Learning based Curriculum Optimization for Neural Machine Translation

  2019cites this paper
• Tencent Minority-Mandarin Translation System

  2019cites this paper
• Sentence and Word Weighting for Neural Machine Translation Domain Adaptation

  2019cites this paper
• Discriminative Clustering for Robust Unsupervised Domain Adaptation

  2019cites this paper
• Word-based Domain Adaptation for Neural Machine Translation

  2019cites this paper
• Spanish-Swedish neural machine translation for the civil engineering domain

  2019cites this paper
• Combining Local and Document-Level Context: The LMU Munich Neural Machine Translation System at WMT19

  2019cites this paper
• An Empirical Study of Domain Adaptation for Unsupervised Neural Machine Translation

  2019cites this paper
• Domain Adaptation for MT: A Study with Unknown and Out-of-Domain Tasks

  2019cites this paper
• Iterative Dual Domain Adaptation for Neural Machine Translation

  2019cites this paper
• Multi-Domain Neural Machine Translation with Word-Level Adaptive Layer-wise Domain Mixing

  2019cites this paper
• Exploring Discriminative Word-Level Domain Contexts for Multi-Domain Neural Machine Translation

  2019influential citation
• Go From the General to the Particular: Multi-Domain Translation with Domain Transformation Networks

  2019influential citation
• Neural Machine Translation: A Review

  2019influential citation
• Generating Medical Assessments Using a Neural Network Model: Algorithm Development and Validation

  2019cites this paper
• Revisiting Simple Domain Adaptation Methods in Unsupervised Neural Machine Translation

  2019influential citation
• Generic and Specialized Word Embeddings for Multi-Domain Machine Translation

  2019cites this paper
• Lexical Micro-adaptation for Neural Machine Translation

  2019cites this paper
• Domain Adversarial Reinforcement Learning for Partial Domain Adaptation

  2019cites this paper
• Learning a Multi-Domain Curriculum for Neural Machine Translation

  2019cites this paper
• Curriculum Learning for Domain Adaptation in Neural Machine Translation

  2019cites this paper
• Document-Level Information as Side Constraints for Improved Neural Patent Translation

  2018cites this paper
• Tencent Neural Machine Translation Systems for WMT18

  2018cites this paper
• Multi-Domain Neural Machine Translation with Word-Level Domain Context Discrimination

  2018influential citation
• Sentence Weighting for Neural Machine Translation Domain Adaptation

  2018influential citation
• Sentence Selection and Weighting for Neural Machine Translation Domain Adaptation

  2018influential citation
• Learning from Chunk-based Feedback in Neural Machine Translation

  2018cites this paper
• A Survey of Domain Adaptation for Neural Machine Translation

  2018influential citation
• Online Learning for Effort Reduction in Interactive Neural Machine Translation

  2018cites this paper
• Learning Hidden Unit Contribution for Adapting Neural Machine Translation Models

  2018cites this paper
• Neural Language Models

  2018cites this paper
• Document-Level Information as Side Constraints for Improved Neural Patent Translation

  2018cites this paper
• A Survey of Unsupervised Deep Domain Adaptation

  2018influential citation
• Extreme Adaptation for Personalized Neural Machine Translation

  2018cites this paper
• Embeddings as a Means of Domain Adaptation for the Machine Translation

  2017cites this paper
• Neural Machine Translation Training in a Multi-Domain Scenario

  2017cites this paper
• NRC Machine Translation System for WMT 2017

  2017cites this paper