Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corpora

In this work we introduce dual conditional cross-entropy filtering for noisy parallel data. For each sentence pair of the noisy parallel corpus we compute cross-entropy scores according to two inverse translation models trained on clean data. We penalize divergent cross-entropies and weigh the penalty by the cross-entropy average of both models. Sorting or thresholding according to these scores results in better subsets of parallel data. We achieve higher BLEU scores with models trained on parallel data filtered only from Paracrawl than with models trained on clean WMT data. We further evaluate our method in the context of the WMT2018 shared task on parallel corpus filtering and achieve the overall highest ranking scores of the shared task, scoring top in three out of four subtasks.

• Publication year

  2018

• Venue

  Conference on Machine Translation

• Publication date

  2018-09-01

• Fields of study

  Computer Science

• Identifiers
  DOI 10.18653/v1/W18-6478arXiv 1809.00197
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Marian: Fast Neural Machine Translation in C++

  2018cited by this paper
• Findings of the WMT 2018 Shared Task on Parallel Corpus Filtering

  2018cited by this paper
• On the Impact of Various Types of Noise on Neural Machine Translation

  2018cited by this paper
• Achieving Human Parity on Automatic Chinese to English News Translation

  2018cited by this paper
• Filtering and Mining Parallel Data in a Joint Multilingual Space

  2018influential reference
• Synthetic and Natural Noise Both Break Neural Machine Translation

  2017cited by this paper
• Attention is All you Need

  2017influential reference
• Dual Learning for Machine Translation

  2016cited by this paper
• A Theoretically Grounded Application of Dropout in Recurrent Neural Networks

  2015influential reference
• Domain Adaptation via Pseudo In-Domain Data Selection

  2011influential reference
• Intelligent Selection of Language Model Training Data

  2010influential reference
• Moses: Open Source Toolkit for Statistical Machine Translation

  2007cited by this paper

• OpusDistillery: A Configurable End-to-End Pipeline for Systematic Multilingual Distillation of Open NMT Models

  2025cites this paper
• Quality Beyond A Glance: Revealing Large Quality Differences Between Web-Crawled Parallel Corpora

  2025cites this paper
• Reference-free Evaluation Metrics for Text Generation: A Survey

  2025influential citation
• Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling

  2024cites this paper
• Critical Learning Periods: Leveraging Early Training Dynamics for Efficient Data Pruning

  2024cites this paper
• Improving Unsupervised Neural Machine Translation via Training Data Self-Correction

  2024cites this paper
• Charles Translator: A Machine Translation System between Ukrainian and Czech

  2024cites this paper
• Machine Translation Models are Zero-Shot Detectors of Translation Direction

  2024cites this paper
• Towards understanding and mitigating the hallucinations in NLP and Speech

  2024cites this paper
• A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelism

  2024cites this paper
• Neural Methods for Aligning Large-Scale Parallel Corpora from the Web for South and East Asian Languages

  2024cites this paper
• There’s No Data like Better Data: Using QE Metrics for MT Data Filtering

  2023cites this paper
• SignBank+: Preparing a Multilingual Sign Language Dataset for Machine Translation Using Large Language Models

  2023cites this paper
• Data Pruning for Efficient Model Pruning in Neural Machine Translation

  2023cites this paper
• PROMT Systems for WMT23 Shared General Translation Task

  2023cites this paper
• A Fast Method to Filter Noisy Parallel Data WMT2023 Shared Task on Parallel Data Curation

  2023cites this paper
• Findings of the WMT 2023 Shared Task on Parallel Data Curation

  2023cites this paper
• Separating the Wheat from the Chaff with BREAD: An open-source benchmark and metrics to detect redundancy in text

  2023cites this paper
• Three-Layer Denoiser: Denoising Parallel Corpora for NMT Systems

  2023cites this paper
• Domain Terminology Integration into Machine Translation: Leveraging Large Language Models

  2023cites this paper
• Learning Multilingual Sentence Representations with Cross-lingual Consistency Regularization

  2023cites this paper
• Robustness of Multi-Source MT to Transcription Errors

  2023cites this paper
• Make Every Example Count: On Stability and Utility of Self-Influence for Learning from Noisy NLP Datasets

  2023cites this paper
• Data Scaling Laws in NMT: The Effect of Noise and Architecture

  2022cites this paper
• Empirical Evaluation of Language Agnostic Filtering of Parallel Data for Low Resource Languages

  2022cites this paper
• Unsupervised Parallel Sentences of Machine Translation for Asian Language Pairs

  2022cites this paper
• Data Cartography for Low-Resource Neural Machine Translation

  2022cites this paper
• Norm-based Noisy Corpora Filtering and Refurbishing in Neural Machine Translation

  2022cites this paper
• PROMT Systems for WMT22 General Translation Task

  2022influential citation
• Samsung R&D Institute Poland Participation in WMT 2022

  2022cites this paper
• eTranslation’s Submissions to the WMT22 General Machine Translation Task

  2022cites this paper
• High-Resource Methodological Bias in Low-Resource Investigations

  2022cites this paper
• CUNI Systems for the WMT 22 Czech-Ukrainian Translation Task

  2022cites this paper
• Analyzing the Use of Influence Functions for Instance-Specific Data Filtering in Neural Machine Translation

  2022cites this paper
• Separating Grains from the Chaff: Using Data Filtering to Improve Multilingual Translation for Low-Resourced African Languages

  2022cites this paper
• Rakuten’s Participation in WAT 2022: Parallel Dataset Filtering by Leveraging Vocabulary Heterogeneity

  2022cites this paper
• Speaking Multiple Languages Affects the Moral Bias of Language Models

  2022cites this paper
• Refining an Almost Clean Translation Memory Helps Machine Translation

  2022cites this paper
• A Comparison of Data Filtering Methods for Neural Machine Translation

  2022cites this paper
• Bitext Mining for Low-Resource Languages via Contrastive Learning

  2022cites this paper
• PROMT Systems for WMT22 Shared General Translation Task

  2022cites this paper
• CMU’s IWSLT 2022 Dialect Speech Translation System

  2022cites this paper
• Detecting Various Types of Noise for Neural Machine Translation

  2022influential citation
• Non-Autoregressive Machine Translation: It’s Not as Fast as it Seems

  2022cites this paper
• NMTScore: A Multilingual Analysis of Translation-based Text Similarity Measures

  2022influential citation
• Data Selection Curriculum for Neural Machine Translation

  2022cites this paper
• Improved Unsupervised Neural Machine Translation with Semantically Weighted Back Translation for Morphologically Rich and Low Resource Languages

  2022cites this paper
• Can Synthetic Translations Improve Bitext Quality?

  2022cites this paper
• Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methods

  2022cites this paper
• Survey of Hallucination in Natural Language Generation

  2022influential citation
• Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation

  2021cites this paper
• Gradient-guided Loss Masking for Neural Machine Translation

  2021cites this paper
• Learning Feature Weights using Reward Modeling for Denoising Parallel Corpora

  2021cites this paper
• Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets

  2021cites this paper
• Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages

  2021cites this paper
• The Curious Case of Hallucinations in Neural Machine Translation

  2021cites this paper
• Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey

  2021cites this paper
• Domain Adaptation for NMT via Filtered Iterative Back-Translation

  2021cites this paper
• Data Filtering using Cross-Lingual Word Embeddings

  2021influential citation
• Prevent the Language Model from being Overconfident in Neural Machine Translation

  2021cites this paper
• Cross-language Sentence Selection via Data Augmentation and Rationale Training

  2021influential citation
• Phenomenon-wise Evaluation Dataset Towards Analyzing Robustness of Machine Translation Models

  2021cites this paper
• Survey of Low-Resource Machine Translation

  2021cites this paper
• ParaMed: a parallel corpus for English–Chinese translation in the biomedical domain

  2021cites this paper
• HintedBT: Augmenting Back-Translation with Quality and Transliteration Hints

  2021cites this paper
• Generalised Unsupervised Domain Adaptation of Neural Machine Translation with Cross-Lingual Data Selection

  2021cites this paper
• The University of Edinburgh’s Bengali-Hindi Submissions to the WMT21 News Translation Task

  2021cites this paper
• The University of Edinburgh’s English-German and English-Hausa Submissions to the WMT21 News Translation Task

  2021cites this paper
• Selecting the Best Data Filtering Method for NMT Training

  2021cites this paper
• Lesan - Machine Translation for Low Resource Languages

  2021cites this paper
• Adapting Neural Machine Translation for Automatic Post-Editing

  2021cites this paper
• A Two-Stage Curriculum Training Framework for NMT

  2021cites this paper
• Dynamic Data Selection and Weighting for Iterative Back-Translation

  2020cites this paper
• Unsupervised Domain Clusters in Pretrained Language Models

  2020cites this paper
• The Roles of Language Models and Hierarchical Models in Neural Sequence-to-Sequence Prediction

  2020cites this paper
• Improving NMT via Filtered Back Translation

  2020cites this paper
• The University of Edinburgh-Uppsala University’s Submission to the WMT 2020 Chat Translation Task

  2020cites this paper
• Bicleaner at WMT 2020: Universitat d’Alacant-Prompsit’s submission to the parallel corpus filtering shared task

  2020cites this paper
• The University of Edinburgh’s submission to the German-to-English and English-to-German Tracks in the WMT 2020 News Translation and Zero-shot Translation Robustness Tasks

  2020influential citation
• Findings of the WMT 2020 Shared Task on Machine Translation Robustness

  2020cites this paper
• An exploratory approach to the Parallel Corpus Filtering shared task WMT20

  2020cites this paper
• Alibaba’s Submission for the WMT 2020 APE Shared Task: Improving Automatic Post-Editing with Pre-trained Conditional Cross-Lingual BERT

  2020cites this paper
• OPPO’s Machine Translation Systems for WMT20

  2020cites this paper
• Dual Conditional Cross Entropy Scores and LASER Similarity Scores for the WMT20 Parallel Corpus Filtering Shared Task

  2020cites this paper
• Findings of the WMT 2020 Shared Task on Parallel Corpus Filtering and Alignment

  2020cites this paper
• Filtering Noisy Parallel Corpus using Transformers with Proxy Task Learning

  2020cites this paper
• The University of Edinburgh’s English-Tamil and English-Inuktitut Submissions to the WMT20 News Translation Task

  2020influential citation
• Filtering Back-Translated Data in Unsupervised Neural Machine Translation

  2020cites this paper
• Alibaba Submission to the WMT20 Parallel Corpus Filtering Task

  2020cites this paper
• Score Combination for Improved Parallel Corpus Filtering for Low Resource Conditions

  2020cites this paper
• Extracting correctly aligned segments from unclean parallel data using character n-gram matching

  2020cites this paper
• Detecting Hallucinated Content in Conditional Neural Sequence Generation

  2020cites this paper
• PheMT: A Phenomenon-wise Dataset for Machine Translation Robustness on User-Generated Contents

  2020cites this paper
• Machine Translation of Open Educational Resources: Evaluating Translation Quality and the Transition to Neural Machine Translation

  2020cites this paper
• Tilde at WMT 2020: News Task Systems

  2020influential citation
• Detecting Fine-Grained Cross-Lingual Semantic Divergences without Supervision by Learning to Rank

  2020cites this paper
• Filtering Noisy Dialogue Corpora by Connectivity and Content Relatedness

  2020cites this paper
• A Self-Refinement Strategy for Noise Reduction in Grammatical Error Correction

  2020cites this paper
• AutoQA: From Databases to Q&A Semantic Parsers with Only Synthetic Training Data

  2020cites this paper
• XMU Evaluation Report for CCMT2020

  2020cites this paper