Zero-Resource Translation with Multi-Lingual Neural Machine Translation

In this paper, we propose a novel finetuning algorithm for the recently introduced multi-way, mulitlingual neural machine translate that enables zero-resource machine translation. When used together with novel many-to-one translation strategies, we empirically show that this finetuning algorithm allows the multi-way, multilingual model to translate a zero-resource language pair (1) as well as a single-pair neural translation model trained with up to 1M direct parallel sentences of the same language pair and (2) better than pivot-based translation strategy, while keeping only one additional copy of attention-related parameters.

• Publication year

  2016

• Venue

  Conference on Empirical Methods in Natural Language Processing

• Publication date

  2016-06-13

• Fields of study

  Linguistics, Computer Science

• Identifiers
  DOI 10.18653/v1/D16-1026arXiv 1606.04164
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Does Multimodality Help Human and Machine for Translation and Image Captioning?

  2016cited by this paper
• Character-based Neural Machine Translation

  2016cited by this paper
• Multi-Source Neural Translation

  2016influential reference
• A Character-level Decoder without Explicit Segmentation for Neural Machine Translation

  2016cited by this paper
• Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism

  2016influential reference
• Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models

  2016cited by this paper
• Montreal Neural Machine Translation Systems for WMT’15

  2015cited by this paper
• Neural Machine Translation of Rare Words with Subword Units

  2015cited by this paper
• Improving Neural Machine Translation Models with Monolingual Data

  2015cited by this paper
• Effective Approaches to Attention-based Neural Machine Translation

  2015cited by this paper
• Multi-Task Learning for Multiple Language Translation

  2015cited by this paper
• Multi-task Sequence to Sequence Learning

  2015cited by this paper
• Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

  2014cited by this paper
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• Sequence to Sequence Learning with Neural Networks

  2014cited by this paper
• Neural Machine Translation by Jointly Learning to Align and Translate

  2014cited by this paper
• Recurrent Continuous Translation Models

  2013cited by this paper
• On the difficulty of training recurrent neural networks

  2012cited by this paper
• Cross-linguistic similarity in foreign language learning

  2011cited by this paper
• Recurrent neural network based language model

  2010cited by this paper
• Improving Arabic-Chinese Statistical Machine Translation using English as Pivot Language

  2009cited by this paper
• A Simplex Armijo Downhill Algorithm for Optimizing Statistical Machine Translation Decoding Parameters

  2009cited by this paper
• Moses: Open Source Toolkit for Statistical Machine Translation

  2007cited by this paper
• Pivot language approach for phrase-based statistical machine translation

  2007influential reference
• A Comparison of Pivot Methods for Phrase-Based Statistical Machine Translation

  2007influential reference
• Cross-linguistic Similarity in Foreign Language Learning

  2006cited by this paper
• Bidirectional recurrent neural networks

  1997cited by this paper
• Biological and Artificial Computation: From Neuroscience to Technology

  1997cited by this paper
• Recursive Hetero-associative Memories for Translation

  1997cited by this paper

• Data Kernel Perspective Space Performance Guarantees for Synthetic Data from Transformer Models

  2026cites this paper
• Implementing and Evaluating Multi-source Retrieval-Augmented Translation

  2025cites this paper
• Scaling Laws for Conditional Emergence of Multilingual Image Captioning via Generalization from Translation

  2025cites this paper
• Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation

  2025cites this paper
• Registering Source Tokens to Target Language Spaces in Multilingual Neural Machine Translation

  2025cites this paper
• Mixup Helps Translation, But Do the Coefficients and the Selection Strategy Influence Translation Quality?

  2025cites this paper
• Effectiveness of Transfer Learning Approach for Low-Resource Bangla Dialect Translation

  2025influential citation
• Large Language Models are Parallel Multilingual Learners

  2024influential citation
• CharED: Character-wise Ensemble Decoding for Large Language Models

  2024cites this paper
• A Review of Mongolian Neural Machine Translation from the Perspective of Training

  2024cites this paper
• Unsupervised Multimodal Machine Translation for Low-resource Distant Language Pairs

  2024cites this paper
• Revealing the Parallel Multilingual Learning within Large Language Models

  2024cites this paper
• Multilingual Neural Machine Translation for Indic to Indic Languages

  2024cites this paper
• How effective is Multi-source pivoting for Translation of Low Resource Indian Languages?

  2024cites this paper
• LCS: A Language Converter Strategy for Zero-Shot Neural Machine Translation

  2024cites this paper
• Defining Boundaries: The Impact of Domain Specification on Cross-Language and Cross-Domain Transfer in Machine Translation

  2024cites this paper
• An Empirical Study on Code Search Pre-trained Models: Academic Progresses vs. Industry Requirements

  2024cites this paper
• Transferring Zero-shot Multilingual Chinese-Chinese Translation Model for Chinese Minority Language Translation

  2024cites this paper
• Languages Transferred Within the Encoder: On Representation Transfer in Zero-Shot Multilingual Translation

  2024cites this paper
• Neural Machine Translation between Low-Resource Languages with Synthetic Pivoting

  2024cites this paper
• Modularized Multilingual NMT with Fine-grained Interlingua

  2024influential citation
• Parallel Corpus Augmentation using Masked Language Models

  2024cites this paper
• Feature Selection With Partial Autoencoding for Zero-Sample Fault Diagnosis

  2024cites this paper
• Improving Language Transfer Capability of Decoder-only Architecture in Multilingual Neural Machine Translation

  2024cites this paper
• Transformers for Bridging Persian Dialects: Transliteration Model for Tajiki and Iranian Scripts

  2024cites this paper
• Robustness of Multi-Source MT to Transcription Errors

  2023cites this paper
• Improving Chinese-Centric Low-Resource Translation Using English-Centric Pivoted Parallel Data

  2023cites this paper
• Unlikelihood Tuning on Negative Samples Amazingly Improves Zero-Shot Translation

  2023cites this paper
• Research on Chinese-Lao Neural Machine Translation Based on Multi-Pivot

  2023cites this paper
• Visual Prediction Improves Zero-Shot Cross-Modal Machine Translation

  2023cites this paper
• Direct Neural Machine Translation with Task-level Mixture of Experts models

  2023cites this paper
• A Sanskrit grammar-based approach to identify and address gaps in Google Translate’s Sanskrit-English zero-shot NMT

  2023cites this paper
• One Adapter for All Programming Languages? Adapter Tuning for Code Search and Summarization

  2023cites this paper
• Kazakh-Chinese neural machine translation based on data augmentation

  2023influential citation
• Filtering and Extended Vocabulary based Translation for Low-resource Language Pair of Sanskrit-Hindi

  2023cites this paper
• Investigating Multi-Pivot Ensembling with Massively Multilingual Machine Translation Models

  2023influential citation
• How To Build Competitive Multi-gender Speech Translation Models For Controlling Speaker Gender Translation

  2023cites this paper
• ZRIGF: An Innovative Multimodal Framework for Zero-Resource Image-Grounded Dialogue Generation

  2023cites this paper
• Summarize While Translating: Universal Model With Parallel Decoding for Summarization and Translation

  2023cites this paper
• Low-resource Neural Machine Translation: Methods and Trends

  2022cites this paper
• Modality-Balanced Embedding for Video Retrieval

  2022cites this paper
• Synchronous Inference for Multilingual Neural Machine Translation

  2022cites this paper
• Towards the Next 1000 Languages in Multilingual Machine Translation: Exploring the Synergy Between Supervised and Self-Supervised Learning

  2022cites this paper
• NEUREM3 Interim Research Report

  2022cites this paper
• Video Pivoting Unsupervised Multi-Modal Machine Translation

  2022cites this paper
• SRT’s Neural Machine Translation System for WMT22 Biomedical Translation Task

  2022cites this paper
• Pre-trained Language Model-based Retrieval and Ranking for Web Search

  2022cites this paper
• Two-Stream Network for Sign Language Recognition and Translation

  2022cites this paper
• From unified phrase representation to bilingual phrase alignment in an unsupervised manner

  2022cites this paper
• UM4: Unified Multilingual Multiple Teacher-Student Model for Zero-Resource Neural Machine Translation

  2022cites this paper
• MMTAfrica: Multilingual Machine Translation for African Languages

  2022cites this paper
• Accelerating Human Translation of Public Health Information into Low-Resource Languages with Machine Translation

  2022cites this paper
• Building Multilingual Machine Translation Systems That Serve Arbitrary XY Translations

  2022cites this paper
• Informative Language Representation Learning for Massively Multilingual Neural Machine Translation

  2022cites this paper
• Twist Decoding: Diverse Generators Guide Each Other

  2022cites this paper
• Multilingual Machine Translation: Deep Analysis of Language-Specific Encoder-Decoders

  2022cites this paper
• Building Machine Translation Systems for the Next Thousand Languages

  2022cites this paper
• Multilingual Mix: Example Interpolation Improves Multilingual Neural Machine Translation

  2022influential citation
• Pre-training Synthetic Cross-lingual Decoder for Multilingual Samples Adaptation in E-Commerce Neural Machine Translation

  2022cites this paper
• Towards Multilingual Transitivity and Bidirectional Multilingual Agreement for Multilingual Document-level Machine Translation

  2022cites this paper
• Unpaired Multimodal Neural Machine Translation via Reinforcement Learning

  2021cites this paper
• Zero-Shot Language Transfer vs Iterative Back Translation for Unsupervised Machine Translation

  2021cites this paper
• Zero-Shot Cross-lingual Semantic Parsing

  2021cites this paper
• Self-Learning for Zero Shot Neural Machine Translation

  2021cites this paper
• Low-Resource Neural Machine Translation for Southern African Languages

  2021cites this paper
• The TALP-UPC Participation in WMT21 News Translation Task: an mBART-based NMT Approach

  2021cites this paper
• Revisiting Language Encoding in Learning Multilingual Representations

  2021cites this paper
• Task Aligned Generative Meta-learning for Zero-shot Learning

  2021cites this paper
• Simultaneous Multi-Pivot Neural Machine Translation

  2021influential citation
• Augmented Spanish-Persian Neural Machine Translation

  2021cites this paper
• Zero-shot reductive paraphrasing for digitally semi-literate

  2021cites this paper
• Transfer Learning with Shallow Decoders: BSC at WMT2021’s Multilingual Low-Resource Translation for Indo-European Languages Shared Task

  2021cites this paper
• A2R2: Robust Unsupervised Neural Machine Translation With Adversarial Attack and Regularization on Representations

  2021cites this paper
• Improving Multilingual Neural Machine Translation with Auxiliary Source Languages

  2021cites this paper
• Word reordering on multiple pivots for the Japanese and Indonesian language pair

  2021cites this paper
• Findings of the Shared Task on Machine Translation in Dravidian languages

  2021cites this paper
• An Empirical Investigation of Word Alignment Supervision for Zero-Shot Multilingual Neural Machine Translation

  2021cites this paper
• Improving Data Augmentation for Low-Resource NMT Guided by POS-Tagging and Paraphrase Embedding

  2021cites this paper
• Alternative Input Signals Ease Transfer in Multilingual Machine Translation

  2021cites this paper
• Impact of Filtering Generated Pseudo Bilingual Texts in Low-Resource Neural Machine Translation Enhancement: The Case of Persian-Spanish

  2021cites this paper
• Cross-lingual learning for text processing: A survey

  2021cites this paper
• CUNI systems for WMT21: Multilingual Low-Resource Translation for Indo-European Languages Shared Task

  2021cites this paper
• IndicBART: A Pre-trained Model for Natural Language Generation of Indic Languages

  2021cites this paper
• Improving Zero-shot Multilingual Neural Machine Translation for Low-Resource Languages

  2021cites this paper
• Multilingual Translation via Grafting Pre-trained Language Models

  2021cites this paper
• Multi-Task Learning for Improving Gender Accuracy in Neural Machine Translation

  2021cites this paper
• Can Cognate Prediction Be Modelled as a Low-Resource Machine Translation Task?

  2021cites this paper
• Edinburgh’s End-to-End Multilingual Speech Translation System for IWSLT 2021

  2021cites this paper
• Learning Collections of Functions

  2021influential citation
• Multilingual Agreement for Multilingual Neural Machine Translation

  2021cites this paper
• Multilingual Simultaneous Neural Machine Translation

  2021cites this paper
• An Improved English-to-Mizo Neural Machine Translation

  2021cites this paper
• Modeling Task-Aware MIMO Cardinality for Efficient Multilingual Neural Machine Translation

  2021cites this paper
• Multi-Lingual Machine Translation Ph.D. Thesis Proposal

  2021cites this paper
• Research on the Application of BERT in Mongolian-Chinese Neural Machine Translation

  2021cites this paper
• Share or Not? Learning to Schedule Language-Specific Capacity for Multilingual Translation

  2021cites this paper
• Pre-trained Language Model for Web-scale Retrieval in Baidu Search

  2021cites this paper
• AugVic: Exploiting BiText Vicinity for Low-Resource NMT

  2021cites this paper
• Neural Machine Translation for Low-resource Languages: A Survey

  2021cites this paper
• Zero-Shot Neural Machine Translation with Self-Learning Cycle

  2021cites this paper