Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism

We propose multi-way, multilingual neural machine translation. The proposed approach enables a single neural translation model to translate between multiple languages, with a number of parameters that grows only linearly with the number of languages. This is made possible by having a single attention mechanism that is shared across all language pairs. We train the proposed multi-way, multilingual model on ten language pairs from WMT'15 simultaneously and observe clear performance improvements over models trained on only one language pair. In particular, we observe that the proposed model significantly improves the translation quality of low-resource language pairs.

• Publication year

  2016

• Venue

  North American Chapter of the Association for Computational Linguistics

• Publication date

  2016-01-06

• Fields of study

  Mathematics, Linguistics, Computer Science

• Identifiers
  DOI 10.18653/v1/N16-1101arXiv 1601.01073
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Connectionist Temporal Classification: Labelling Unsegmented Sequences with Recurrent Neural Networks

  2016cited by this paper
• Blocks and Fuel: Frameworks for deep learning

  2015cited by this paper
• On Using Monolingual Corpora in Neural Machine Translation

  2015cited by this paper
• Multi-task Sequence to Sequence Learning

  2015cited by this paper
• Improving Neural Machine Translation Models with Monolingual Data

  2015cited by this paper
• Attention-Based Models for Speech Recognition

  2015cited by this paper
• Effective Approaches to Attention-based Neural Machine Translation

  2015influential reference
• Montreal Neural Machine Translation Systems for WMT’15

  2015influential reference
• Neural Machine Translation of Rare Words with Subword Units

  2015cited by this paper
• Multi-Task Learning for Multiple Language Translation

  2015influential reference
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• Theano-based Large-Scale Visual Recognition with Multiple GPUs

  2014cited by this paper
• On Using Very Large Target Vocabulary for Neural Machine Translation

  2014influential reference
• Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

  2014influential reference
• Sequence to Sequence Learning with Neural Networks

  2014cited by this paper
• Neural Machine Translation by Jointly Learning to Align and Translate

  2014influential reference
• On the Properties of Neural Machine Translation: Encoder–Decoder Approaches

  2014influential reference
• Theano: new features and speed improvements

  2012cited by this paper
• On the difficulty of training recurrent neural networks

  2012cited by this paper
• Natural Language Processing (Almost) from Scratch

  2011cited by this paper
• A Novel Connectionist System for Unconstrained Handwriting Recognition

  2009cited by this paper
• Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks

  2006cited by this paper
• Long Short-Term Memory

  1997cited by this paper
• Global training of document processing systems using graph transformer networks

  1997cited by this paper
• Multitask Learning

  1997cited by this paper

• Contrastive learning for antibody-antigen sequence-to-specificity prediction

  2026cites this paper
• Multilingual Neural Machine Translation for Asian Language Treebank

  2026cites this paper
• Engine remaining useful life prediction method based on deep residual network and attention mechanism

  2026cites this paper
• Maithilimt: Developing Multi-Domain Parallel Corpus for Hindi-Maithili Machine Translation

  2026cites this paper
• The Unreasonable Effectiveness of Model Merging for Cross-Lingual Transfer in LLMs

  2025cites this paper
• FuxiMT: Sparsifying Large Language Models for Chinese-Centric Multilingual Machine Translation

  2025cites this paper
• The Use of Artificial Intelligence in the Rendition of Certain Articles of Ḥanbalī Sharīʿa Code into English: A Comparison of Automatic and Human Evaluation of Translation Output Quality of ChatGPT and Gemini in Arabic–English Legal Translation

  2025cites this paper
• Fostering Digital Inclusion for Low-Resource Nigerian Languages: A Case Study of Igbo and Nigerian Pidgin

  2025cites this paper
• Exploration of Vocabulary Expansion and Self-Supervised Enhancement Methods for Han-Zhuang Machine Translation

  2025cites this paper
• How Vital Is the Jurisprudential Relevance: Law Article-Intervened Legal Case Retrieval and Matching

  2025cites this paper
• A Novel Approach to Continual Knowledge Transfer in Multilingual Neural Machine Translation Using Autoregressive and Non-Autoregressive Models for Indic Languages

  2025cites this paper
• LegoMT2: Selective Asynchronous Sharded Data Parallel Training for Massive Neural Machine Translation

  2025cites this paper
• Contextual Cues in Machine Translation: Investigating the Potential of Multi-Source Input Strategies in LLMs and NMT Systems

  2025cites this paper
• Multilingual Non-Autoregressive Machine Translation without Knowledge Distillation

  2025cites this paper
• Self-attention encoder-decoder with model adaptation for transliteration and translation tasks in regional language

  2025cites this paper
• Deromanization of Hindi-English Code-Mixed Text and its Influence on Toxic Comment Classification and Sentiment Analysis

  2025cites this paper
• Character-Level Encoding based Neural Machine Translation for Hindi language

  2025influential citation
• Transferring Zero-shot Multilingual Chinese-Chinese Translation Model for Chinese Minority Language Translation

  2024cites this paper
• Large Language Models are Parallel Multilingual Learners

  2024cites this paper
• AnyTrans: Translate AnyText in the Image with Large Scale Models

  2024cites this paper
• End-To-End Lingao Dialect Speech Recognition

  2024cites this paper
• CULL-MT: Compression Using Language and Layer pruning for Machine Translation

  2024cites this paper
• Can the Variation of Model Weights be used as a Criterion for Self-Paced Multilingual NMT?

  2024cites this paper
• Modularized Multilingual NMT with Fine-grained Interlingua

  2024influential citation
• Application of Hierarchical Attention Network in Vulnerability Detection Model

  2024cites this paper
• Language-agnostic Zero-Shot Machine Translation with Language-specific Modeling

  2024cites this paper
• Learned image compression via neighborhood-based attention optimization and context modeling with multi-scale guiding

  2024cites this paper
• TAeKD: Teacher Assistant Enhanced Knowledge Distillation for Closed-Source Multilingual Neural Machine Translation

  2024cites this paper
• Attention Mechanisms in Clinical Text Classification: A Comparative Evaluation

  2024cites this paper
• Parallel Corpus Augmentation using Masked Language Models

  2024cites this paper
• Investigating the translation capabilities of Large Language Models trained on parallel data only

  2024cites this paper
• Modality Invariant Multimodal Learning to Handle Missing Modalities: A Single-Branch Approach

  2024cites this paper
• LCS: A Language Converter Strategy for Zero-Shot Neural Machine Translation

  2024cites this paper
• Improving NMT from a Low-Resource Source Language: A Use Case from Catalan to Chinese via Spanish

  2024cites this paper
• Accurate House Price Projections

  2024cites this paper
• Translating scientific abstracts in the bio-medical domain with structure-aware models

  2024cites this paper
• Improving Multilingual Neural Machine Translation by Utilizing Semantic and Linguistic Features

  2024cites this paper
• A Systematic Analysis of Subwords and Cross-Lingual Transfer in Multilingual Translation

  2024cites this paper
• SkillNet-X: A Multilingual Multitask Model with Sparsely Activated Skills

  2023cites this paper
• A Local Attention-based Neural Networks for Abstractive Text Summarization

  2023cites this paper
• A Voyage on Neural Machine Translation for Indic Languages

  2023cites this paper
• Transformer: A General Framework from Machine Translation to Others

  2023cites this paper
• Communication Efficient Federated Learning for Multilingual Neural Machine Translation with Adapter

  2023cites this paper
• Multilingual Knowledge Graph Completion with Language-Sensitive Multi-Graph Attention

  2023cites this paper
• Deep Learning Based Text Translation and Summarization Tool for Hearing Impaired Using Indian Sign Language

  2023cites this paper
• Improving Zero-shot Multilingual Neural Machine Translation by Leveraging Cross-lingual Consistency Regularization

  2023cites this paper
• Variable-length Neural Interlingua Representations for Zero-shot Neural Machine Translation

  2023cites this paper
• An Empirical Study of Leveraging Knowledge Distillation for Compressing Multilingual Neural Machine Translation Models

  2023cites this paper
• Bilex Rx: Lexical Data Augmentation for Massively Multilingual Machine Translation

  2023cites this paper
• Dual channel Chinese sentiment analysis of characters and words based on deep learning

  2023cites this paper
• Low-resource Multilingual Neural Translation Using Linguistic Feature-based Relevance Mechanisms

  2023cites this paper
• Efficiently Upgrading Multilingual Machine Translation Models to Support More Languages

  2023cites this paper
• Train Global, Tailor Local: Minimalist Multilingual Translation into Endangered Languages

  2023cites this paper
• Joint Training and Decoding for Multilingual End-to-End Simultaneous Speech Translation

  2023cites this paper
• Exploiting Multilingualism in Low-resource Neural Machine Translation via Adversarial Learning

  2023cites this paper
• Dozens of Translation Directions or Millions of Shared Parameters? Comparing Two Types of Multilinguality in Modular Machine Translation

  2023cites this paper
• Exploring Geometric Representational Disparities between Multilingual and Bilingual Translation Models

  2023cites this paper
• Towards Higher Pareto Frontier in Multilingual Machine Translation

  2023cites this paper
• Small Data, Big Impact: Leveraging Minimal Data for Effective Machine Translation

  2023cites this paper
• Deep Learning for Natural Language Processing: A Survey

  2023cites this paper
• Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOM

  2023cites this paper
• Combination of deep learning with representation learning in X-ray prohibited item detection

  2023cites this paper
• Multilingual Entity and Relation Extraction from Unified to Language-specific Training

  2023cites this paper
• Filtering and Extended Vocabulary based Translation for Low-resource Language Pair of Sanskrit-Hindi

  2023cites this paper
• Transfer learning in low-resourced MT: An empirical study

  2023cites this paper
• Can Big Models Help Diverse Languages? Investigating Large Pretrained Multilingual Models for Machine Translation of Indian Languages

  2023cites this paper
• Neural Machine Translation System for Pakistani Languages

  2023cites this paper
• Integrating LSTM and NLP techniques for essay generation

  2023cites this paper
• Use of Neural Machine Translation in Multimodal Translation

  2023cites this paper
• GATITOS: Using a New Multilingual Lexicon for Low-resource Machine Translation

  2023cites this paper
• Order Matters in the Presence of Dataset Imbalance for Multilingual Learning

  2023cites this paper
• Improving Chinese-Centric Low-Resource Translation Using English-Centric Pivoted Parallel Data

  2023cites this paper
• A Closer Look at Transformer Attention for Multilingual Translation

  2023cites this paper
• КОНТЕКСТУАЛЬНИЙ ОПЕРАЦІОНАЛІЗМ В АВТОМАТИЗОВАНОМУ ПЕРЕКЛАДІ ТЕРМІНОСПОЛУКИ: НОВИЙ ПІДХІД ДО ПЕРЕКЛАДУ

  2023cites this paper
• Joint Modeling of Chinese Minority Language Translation Tasks

  2023cites this paper
• Direct Neural Machine Translation with Task-level Mixture of Experts models

  2023cites this paper
• Research on Improving Neural Machine Translation Performance with Multi-Source Input Data

  2023cites this paper
• How To Build Competitive Multi-gender Speech Translation Models For Controlling Speaker Gender Translation

  2023cites this paper
• A Benchmark for Learning to Translate a New Language from One Grammar Book

  2023cites this paper
• Unlikelihood Tuning on Negative Samples Amazingly Improves Zero-Shot Translation

  2023cites this paper
• Towards English-centric Zero-shot Neural Machine Translation: The Analysis and Solution

  2023cites this paper
• Is Robustness Transferable across Languages in Multilingual Neural Machine Translation?

  2023cites this paper
• An Ensemble Approach to Personalized Real Time Predictive Writing for Experts

  2023cites this paper
• Unsupervised anomalies detection in IIoT edge devices networks using federated learning

  2023cites this paper
• TAN-IBE: Neural Machine Translation for the romance languages of the Iberian Peninsula

  2023cites this paper
• Investigating Unsupervised Neural Machine Translation for Low-resource Language Pair English-Mizo via Lexically Enhanced Pre-trained Language Models

  2023cites this paper
• Learning Nearest Neighbour Informed Latent Word Embeddings to Improve Zero-Shot Machine Translation

  2023cites this paper
• Exploring Linguistic Similarity and Zero-Shot Learning for Multilingual Translation of Dravidian Languages

  2023cites this paper
• Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decoding

  2023cites this paper
• Investigating Multi-Pivot Ensembling with Massively Multilingual Machine Translation Models

  2023cites this paper
• Exploring Representational Disparities Between Multilingual and Bilingual Translation Models

  2023cites this paper
• Incorporating attentive multi-scale context information for image captioning

  2022cites this paper
• Backpropagation-Based Decoding for Multimodal Machine Translation

  2022cites this paper
• Multilingual Neural Machine Translation with Deep Encoder and Multiple Shallow Decoders

  2022cites this paper
• Knowledge Distillation-Based Multilingual Code Retrieval

  2022cites this paper
• Investigating Massive Multilingual Pre-Trained Machine Translation Models for Clinical Domain via Transfer Learning

  2022cites this paper
• MMTAfrica: Multilingual Machine Translation for African Languages

  2022influential citation
• Towards the Next 1000 Languages in Multilingual Machine Translation: Exploring the Synergy Between Supervised and Self-Supervised Learning

  2022cites this paper
• Language-Family Adapters for Low-Resource Multilingual Neural Machine Translation

  2022cites this paper
• A two-stream attention mechanisms network

  2022cites this paper