One-Shot Neural Cross-Lingual Transfer for Paradigm Completion

We present a novel cross-lingual transfer method for paradigm completion, the task of mapping a lemma to its inflected forms, using a neural encoder-decoder model, the state of the art for the monolingual task. We use labeled data from a high-resource language to increase performance on a low-resource language. In experiments on 21 language pairs from four different language families, we obtain up to 58% higher accuracy than without transfer and show that even zero-shot and one-shot learning are possible. We further find that the degree of language relatedness strongly influences the ability to transfer morphological knowledge.

• Publication year

  2017

• Venue

  Annual Meeting of the Association for Computational Linguistics

• Publication date

  2017-03-31

• Fields of study

  Linguistics, Computer Science

• Identifiers
  DOI 10.18653/v1/P17-1182arXiv 1704.00052
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• An Empirical Comparison of Simple Domain Adaptation Methods for Neural Machine Translation

  2017cited by this paper
• An Empirical Comparison of Domain Adaptation Methods for Neural Machine Translation

  2017cited by this paper
• Single-Model Encoder-Decoder with Explicit Morphological Representation for Reinflection

  2016cited by this paper
• Polyglot Neural Language Models: A Case Study in Cross-Lingual Phonetic Representation Learning

  2016cited by this paper
• Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism

  2016cited by this paper
• Cross-Lingual Morphological Tagging for Low-Resource Languages

  2016cited by this paper
• Improving Sequence to Sequence Learning for Morphological Inflection Generation: The BIU-MIT Systems for the SIGMORPHON 2016 Shared Task for Morphological Reinflection

  2016influential reference
• MED: The LMU System for the SIGMORPHON 2016 Shared Task on Morphological Reinflection

  2016influential reference
• The Composition and Use of the Universal Morphological Feature Schema (UniMorph Schema)

  2016cited by this paper
• Many Languages, One Parser

  2016cited by this paper
• Very-large Scale Parsing and Normalization of Wiktionary Morphological Paradigms

  2016cited by this paper
• Neural Morphological Analysis: Encoding-Decoding Canonical Segments

  2016cited by this paper
• Morphological Smoothing and Extrapolation of Word Embeddings

  2016influential reference
• Toward Multilingual Neural Machine Translation with Universal Encoder and Decoder

  2016influential reference
• Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation

  2016cited by this paper
• The SIGMORPHON 2016 Shared Task—Morphological Reinflection

  2016cited by this paper
• Multi-Source Neural Translation

  2016cited by this paper
• Modeling Word Forms Using Latent Underlying Morphs and Phonology

  2015cited by this paper
• A Simple Way to Initialize Recurrent Networks of Rectified Linear Units

  2015cited by this paper
• Morphological Inflection Generation Using Character Sequence to Sequence Learning

  2015cited by this paper
• Multi-Task Learning for Multiple Language Translation

  2015cited by this paper
• Corpus-level Fine-grained Entity Typing Using Contextual Information

  2015cited by this paper
• A Language-Independent Feature Schema for Inflectional Morphology

  2015cited by this paper
• Multi-task Sequence to Sequence Learning

  2015cited by this paper
• Inflection Generation as Discriminative String Transduction

  2015influential reference
• A Graph-based Lattice Dependency Parser for Joint Morphological Segmentation and Syntactic Analysis

  2015cited by this paper
• Dropout: a simple way to prevent neural networks from overfitting

  2014influential reference
• Sequence to Sequence Learning with Neural Networks

  2014cited by this paper
• Grammar as a Foreign Language

  2014cited by this paper
• Neural Machine Translation by Jointly Learning to Align and Translate

  2014influential reference
• On the Properties of Neural Machine Translation: Encoder–Decoder Approaches

  2014cited by this paper
• Semi-supervised learning of morphological paradigms and lexicons

  2014cited by this paper
• Morphological Segmentation for Keyword Spotting

  2014cited by this paper
• Cross-lingual Projected Expectation Regularization for Weakly Supervised Learning

  2014cited by this paper
• Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers

  2013cited by this paper
• Cross-lingual Pseudo-Projected Expectation Regularization for Weakly Supervised Learning

  2013cited by this paper
• Supervised Learning of Complete Morphological Paradigms

  2013influential reference
• Speech recognition with deep recurrent neural networks

  2013cited by this paper
• Selective Sharing for Multilingual Dependency Parsing

  2012cited by this paper
• ADADELTA: An Adaptive Learning Rate Method

  2012cited by this paper
• Unsupervised Structure Prediction with Non-Parallel Multilingual Guidance

  2011cited by this paper
• A Universal Part-of-Speech Tagset

  2011influential reference
• Natural Language Processing (Almost) from Scratch

  2011cited by this paper
• Data point selection for cross-language adaptation of dependency parsers

  2011cited by this paper
• Cross Language Text Classification by Model Translation and Semi-Supervised Learning

  2010cited by this paper
• Generalizing Word Lattice Translation

  2008cited by this paper
• Unsupervised Multilingual Learning for Morphological Segmentation

  2008cited by this paper
• Cross-lingual Propagation for Morphological Analysis

  2008cited by this paper
• Analysis of Morph-Based Speech Recognition and the Modeling of Out-of-Vocabulary Words Across Languages

  2007cited by this paper
• Cross-linguistic Projection of Role-Semantic Information

  2005cited by this paper
• Cross-language text classification

  2005cited by this paper
• The use of machine translation tools for cross-lingual text mining

  2005cited by this paper
• 2005 Special Issue: Framewise phoneme classification with bidirectional LSTM and other neural network architectures

  2005cited by this paper
• Semi-Markov Conditional Random Fields for Information Extraction

  2004cited by this paper
• Inducing Multilingual Text Analysis Tools via Robust Projection across Aligned Corpora

  2001cited by this paper
• Ethnologue: Languages of the World

  1999cited by this paper
• Multitask Learning

  1997cited by this paper
• The Slavonic Languages

  1993cited by this paper
• The Uralic Languages

  1988influential reference
• Semitic Languages

  1954cited by this paper

• Revisiting and Amending Central Kurdish Data on UniMorph 4.0

  2023cites this paper
• Morphological Processing of Low-Resource Languages: Where We Are and What’s Next

  2022influential citation
• OSU at SigMorphon 2022: Analogical Inflection With Rule Features

  2022cites this paper
• Minimal Supervision for Morphological Inflection

  2021cites this paper
• To POS Tag or Not to POS Tag: The Impact of POS Tags on Morphological Learning in Low-Resource Settings

  2021cites this paper
• Computational Morphology with Neural Network Approaches

  2021influential citation
• Cross-lingual learning for text processing: A survey

  2021cites this paper
• What transfers in morphological inflection? Experiments with analogical models

  2021cites this paper
• Unsupervised Morphological Paradigm Completion

  2020cites this paper
• Weakly Supervised POS Taggers Perform Poorly on Truly Low-Resource Languages

  2020cites this paper
• On understanding character-level models for representing morphology

  2020cites this paper
• Learning to Learn Morphological Inflection for Resource-Poor Languages

  2020influential citation
• Parameter Space Factorization for Zero-Shot Learning across Tasks and Languages

  2020cites this paper
• Analogy Models for Neural Word Inflection

  2020cites this paper
• IGT2P: From Interlinear Glossed Texts to Paradigms

  2020cites this paper
• The NYU-CUBoulder Systems for SIGMORPHON 2020 Task 0 and Task 2

  2020cites this paper
• Neural sequence-to-sequence models for low-resource morphology

  2019cites this paper
• Few-Shot and Zero-Shot Learning for Historical Text Normalization

  2019cites this paper
• Char-RNN for Word Stress Detection in East Slavic Languages

  2019cites this paper
• Adversarial Multitask Learning for Joint Multi-Feature and Multi-Dialect Morphological Modeling

  2019cites this paper
• Pushing the Limits of Low-Resource Morphological Inflection

  2019cites this paper
• Towards Realistic Practices In Low-Resource Natural Language Processing: The Development Set

  2019cites this paper
• Acquisition of Inflectional Morphology in Artificial Neural Networks With Prior Knowledge

  2019influential citation
• Neural Unsupervised Parsing Beyond English

  2019cites this paper
• Fortification of Neural Morphological Segmentation Models for Polysynthetic Minimal-Resource Languages

  2018influential citation
• Near or Far, Wide Range Zero-Shot Cross-Lingual Dependency Parsing

  2018cites this paper
• On Difficulties of Cross-Lingual Transfer with Order Differences: A Case Study on Dependency Parsing

  2018cites this paper
• Normalization of historical texts with neural network models

  2018cites this paper
• Exploiting Cross-Lingual Subword Similarities in Low-Resource Document Classification

  2018cites this paper
• Does Syntactic Knowledge in Multilingual Language Models Transfer Across Languages?

  2018cites this paper
• Neural Transductive Learning and Beyond: Morphological Generation in the Minimal-Resource Setting

  2018cites this paper
• Copenhagen at CoNLL–SIGMORPHON 2018: Multilingual Inflection in Context with Explicit Morphosyntactic Decoding

  2018cites this paper
• Character-level Supervision for Low-resource POS Tagging

  2018cites this paper
• Incorporating Chinese Characters of Words for Lexical Sememe Prediction

  2018cites this paper
• The NYU System for the CoNLL–SIGMORPHON 2018 Shared Task on Universal Morphological Reinflection

  2018cites this paper
• CoNLL-SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection in 52 Languages

  2017cites this paper
• Cross-lingual Character-Level Neural Morphological Tagging

  2017cites this paper
• Training Data Augmentation for Low-Resource Morphological Inflection

  2017influential citation
• Unlabeled Data for Morphological Generation With Character-Based Sequence-to-Sequence Models

  2017cites this paper
• Exploring Cross-Lingual Transfer of Morphological Knowledge In Sequence-to-Sequence Models

  2017influential citation