Assessing the Tolerance of Neural Machine Translation Systems Against Speech Recognition Errors

Machine translation systems are conventionally trained on textual resources that do not model phenomena that occur in spoken language. While the evaluation of neural machine translation systems on textual inputs is actively researched in the literature , little has been discovered about the complexities of translating spoken language data with neural models. We introduce and motivate interesting problems one faces when considering the translation of automatic speech recognition (ASR) outputs on neural machine translation (NMT) systems. We test the robustness of sentence encoding approaches for NMT encoder-decoder modeling, focusing on word-based over byte-pair encoding. We compare the translation of utterances containing ASR errors in state-of-the-art NMT encoder-decoder systems against a strong phrase-based machine translation baseline in order to better understand which phenomena present in ASR outputs are better represented under the NMT framework than approaches that represent translation as a linear model.

• Publication year

  2017

• Venue

  Interspeech

• Publication date

  2017-08-20

• Fields of study

  Linguistics, Computer Science

• Identifiers
  DOI 10.21437/Interspeech.2017-1690arXiv 1904.10997
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Multi-way, multilingual neural machine translation

  2017cited by this paper
• Neural versus Phrase-Based Machine Translation Quality: a Case Study

  2016cited by this paper
• A Character-level Decoder without Explicit Segmentation for Neural Machine Translation

  2016cited by this paper
• FBK’s Neural Machine Translation Systems for IWSLT 2016

  2016cited by this paper
• What Makes Word-level Neural Machine Translation Hard: A Case Study on English-German Translation

  2016cited by this paper
• Phonetically-oriented word error alignment for speech recognition error analysis in speech translation

  2015influential reference
• Neural Machine Translation of Rare Words with Subword Units

  2015cited by this paper
• Stanford Neural Machine Translation Systems for Spoken Language Domains

  2015cited by this paper
• R: A language and environment for statistical computing.

  2014cited by this paper
• Assessing the impact of speech recognition errors on machine translation quality

  2014cited by this paper
• Neural Machine Translation by Jointly Learning to Align and Translate

  2014cited by this paper
• On the Properties of Neural Machine Translation: Encoder–Decoder Approaches

  2014cited by this paper
• Report on the 10th IWSLT evaluation campaign

  2013cited by this paper
• A Quantitative Analysis of Reordering Phenomena

  2009cited by this paper
• A STUDY OF TRANSLATION ERROR RATE WITH TARGETED HUMAN ANNOTATION

  2005cited by this paper
• Bleu: a Method for Automatic Evaluation of Machine Translation

  2002influential reference
• Motivations

  2000cited by this paper
• Prediction, Mixed Models, and Variance Components

  1973cited by this paper
• Edinburgh Research Explorer Experiments in Domain Adaptation for Statistical Machine Translation

  year unknowncited by this paper

• Evaluation of Speech Translation Subtitles Generated by ASR with Unnecessary Word Detection

  2024cites this paper
• Robustness of Multi-Source MT to Transcription Errors

  2023cites this paper
• Expanded Lattice Embeddings for Spoken Document Retrieval on Informal Meetings

  2022cites this paper
• The Routledge Handbook of Conference Interpreting

  2021cites this paper
• Lexical Modeling of ASR Errors for Robust Speech Translation

  2021cites this paper
• Inverted Projection for Robust Speech Translation

  2021cites this paper
• Manifold Adversarial Augmentation for Neural Machine Translation

  2021cites this paper
• Empirical Error Modeling Improves Robustness of Noisy Neural Sequence Labeling

  2021cites this paper
• Crafting ASR and Conversational Models for an Agriculture Chatbot

  2021cites this paper
• A Multi-Modal English-Italian Parallel Corpus for End-to-End Speech-to-Text Machine Translation

  2020cites this paper
• The Roles of Language Models and Hierarchical Models in Neural Sequence-to-Sequence Prediction

  2020cites this paper
• Data Augmentation for Pipeline-Based Speech Translation

  2020cites this paper
• Computer Technology Applied to Machine Translation and Speech Recognition

  2020cites this paper
• Robust Neural Machine Translation for Clean and Noisy Speech Transcripts

  2019cites this paper
• Neural Machine Translation: A Review

  2019cites this paper
• Multimodal machine translation through visuals and speech

  2019cites this paper
• Enhancing Transformer for End-to-end Speech-to-Text Translation

  2019cites this paper
• Breaking the Data Barrier: Towards Robust Speech Translation via Adversarial Stability Training

  2019cites this paper
• Adapting Transformer to End-to-End Spoken Language Translation

  2019cites this paper
• Robust Neural Machine Translation with Joint Textual and Phonetic Embedding

  2018cites this paper
• KIT’s Multilingual Neural Machine Translation systems for IWSLT 2017

  2017cites this paper
• Toward Robust Neural Machine Translation for Noisy Input Sequences

  2017cites this paper
• 36 Conference interpreting and new technologies

  year unknowncites this paper