Continuous Space Language Models for Statistical Machine Translation

Statistical machine translation systems are based on one or more translation models and a language model of the target language. While many different translation models and phrase extraction algorithms have been proposed, a standard word n-gram back-off language model is used in most systems. In this work, we propose to use a new statistical language model that is based on a continuous representation of the words in the vocabulary. A neural network is used to perform the projection and the probability estimation. We consider the translation of European Parliament Speeches. This task is part of an international evaluation organized by the TC-STAR project in 2006. The proposed method achieves consistent improvements in the BLEU score on the development and test data. We also present algorithms to improve the estimation of the language model probabilities when splitting long sentences into shorter chunks.

• Publication year

  2006

• Venue

  Annual Meeting of the Association for Computational Linguistics

• Publication date

  2006-07-17

• Fields of study

  Linguistics, Computer Science

• Identifiers
  DOI 10.3115/1273073.1273166
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Smooth Bilingual N-Gram Translation

  2007cited by this paper
• Efficient Handling of N-gram Language Models for Statistical Machine Translation

  2007cited by this paper
• Moses: Open Source Toolkit for Statistical Machine Translation

  2007cited by this paper
• Randomised Language Modelling for Statistical Machine Translation

  2007cited by this paper
• Continuous space language models

  2007cited by this paper
• Continuous space language models for the IWSLT 2006 task

  2006cited by this paper
• The 2006 LIMSI Statistical Machine Translation System for TC-STAR

  2006cited by this paper
• Reranking Translation Hypotheses Using Structural Properties

  2006cited by this paper
• Cross domain automatic transcription on the TC-STAR EPPS corpus

  2005cited by this paper
• CONDOR, a new parallel, constrained extension of Powell's UOBYQA algorithm: experimental results and comparison with the DFO algorithm

  2005cited by this paper
• Random clusterings for language modeling

  2005influential reference
• Bilingual N-gram Statistical Machine Translation

  2005cited by this paper
• Improved Language Modeling for Statistical Machine Translation

  2005cited by this paper
• Training Neural Network Language Models on Very Large Corpora

  2005influential reference
• Random Forests in Language Modelin

  2004influential reference
• A Smorgasbord of Features for Statistical Machine Translation

  2004cited by this paper
• Efficient training of large neural networks for language modeling

  2004cited by this paper
• A Neural Probabilistic Language Model

  2003cited by this paper
• SRILM - an extensible language modeling toolkit

  2002cited by this paper
• Discriminative Training and Maximum Entropy Models for Statistical Machine Translation

  2002cited by this paper
• A Maximum Entropy Approach to Natural Language Processing

  1996cited by this paper
• An Empirical Study of Smoothing Techniques for Language Modeling

  1996cited by this paper
• The Mathematics of Statistical Machine Translation: Parameter Estimation

  1993cited by this paper
• The mathematics of statistical machine translation

  1993cited by this paper

• Using Kolmogorov–Arnold Networks in Transformer Model: A Study on Low-Resource Neural Machine Translation

  2025cites this paper
• An Analysis of the Training Data Impact for Domain-Adapted Tokenizer Performances—The Case of Serbian Legal Domain Adaptation

  2025cites this paper
• Large Language Models: A Survey

  2024cites this paper
• Review of Hierarchical Transfer Learning Architecture in Low-Resource Machine Translation

  2024cites this paper
• From Bytes to Borsch: Fine-Tuning Gemma and Mistral for the Ukrainian Language Representation

  2024cites this paper
• Open foundation models for Azerbaijani language

  2024cites this paper
• First Tragedy, then Parse: History Repeats Itself in the New Era of Large Language Models

  2024cites this paper
• The role of large language models in agriculture: harvesting the future with LLM intelligence

  2024cites this paper
• First Tragedy, then Parse: History Repeats Itself in the New Era of Large Language Models

  2023cites this paper
• Fundamentals of Generative Large Language Models and Perspectives in Cyber-Defense

  2023cites this paper
• Global Ontology Entities Embeddings

  2023cites this paper
• Low-Resource Neural Machine Translation: A Systematic Literature Review

  2023cites this paper
• ChatGPT Perpetuates Gender Bias in Machine Translation and Ignores Non-Gendered Pronouns: Findings across Bengali and Five other Low-Resource Languages

  2023cites this paper
• The importance of Term Weighting in semantic understanding of text: A review of techniques

  2022cites this paper
• Non-Autoregressive Neural Machine Translation: A Call for Clarity

  2022cites this paper
• Computational authorship analysis of the homeric poems

  2022cites this paper
• A Transformer-Based Neural Machine Translation Model for Arabic Dialects That Utilizes Subword Units

  2021cites this paper
• CHAPTER 10 Machine Translation andEncoder-Decoder Models

  2021cites this paper
• Lerna: transformer architectures for configuring error correction tools for short- and long-read genome sequencing

  2021cites this paper
• Revisiting Words

  2020cites this paper
• Neural Machine Translation

  2020cites this paper
• Neural Language Models

  2020cites this paper
• Uses of Machine Translation

  2020cites this paper
• The Translation Problem

  2020cites this paper
• Phrase-Based Statistical Machine Translation of Hindi Poetries into English

  2020cites this paper
• Towards a Better Understanding of Label Smoothing in Neural Machine Translation

  2020cites this paper
• Detecting the Most Insightful Parts of Documents Using a Regularized Attention-Based Model

  2020cites this paper
• Computational Science – ICCS 2020: 20th International Conference, Amsterdam, The Netherlands, June 3–5, 2020, Proceedings, Part I

  2020cites this paper
• Computational Science – ICCS 2020: 20th International Conference, Amsterdam, The Netherlands, June 3–5, 2020, Proceedings, Part III

  2020cites this paper
• Deep learning techniques applied to constituency parsing of German

  2020cites this paper
• The Roles of Language Models and Hierarchical Models in Neural Sequence-to-Sequence Prediction

  2020cites this paper
• Current Challenges

  2020cites this paper
• A Survey of Deep Learning Techniques for Neural Machine Translation

  2020cites this paper
• Linguistic Structure

  2020cites this paper
• Index

  2020cites this paper
• Preface

  2020cites this paper
• Alternate Architectures

  2020cites this paper
• Bibliography

  2020cites this paper
• Beyond Parallel Corpora

  2020cites this paper
• Computation Graphs

  2020cites this paper
• Machine Learning Tricks

  2020cites this paper
• Analysis and Visualization

  2020cites this paper
• Neural Translation Models

  2020cites this paper
• Réseaux de neurones profonds appliqués à la compréhension de la parole. (Deep learning applied to spoken langage understanding)

  2019cites this paper
• Multi-sense embeddings through a word sense disambiguation process

  2019cites this paper
• Context in Neural Machine Translation: A Review of Models and Evaluations

  2019influential citation
• A Subword Level Language Model for Bangla Language

  2019cites this paper
• Online Parallel Data Extraction with Neural Machine Translation

  2019cites this paper
• Neural Machine Translation: A Review

  2019cites this paper
• Fast and Efficient Parallel Alignment Model for Aligning both Long and Short Sentences

  2019cites this paper
• A Continuous Space Neural Language Model for Bengali Language

  2019cites this paper
• Multimodal machine translation through visuals and speech

  2019cites this paper
• Efficient Estimation of Ontology Entities Distributed Representations

  2019cites this paper
• Attention Mechanism in Machine Translation

  2019cites this paper
• Evaluation

  2019cites this paper
• On internal language representations in deep learning: an analysis of machine translation and speech recognition

  2018cites this paper
• Fallback Variable History NNLMs: Efficient NNLMs by precomputation and stochastic training

  2018cites this paper
• Knowledge Base Population based on Entity Graph Analysis. (Peuplement d'une base de connaissance fondé sur l'exploitation d'un graphe d'entités)

  2018cites this paper
• Apprentissage profond pour le traitement automatique des langues [Deep Learning for Natural Language Procesing]

  2018cites this paper
• Apply Chinese Radicals Into Neural Machine Translation: Deeper Than Character Level

  2018cites this paper
• Incorporating Chinese Radicals Into Neural Machine Translation: Deeper Than Character Level

  2018cites this paper
• A Multitask-Based Neural Machine Translation Model with Part-of-Speech Tags Integration for Arabic Dialects

  2018cites this paper
• Neural and non-neural approaches to authorship attribution

  2018cites this paper
• Syntactic and semantic features for statistical and neural machine translation

  2018cites this paper
• Neural Language Models

  2018cites this paper
• Going beyond the sentence: Contextual Machine Translation of Dialogue. (Au-delà de la phrase: traduction automatique de dialogue en contexte)

  2018cites this paper
• Predicting and discovering linguistic structure with neural networks

  2018cites this paper
• A Character Prediction Approach in a Security Context using a Recurrent Neural Network

  2018cites this paper
• Tightly-coupled convolutional neural network with spatial-temporal memory for text classification

  2017cites this paper
• A Content-Based Neural Reordering Model for Statistical Machine Translation

  2017cites this paper
• Étude sur les représentations continues de mots appliquées à la détection automatique des erreurs de reconnaissance de la parole. (A study of continuous word representations applied to the automatic detection of speech recognition errors)

  2017cites this paper
• Towards Document-Level Neural Machine Translation

  2017cites this paper
• Exploring the language modeling toolkits for Arabic text

  2017cites this paper
• Integrated Speech and Language Technology for Intelligence, Surveillance, and Reconnaissance (ISR)

  2017cites this paper
• Encoder-decoder neural networks

  2017cites this paper
• Practical Neural Machine Translation

  2017cites this paper
• LIUM Machine Translation Systems for WMT17 News Translation Task

  2017cites this paper
• Étude sur les représentations continues de mots appliquées à la détection automatique des erreurs de reconnaissance de la parole

  2017cites this paper
• Translating Low-Resource Languages by Vocabulary Adaptation from Close Counterparts

  2017cites this paper
• Neural Network Methods for Natural Language Processing

  2017cites this paper
• Text, Speech, and Dialogue

  2016influential citation
• Does Multimodality Help Human and Machine for Translation and Image Captioning?

  2016influential citation
• Automatic Speech Recognition Based on Neural Networks

  2016cites this paper
• Exponentially Decaying Bag-of-Words Input Features for Feed-Forward Neural Network in Statistical Machine Translation

  2016cites this paper
• SubGram: Extending Skip-Gram Word Representation with Substrings

  2016cites this paper
• Scalable Machine Translation in Memory Constrained Environments

  2016cites this paper
• LIMSI@WMT’16: Machine Translation of News

  2016cites this paper
• Converting Continuous-Space Language Models into N-gram Language Models with Efficient Bilingual Pruning for Statistical Machine Translation

  2016influential citation
• Learning Distributed Representations of Data in Community Question Answering for Question Retrieval

  2016cites this paper
• 15th International Conference on Frontiers in Handwriting Recognition, ICFHR 2016, Shenzhen, China, October 23-26, 2016

  2016cites this paper
• The 54th Annual Meeting of the Association for Computational Linguistics

  2016cites this paper
• Aligning the foundations of hierarchical statistical machine translation

  2016cites this paper
• Handwriting and Speech Recognition: From Bayes Decision Rule to Deep Neural Networks

  2016cites this paper
• Utilisation des représentations continues des mots et des paramètres prosodiques pour la détection d’erreurs dans les transcriptions automatiques de la parole (Combining continuous word representation and prosodic features for ASR error detection)[In French]

  2016cites this paper
• Deliverable D 1 . 5 Improved Learning for Machine Translation

  2016cites this paper
• Neural Machine Translation

  2016cites this paper
• A Primer on Neural Network Models for Natural Language Processing

  2015cites this paper
• The Geometry of Statistical Machine Translation

  2015cites this paper
• Neural Network Language Model for Chinese Pinyin Input Method Engine

  2015cites this paper
• Learning visually grounded meaning representations

  2015cites this paper