Referential Translation Machines for Predicting Translation Quality

We use referential translation machines (RTM) for quality estimation of translation outputs. RTMs are a computational model for identifying the translation acts between any two data sets with respect to interpretants selected in the same domain, which are effective when making monolingual and bilingual similarity judgments. RTMs achieve top performance in automatic, accurate, and language independent prediction of sentence-level and word-level statistical machine translation (SMT) quality. RTMs remove the need to access any SMT system specific information or prior knowledge of the training data or models used when generating the translations and achieve the top performance in WMT13 quality estimation task (QET13). We improve our RTM models with the Parallel FDA5 instance selection model, with additional features for predicting the translation performance, and with improved learning models. We develop RTM models for each WMT14 QET (QET14) subtask, obtain improvements over QET13 results, and rank 1st in all of the tasks and subtasks of QET14.

• Publication year

  2014

• Venue

  WMT@ACL

• Publication date

  2014-06-26

• Fields of study

  Linguistics, Computer Science

• Identifiers
  DOI 10.3115/v1/W14-3339
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Optimizing Instance Selection for Statistical Machine Translation with Feature Decay Algorithms

  2015cited by this paper
• SemEval-2014 Task 1: Evaluation of Compositional Distributional Semantic Models on Full Sentences through Semantic Relatedness and Textual Entailment

  2014cited by this paper
• SemEval-2014 Task 10: Multilingual Semantic Textual Similarity

  2014cited by this paper
• SemEval-2014 Task 3: Cross-Level Semantic Similarity

  2014cited by this paper
• RTM-DCU: Referential Translation Machines for Semantic Similarity

  2014cited by this paper
• Parallel FDA5 for Fast Deployment of Accurate Statistical Machine Translation Systems

  2014influential reference
• Findings of the 2014 Workshop on Statistical Machine Translation

  2014cited by this paper
• Gene Selection for Cancer Classification using Support Vector Machines

  2014cited by this paper
• QuEst – Design, Implementation and Extensions of a Framework for Machine Translation Quality Estimation

  2013cited by this paper
• Findings of the 2013 Workshop on Statistical Machine Translation

  2013cited by this paper
• Referential Translation Machines for Quality Estimation

  2013influential reference
• Predicting sentence translation quality using extrinsic and language independent features

  2013influential reference
• FBK-UEdin Participation to the WMT13 Quality Estimation Shared Task

  2013cited by this paper
• The Regression Model of Machine Translation

  2012cited by this paper
• Findings of the 2012 Workshop on Statistical Machine Translation

  2012influential reference
• UKP: Computing Semantic Textual Similarity by Combining Multiple Content Similarity Measures

  2012cited by this paper
• RegMT System for Machine Translation, System Combination, and Evaluation

  2011cited by this paper
• Instance Selection for Machine Translation using Feature Decay Algorithms

  2011influential reference
• Distributed Training Strategies for the Structured Perceptron

  2010cited by this paper
• Estimating the Sentence-Level Quality of Machine Translation Systems

  2009influential reference
• Fluency, Adequacy, or HTER? Exploring Different Human Judgments with a Tunable MT Metric

  2009influential reference
• Consensus ontologies in socially interacting MultiAgent systems

  2008cited by this paper
• Learning syntactic structure

  2007influential reference
• Moses: Open Source Toolkit for Statistical Machine Translation

  2007cited by this paper
• Tregex and Tsurgeon: tools for querying and manipulating tree data structures

  2006cited by this paper
• Extremely randomized trees

  2006cited by this paper
• A STUDY OF TRANSLATION ERROR RATE WITH TARGETED HUMAN ANNOTATION

  2005cited by this paper
• A tutorial on support vector regression

  2004cited by this paper
• Bleu: a Method for Automatic Evaluation of Machine Translation

  2002influential reference
• Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms

  2002influential reference
• Automatic evaluation of machine translation quality using n-gram co-occurrence statistics

  2002influential reference
• SRILM - an extensible language modeling toolkit

  2002cited by this paper
• Gene Selection for Cancer Classification using Support Vector Machines

  2002cited by this paper
• Automatic Evaluation of Machine Translation Quality Using N-gram Co-Occurrence Statistics

  2002influential reference
• Motivations

  2000influential reference
• Asymptotically Optimal Choice of ε-Loss for Support Vector Machines

  1998cited by this paper
• Building a Large Annotated Corpus of English: The Penn Treebank

  1993cited by this paper
• The Mathematics of Statistical Machine Translation: Parameter Estimation

  1993cited by this paper

• Automatic Prediction of the Performance of Every Parser

  2024cites this paper
• Word-Level Quality Estimation for Korean-English Neural Machine Translation

  2022cites this paper
• Word-level Korean-English Quality Estimation

  2021cites this paper
• Quality Estimation with Transformer and RNN Architectures

  2019cites this paper
• Towards a post-editing recommendation system for Spanish-Basque machine translation

  2018cites this paper
• Proceedings of the 21st Annual Conference of the European Association for Machine Translation

  2018cites this paper
• Dieses Modell erweitert das " Multi-Layer Perceptron " von

  2016cites this paper
• Document-level machine translation quality estimation

  2016cites this paper
• Referential Translation Machines for Predicting Translation Performance

  2016cites this paper
• The role of syntax and semantics in machine translation and quality estimation of machine-translated user-generated content

  2015cites this paper
• Correlations of perceived post-editing effort with measurements of actual effort

  2015cites this paper
• RTM-DCU: Predicting Semantic Similarity with Referential Translation Machines

  2015influential citation
• Referential translation machines for predicting semantic similarity

  2015influential citation
• Multi-level Translation Quality Prediction with QuEst++

  2015cites this paper
• Findings of the 2015 Workshop on Statistical Machine Translation

  2015cites this paper
• Referential Translation Machines for Predicting Translation Quality and Related Statistics

  2015influential citation
• Optimizing Instance Selection for Statistical Machine Translation with Feature Decay Algorithms

  2015cites this paper
• Large Scale Translation Quality Estimation

  2015cites this paper
• Using on-line available sources of bilingual information for word-level machine translation quality estimation

  2015cites this paper
• Using on-line available sources of bilingual information for word-level machine translation quality estimation

  2015cites this paper
• Correlating decoding events with errors in Statistical Machine Translation

  2014cites this paper
• Findings of the 2014 Workshop on Statistical Machine Translation

  2014cites this paper
• RTM-DCU: Referential Translation Machines for Semantic Similarity

  2014influential citation
• Target-Centric Features for Translation Quality Estimation

  2014cites this paper
• Workshop and Shared Task Proceedings

  2014cites this paper