Findings of the 2015 Workshop on Statistical Machine Translation

This paper presents the results of the WMT15 shared tasks, which included a standard news translation task, a metrics task, a tuning task, a task for run-time estimation of machine translation quality, and an automatic post-editing task. This year, 68 machine translation systems from 24 institutions were submitted to the ten translation directions in the standard translation task. An additional 7 anonymized systems were included, and were then evaluated both automatically and manually. The quality estimation task had three subtasks, with a total of 10 teams, submitting 34 entries. The pilot automatic postediting task had a total of 4 teams, submitting 7 entries.

• Publication year

  2015

• Venue

  WMT@EMNLP

• Publication date

  2015-09-01

• Fields of study

  Linguistics, Computer Science

• Identifiers
  DOI 10.18653/v1/W15-3001
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Multi-level Translation Quality Prediction with QuEst++

  2015cited by this paper
• The FBK Participation in the WMT15 Automatic Post-editing Shared Task

  2015cited by this paper
• Montreal Neural Machine Translation Systems for WMT’15

  2015cited by this paper
• Abu-MaTran at WMT 2015 Translation Task: Morphological Segmentation and Web Crawling

  2015influential reference
• LIMSI@WMT’15 : Translation Task

  2015cited by this paper
• Results of the WMT15 Tuning Shared Task

  2015cited by this paper
• Why Predicting Post-Edition is so Hard? Failure Analysis of LIMSI Submission to the APE Shared Task

  2015cited by this paper
• The KIT-LIMSI Translation System for WMT 2015

  2015cited by this paper
• USHEF and USAAR-USHEF participation in the WMT15 QE shared task

  2015influential reference
• UdS-Sant: English–German Hybrid Machine Translation System

  2015cited by this paper
• DFKI’s experimental hybrid MT system for WMT 2015

  2015cited by this paper
• Data enhancement and selection strategies for the word-level Quality Estimation

  2015cited by this paper
• Searching for Context: a Study on Document-Level Labels for Translation Quality Estimation

  2015influential reference
• CimS - The CIS and IMS Joint Submission to WMT 2015 addressing morphological and syntactic differences in English to German SMT

  2015cited by this paper
• The AFRL-MITLL WMT15 System: There’s More than One Way to Decode It!

  2015cited by this paper
• GF Wide-coverage English-Finnish MT system for WMT 2015

  2015cited by this paper
• Exploring the Planet of the APEs: a Comparative Study of State-of-the-art Methods for MT Automatic Post-Editing

  2015cited by this paper
• Tuning Phrase-Based Segmented Translation for a Morphologically Complex Target Language

  2015cited by this paper
• QUality Estimation from ScraTCH (QUETCH): Deep Learning for Word-level Translation Quality Estimation

  2015cited by this paper
• The University of Illinois submission to the WMT 2015 Shared Translation Task

  2015cited by this paper
• CUNI in WMT15: Chimera Strikes Again

  2015cited by this paper
• Referential Translation Machines for Predicting Translation Quality and Related Statistics

  2015cited by this paper
• Morphological Segmentation and OPUS for Finnish-English Machine Translation

  2015cited by this paper
• Sheffield Systems for the Finnish-English WMT Translation Task

  2015cited by this paper
• UGENT-LT3 SCATE System for Machine Translation Quality Estimation

  2015cited by this paper
• Strategy-Based Technology for Estimating MT Quality

  2015cited by this paper
• The Edinburgh/JHU Phrase-based Machine Translation Systems for WMT 2015

  2015cited by this paper
• New Language Pairs in TectoMT

  2015cited by this paper
• Exact Decoding with Multi Bottom-Up Tree Transducers

  2015cited by this paper
• UAlacant word-level machine translation quality estimation system at WMT 2015

  2015cited by this paper
• Improving Evaluation of Machine Translation Quality Estimation

  2015cited by this paper
• USAAR-SAPE: An English–Spanish Statistical Automatic Post-Editing System

  2015cited by this paper
• SHEF-NN: Translation Quality Estimation with Neural Networks

  2015cited by this paper
• Using on-line available sources of bilingual information for word-level machine translation quality estimation

  2015cited by this paper
• The Karlsruhe Institute of Technology Translation Systems for the WMT 2015

  2015cited by this paper
• Results of the WMT15 Metrics Shared Task

  2015cited by this paper
• LORIA System for the WMT15 Quality Estimation Shared Task

  2015cited by this paper
• The RWTH Aachen German-English Machine Translation System for WMT 2015

  2015cited by this paper
• Using on-line available sources of bilingual information for word-level machine translation quality estimation

  2015cited by this paper
• Statistical Machine Translation

  2014cited by this paper
• The repetition rate of text as a predictor of the effectiveness of machine translation adaptation

  2014cited by this paper
• FBK-UPV-UEdin participation in the WMT14 Quality Estimation shared-task

  2014cited by this paper
• Findings of the 2014 Workshop on Statistical Machine Translation

  2014cited by this paper
• Referential Translation Machines for Predicting Translation Quality

  2014cited by this paper
• Edinburgh’s Syntax-Based Systems at WMT 2015

  2014cited by this paper
• The RWTH Aachen German-English Machine Translation System for WMT 2014

  2014cited by this paper
• LIG System for Word Level QE task at WMT14

  2014cited by this paper
• Design and Analysis of a Large Corpus of Post-Edited Translations: Quality Estimation, Failure Analysis and the Variability of Post-Edition

  2013influential reference
• QuEst - A translation quality estimation framework

  2013cited by this paper
• Reducing Annotation Effort for Quality Estimation via Active Learning

  2013cited by this paper
• LIG System for WMT13 QE Task: Investigating the Usefulness of Features in Word Confidence Estimation for MT

  2013cited by this paper
• SHEF-Lite: When Less is More for Translation Quality Estimation

  2013cited by this paper
• A Hybrid Word Alignment Model for Phrase-Based Statistical Machine Translation

  2013cited by this paper
• Referential Translation Machines for Quality Estimation

  2013cited by this paper
• Findings of the 2013 Workshop on Statistical Machine Translation

  2013cited by this paper
• Can Automatic Post-Editing Make MT More Meaningful

  2012cited by this paper
• Incremental adaptation using translation information and post-editing analysis

  2012cited by this paper
• Appraise: an Open-Source Toolkit for Manual Evaluation of MT Output

  2012cited by this paper
• Collection of a Large Database of French-English SMT Output Corrections

  2012cited by this paper
• FreeLing 3.0: Towards Wider Multilinguality

  2012cited by this paper
• Findings of the 2012 Workshop on Statistical Machine Translation

  2012influential reference
• “This sentence is wrong.” Detecting errors in machine-translated sentences

  2011cited by this paper
• KenLM: Faster and Smaller Language Model Queries

  2011cited by this paper
• Statistical Post-Editing for a Statistical MT System

  2011cited by this paper
• Findings of the 2011 Workshop on Statistical Machine Translation

  2011cited by this paper
• SyMGiza++: Symmetrized Word Alignment Models for Statistical Machine Translation

  2011cited by this paper
• cdec: A Decoder, Alignment, and Learning Framework for Finite- State and Context-Free Translation Models

  2010cited by this paper
• Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation

  2010cited by this paper
• Asiya: An Open Toolkit for Automatic Machine Translation (Meta-)Evaluation

  2010cited by this paper
• Findings of the 2009 Workshop on Statistical Machine Translation

  2009cited by this paper
• Further Meta-Evaluation of Machine Translation

  2008cited by this paper
• Parallel Implementations of Word Alignment Tool

  2008cited by this paper
• Moses: Open Source Toolkit for Statistical Machine Translation

  2007influential reference
• METEOR: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments

  2007cited by this paper
• Statistical Phrase-Based Post-Editing

  2007influential reference
• (Meta-) Evaluation of Machine Translation

  2007cited by this paper
• Alignment by Agreement

  2006cited by this paper
• A Study of Translation Edit Rate with Targeted Human Annotation

  2006influential reference
• Open Source Toolkit for Statistical Machine Translation: Factored Translation Models and Lattice Decoding

  2006cited by this paper
• Manual and Automatic Evaluation of Machine Translation between European Languages

  2006cited by this paper
• Europarl: A Parallel Corpus for Statistical Machine Translation

  2005cited by this paper
• METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments

  2005influential reference
• A STUDY OF TRANSLATION ERROR RATE WITH TARGETED HUMAN ANNOTATION

  2005cited by this paper
• Statistical Significance Tests for Machine Translation Evaluation

  2004cited by this paper
• Minimum Error Rate Training in Statistical Machine Translation

  2003cited by this paper
• Bleu: a Method for Automatic Evaluation of Machine Translation

  2002influential reference
• Language Processing

  1983cited by this paper
• The measurement of observer agreement for categorical data.

  1977cited by this paper
• Machine Translation

  1972cited by this paper
• A Coefficient of Agreement for Nominal Scales

  1960cited by this paper

• Scaling Model and Data for Multilingual Machine Translation with Open Large Language Models

  2026cites this paper
• OpenLID-v3: Improving the Precision of Closely Related Language Identification -- An Experience Report

  2026cites this paper
• KD4MT: A Survey of Knowledge Distillation for Machine Translation

  2026cites this paper
• AP-OOD: Attention Pooling for Out-of-Distribution Detection

  2026influential citation
• A decade of gender bias in machine translation

  2025cites this paper
• Assessing the Political Fairness of Multilingual LLMs: A Case Study based on a 21-way Multiparallel EuroParl Dataset

  2025cites this paper
• Multi-objective Hyperparameter Optimization in the Age of Deep Learning

  2025cites this paper
• Giving the Old a Fresh Spin: Quality Estimation-Assisted Constrained Decoding for Automatic Post-Editing

  2025cites this paper
• Dynamic Priors in Bayesian Optimization for Hyperparameter Optimization

  2025cites this paper
• SemiAdapt and SemiLoRA: Efficient Domain Adaptation for Transformer-based Low-Resource Language Translation with a Case Study on Irish

  2025cites this paper
• Improving DNN Inference Throughput Using Practical, Per-Input Compute Adaptation

  2024cites this paper
• To Diverge or Not to Diverge: A Morphosyntactic Perspective on Machine Translation vs Human Translation

  2024cites this paper
• Evaluation and Large-scale Training for Contextual Machine Translation

  2024cites this paper
• Together We Can: Multilingual Automatic Post-Editing for Low-Resource Languages

  2024cites this paper
• Critical Learning Periods: Leveraging Early Training Dynamics for Efficient Data Pruning

  2024cites this paper
• Multitask Mayhem: Unveiling and Mitigating Safety Gaps in LLMs Fine-tuning

  2024cites this paper
• In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization

  2024cites this paper
• Findings of the WMT24 General Machine Translation Shared Task: The LLM Era Is Here but MT Is Not Solved Yet

  2024cites this paper
• Advancing electron microscopy using deep learning

  2024cites this paper
• Document-Level Machine Translation with Large-Scale Public Parallel Corpora

  2024cites this paper
• Sparks of Large Audio Models: A Survey and Outlook

  2023cites this paper
• Multi-way Variational NMT for UGC: Improving Robustness in Zero-shot Scenarios via Mixture Density Networks

  2023cites this paper
• Aligning Neural Machine Translation Models: Human Feedback in Training and Inference

  2023cites this paper
• There’s No Data like Better Data: Using QE Metrics for MT Data Filtering

  2023cites this paper
• Findings of the 2023 Conference on Machine Translation (WMT23): LLMs Are Here but Not Quite There Yet

  2023cites this paper
• GlotLID: Language Identification for Low-Resource Languages

  2023cites this paper
• Data Sampling and (In)stability in Machine Translation Evaluation

  2023cites this paper
• Language Model is a Branch Predictor for Simultaneous Machine Translation

  2023cites this paper
• TeCS: A Dataset and Benchmark for Tense Consistency of Machine Translation

  2023cites this paper
• Escaping the sentence-level paradigm in machine translation

  2023cites this paper
• PriorBand: Practical Hyperparameter Optimization in the Age of Deep Learning

  2023cites this paper
• Bring More Attention to Syntactic Symmetry for Automatic Postediting of High-Quality Machine Translations

  2023cites this paper
• AudioPaLM: A Large Language Model That Can Speak and Listen

  2023cites this paper
• Review of Machine Translation

  2023cites this paper
• MuLER: Detailed and Scalable Reference-based Evaluation

  2023cites this paper
• Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement

  2023cites this paper
• Assessing the Importance of Frequency versus Compositionality for Subword-based Tokenization in NMT

  2023cites this paper
• Predictive Pipelined Decoding: A Compute-Latency Trade-off for Exact LLM Decoding

  2023influential citation
• mSLAM: Massively multilingual joint pre-training for speech and text

  2022cites this paper
• Mu2SLAM: Multitask, Multilingual Speech and Language Models

  2022cites this paper
• Anticipation-free Training for Simultaneous Translation

  2022cites this paper
• DEMETR: Diagnosing Evaluation Metrics for Translation

  2022cites this paper
• PEACook: Post-editing Advancement Cookbook

  2022cites this paper
• High-Resource Methodological Bias in Low-Resource Investigations

  2022cites this paper
• IndicMT Eval: A Dataset to Meta-Evaluate Machine Translation Metrics for Indian Languages

  2022cites this paper
• A Linguistically Motivated Test Suite to Semi-Automatically Evaluate German–English Machine Translation Output

  2022cites this paper
• Perceptual Quality Dimensions of Machine-Generated Text with a Focus on Machine Translation

  2022cites this paper
• Advancing Semi-Supervised Learning for Automatic Post-Editing: Data-Synthesis by Mask-Infilling with Erroneous Terms

  2022cites this paper
• Additive Interventions Yield Robust Multi-Domain Machine Translation Models

  2022cites this paper
• Confident Adaptive Language Modeling

  2022cites this paper
• Leveraging unsupervised and weakly-supervised data to improve direct speech-to-speech translation

  2022cites this paper
• Modeling Target-Side Morphology in Neural Machine Translation: A Comparison of Strategies

  2022cites this paper
• Training language models to follow instructions with human feedback

  2022cites this paper
• Toward a Test Set of Dislocations in Persian for Neural Machine Translation

  2022cites this paper
• DataShift: A Cross-Modal Data Augmentation Method for Speech Recognition and Machine Translation

  2022cites this paper
• Active Evaluation: Efficient NLG Evaluation with Few Pairwise Comparisons

  2022influential citation
• PASHA: Efficient HPO and NAS with Progressive Resource Allocation

  2022cites this paper
• Findings of the 2022 Conference on Machine Translation (WMT22)

  2022cites this paper
• Exploring the Benefits and Limitations of Multilinguality for Non-autoregressive Machine Translation

  2022cites this paper
• Out-of-Distribution Detection and Selective Generation for Conditional Language Models

  2022cites this paper
• Analyzing Gender Translation Errors to Identify Information Flows between the Encoder and Decoder of a NMT System

  2022cites this paper
• An empirical analysis of phrase-based and neural machine translation

  2021cites this paper
• Many-to-English Machine Translation Tools, Data, and Pretrained Models

  2021cites this paper
• ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning

  2021cites this paper
• Dataset Geography: Mapping Language Data to Language Users

  2021cites this paper
• Exploration of Effective Attention Strategies for Neural Automatic Post-editing with Transformer

  2021influential citation
• SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer

  2021cites this paper
• Adapting Neural Machine Translation for Automatic Post-Editing

  2021cites this paper
• Monotonic Simultaneous Translation with Chunk-wise Reordering and Refinement

  2021cites this paper
• Automatic Text Evaluation through the Lens of Wasserstein Barycenters

  2021cites this paper
• Product Review Translation using Phrase Replacement and Attention Guided Noise Augmentation

  2021cites this paper
• Observing the Learning Curve of NMT Systems With Regard to Linguistic Phenomena

  2021cites this paper
• 融合XLM词语表示的神经机器译文自动评价方法(Neural Automatic Evaluation of Machine Translation Method Combined with XLM Word Representation)

  2021cites this paper
• Exploring the Importance of Source Text in Automatic Post-Editing for Context-Aware Machine Translation

  2021cites this paper
• Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translation

  2021cites this paper
• On nature and causes of observed MT errors

  2021cites this paper
• Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?

  2021cites this paper
• On the differences between BERT and MT encoder spaces and how to address them in translation tasks

  2021cites this paper
• Machine translation systems and quality assessment: a systematic review

  2021cites this paper
• Translation Quality Assessment: A Brief Survey on Manual and Automatic Methods

  2021cites this paper
• Editing Actions: A Missing Link Between Translation Process Research and Machine Translation Research

  2021cites this paper
• Alignment verification to improve NMT translation towards highly inflectional languages with limited resources

  2021cites this paper
• Efficient Unsupervised NMT for Related Languages with Cross-Lingual Language Models and Fidelity Objectives

  2021cites this paper
• Global Explainability of BERT-Based Evaluation Metrics by Disentangling along Linguistic Factors

  2021cites this paper
• The German EU Council Presidency Translator

  2021influential citation
• A Generative Framework for Simultaneous Machine Translation

  2021influential citation
• Translation Quality Estimation by Jointly Learning to Score and Rank

  2020cites this paper
• Dataset Augmentation for Aspect Level Sentiment Analysis

  2020cites this paper
• Synthesizing Parallel Data of User-Generated Texts with Zero-Shot Neural Machine Translation

  2020cites this paper
• 引入源端信息的机器译文自动评价方法研究(Research on Incorporating the Source Information to Automatic Evaluation of Machine Translation)

  2020cites this paper
• COMET: A Neural Framework for MT Evaluation

  2020cites this paper
• Can Automatic Post-Editing Improve NMT?

  2020cites this paper
• Controlled Document Authoring in a Machine Translation Age

  2020cites this paper
• Quality In, Quality Out: Learning from Actual Mistakes

  2020cites this paper
• Improved feature decay algorithms for statistical machine translation

  2020cites this paper
• Pragmatic information in translation: a corpus-based study of tense and mood in English and German

  2020cites this paper
• Improving Fluency of Non-Autoregressive Machine Translation

  2020cites this paper
• Can Your Context-Aware MT System Pass the DiP Benchmark Tests? : Evaluation Benchmarks for Discourse Phenomena in Machine Translation

  2020cites this paper
• Selecting Backtranslated Data from Multiple Sources for Improved Neural Machine Translation

  2020cites this paper
• Neural Polysynthetic Language Modelling

  2020cites this paper