UHH Submission to the WMT17 Metrics Shared Task

In this paper the UHH submission to the WMT17 Metrics Shared Task is presented, which is based on sequence and tree kernel functions applied to the reference and candidate translations. In addition we also explore the effect of applying the kernel functions on the source sentence and a back-translation of the MT output, but also on the pair composed of the candidate translation and a pseudo-reference of the source segment. The newly proposed metric was evaluated using the data from WMT16, with the results demonstrating a high correlation with human judgments.

• Publication year

  2017

• Venue

  Conference on Machine Translation

• Publication date

  Unknown publication date

• Fields of study

  Linguistics, Computer Science

• Identifiers
  DOI 10.18653/v1/W17-4766
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• KELP: a Kernel-based Learning Platform

  2018cited by this paper
• Findings of the 2016 Conference on Machine Translation

  2016cited by this paper
• KeLP: a Kernel-based Learning Platform for Natural Language Processing

  2015cited by this paper
• LAYERED: Metric for Machine Translation Evaluation

  2014cited by this paper
• Using Discourse Structure Improves Machine Translation Evaluation

  2014cited by this paper
• The CMU Machine Translation Systems at WMT 2014

  2014cited by this paper
• Semantic Compositionality in Tree Kernels

  2014cited by this paper
• Collaborative Dependency Annotation

  2013cited by this paper
• The CMU Machine Translation Systems at WMT 2013: Syntax, Synthetic Translation Options, and Pseudo-References

  2013cited by this paper
• Towards Compositional Tree Kernels

  2013cited by this paper
• Semantic Textual Similarity for MT evaluation

  2012cited by this paper
• Structured Lexical Similarity via Convolution Kernels on Dependency Trees

  2011influential reference
• Top Accuracy and Fast Dependency Parsing is not a Contradiction

  2010cited by this paper
• Convolution Kernels on Constituent, Dependency and Sequential Structures for Relation Extraction

  2009cited by this paper
• Syntax-Oriented Evaluation Measures for Machine Translation Output

  2009cited by this paper
• The Backtranslation Score: Automatic MT Evalution at the Sentence Level without Reference Translations

  2009cited by this paper
• The Role of Pseudo References in MT Evaluation

  2008cited by this paper
• Word Error Rates: Decomposition over POS classes and Applications for Error Analysis

  2007cited by this paper
• Regression for Sentence-Level MT Evaluation with Pseudo References

  2007cited by this paper
• Linguistic Features for Automatic Evaluation of Heterogenous MT Systems

  2007cited by this paper
• CDER: Efficient MT Evaluation Using Block Movements

  2006cited by this paper
• Efficient Convolution Kernels for Dependency and Constituent Syntactic Trees

  2006influential reference
• A Study of Translation Edit Rate with Targeted Human Annotation

  2006cited by this paper
• A STUDY OF TRANSLATION ERROR RATE WITH TARGETED HUMAN ANNOTATION

  2005cited by this paper
• Syntactic Features for Evaluation of Machine Translation

  2005cited by this paper
• Subsequence Kernels for Relation Extraction

  2005cited by this paper
• Automatic evaluation of machine translation quality using n-gram co-occurrence statistics

  2002cited by this paper
• Bleu: a Method for Automatic Evaluation of Machine Translation

  2002cited by this paper
• New Ranking Algorithms for Parsing and Tagging: Kernels over Discrete Structures, and the Voted Perceptron

  2002cited by this paper
• Automatic Evaluation of Machine Translation Quality Using N-gram Co-Occurrence Statistics

  2002cited by this paper
• Convolution Kernels for Natural Language

  2001cited by this paper
• Accelerated DP based search for statistical translation

  1997cited by this paper

• Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing

  2020cites this paper
• Revisit Automatic Error Detection for Wrong and Missing Translation – A Supervised Approach

  2019influential citation
• Results of the WMT18 Metrics Shared Task: Both characters and embeddings achieve good performance

  2018cites this paper
• Results of the WMT17 Metrics Shared Task

  2017cites this paper