Confidence Estimation for Machine Translation

We present a detailed study of confidence estimation for machine translation. Various methods for determining whether MT output is correct are investigated, for both whole sentences and words. Since the notion of correctness is not intuitively clear in this context, different ways of defining it are proposed. We present results on data from the NIST 2003 Chinese-to-English MT evaluation.

• Publication year

  2004

• Venue

  International Conference on Computational Linguistics

• Publication date

  2004-08-23

• Fields of study

  Linguistics, Computer Science

• Identifiers
  DOI 10.3115/1220355.1220401
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Pattern Classification

  2012cited by this paper
• Applied Logistic Regression: Hosmer/Applied Logistic Regression

  2005cited by this paper
• The Alignment Template Approach to Statistical Machine Translation

  2004cited by this paper
• The Mathematics of Machine Translation : Parameter

  2004cited by this paper
• Confidence measures for statistical machine translation

  2003influential reference
• Precision and Recall of Machine Translation

  2003cited by this paper
• Improving utterance verification using a smoothed naive Bayes model

  2003cited by this paper
• Confidence estimation for translation prediction

  2003cited by this paper
• Automatic Evaluation of Machine Translation Quality Using N-gram Co-Occurrence Statistics

  2002cited by this paper
• Discriminative Training and Maximum Entropy Models for Statistical Machine Translation

  2002cited by this paper
• Recognition confidence scoring and its use in speech understanding systems

  2002cited by this paper
• A formal approach to score normalization for meta-search

  2002cited by this paper
• Bleu: a Method for Automatic Evaluation of Machine Translation

  2002cited by this paper
• Automatic evaluation of machine translation quality using n-gram co-occurrence statistics

  2002influential reference
• Robust semantic confidence scoring

  2002cited by this paper
• Torch: a modular machine learning software library

  2002cited by this paper
• An Adapted Lesk Algorithm for Word Sense Disambiguation Using WordNet

  2002cited by this paper
• Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL)

  2002cited by this paper
• Is this conversation on track?

  2001cited by this paper
• Word level confidence annotation using combinations of features

  2001cited by this paper
• Robust confidence annotation and rejection for continuous speech recognition

  2001influential reference
• A support vector machines-based rejection technique for speech recognition

  2001cited by this paper
• Confidence measures for large vocabulary continuous speech recognition

  2001cited by this paper
• A boosting approach for confidence scoring

  2001cited by this paper
• Rule Writing or Annotation: Cost-efficient Resource Usage for Base Noun Phrase Chunking

  2000cited by this paper
• Finding consensus in speech recognition: word error minimization and other applications of confusion networks

  2000cited by this paper
• Evaluation of word confidence for speech recognition systems

  1999influential reference
• Maximum entropy models for natural language ambiguity resolution

  1998cited by this paper
• Word-based confidence measures as a guide for stack search in speech recognition

  1997cited by this paper
• Neural-network based measures of confidence for word recognition

  1997cited by this paper
• Estimating confidence using word lattices

  1997cited by this paper
• The Bootstrap and Edgeworth Expansion

  1997cited by this paper
• A probabilistic approach to confidence estimation and evaluation

  1997cited by this paper
• A Maximum Entropy Approach to Natural Language Processing

  1996cited by this paper
• An Introduction to the Bootstrap

  1995cited by this paper
• Using Information Content to Evaluate Semantic Similarity in a Taxonomy

  1995cited by this paper
• Lexical chains as representations of context for the detection and correction of malapropisms

  1995influential reference
• The Mathematics of Statistical Machine Translation: Parameter Estimation

  1993cited by this paper
• Neural Networks for Pattern Recognition

  1993cited by this paper
• The Bootstrap and Edgeworth Expansion

  1992cited by this paper
• Information Retrieval: Data Structures and Algorithms

  1992cited by this paper
• Stacked generalization

  1992cited by this paper
• Multilayer feedforward networks are universal approximators

  1989cited by this paper
• Numerical Recipes in C: The Art of Scientific Computing

  1989cited by this paper
• Mathematical Statistics and Data Analysis

  1988cited by this paper

• Hindsight Quality Prediction Experiments in Multi-Candidate Human-Post-Edited Machine Translation

  2026cites this paper
• MICE for CATs: Model-Internal Confidence Estimation for Calibrating Agents with Tools

  2025cites this paper
• Faithfulness metric fusion: Improving the evaluation of LLM trustworthiness across domains

  2025cites this paper
• Calibrating LLMs for Text-to-SQL Parsing by Leveraging Sub-clause Frequencies

  2025cites this paper
• Lost in Translation? Found in Evaluation: A Comprehensive Survey on Sentence-Level Translation Evaluation

  2025cites this paper
• Can Multiple Responses from an LLM Reveal the Sources of Its Uncertainty?

  2025cites this paper
• QE4PE: Word-level Quality Estimation for Human Post-Editing

  2025cites this paper
• Towards More Reliable Chinese Spelling Correction: Fine-Grained Confidence Estimation Against Suboptimal Corrections

  2025cites this paper
• PIE: Performance Interval Estimation for Free-Form Generation Tasks

  2025cites this paper
• Using Source-Side Confidence Estimation for Reliable Translation into Unfamiliar Languages

  2025cites this paper
• A confidence-based knowledge integration framework for cross-domain table question answering

  2024cites this paper
• From Handcrafted Features to LLMs: A Brief Survey for Machine Translation Quality Estimation

  2024cites this paper
• EPOQUE: An English-Persian Quality Estimation Dataset

  2024cites this paper
• A Review of Faithfulness Metrics for Hallucination Assessment in Large Language Models

  2024cites this paper
• Interpreting Predictive Probabilities: Model Confidence or Human Label Variation?

  2024cites this paper
• Calibrating Large Language Models Using Their Generations Only

  2024cites this paper
• Word-level Translation Quality Estimation Based on Optimal Transport

  2024cites this paper
• Multi-view fusion for universal translation quality estimation

  2024cites this paper
• Towards Precise Localization of Critical Errors in Machine Translation

  2024cites this paper
• Bad MT Systems are Good for Quality Estimation

  2023cites this paper
• SemSyn: Semantic-Syntactic Similarity Based Automatic Machine Translation Evaluation Metric

  2023cites this paper
• A Survey on Learning to Reject

  2023influential citation
• Poor Man’s Quality Estimation: Predicting Reference-Based MT Metrics Without the Reference

  2023cites this paper
• CEScore: Simple and Efficient Confidence Estimation Model for Evaluating Split and Rephrase

  2023cites this paper
• Approximating to the Real Translation Quality for Neural Machine Translation via Causal Motivated Methods

  2023cites this paper
• Physician Detection of Clinical Harm in Machine Translation: Quality Estimation Aids in Reliance and Backtranslation Identifies Critical Errors

  2023cites this paper
• Do Text Simplification Systems Preserve Meaning? A Human Evaluation via Reading Comprehension

  2023cites this paper
• Improving Translation Quality Estimation with Bias Mitigation

  2023cites this paper
• Unsupervised Translation Quality Estimation Exploiting Synthetic Data and Pre-trained Multilingual Encoder

  2023cites this paper
• Learning from Past Mistakes: Quality Estimation from Monolingual Corpora and Machine Translation Learning Stages

  2023cites this paper
• Research on quality estimation of Chinese-English translation based on machine learning

  2023cites this paper
• Quality-Aware Translation Models: Efficient Generation and Quality Estimation in a Single Model

  2023cites this paper
• ELIZAVETA YANKOVSKAYA Quality Estimation through Attention

  2022cites this paper
• Papago’s Submission to the WMT22 Quality Estimation Shared Task

  2022cites this paper
• REUSE: REference-free UnSupervised Quality Estimation Metric

  2022cites this paper
• A review of machine transliteration, translation, evaluation metrics and datasets in Indian Languages

  2022cites this paper
• On Dimensions of Plausibility for Narrative Information Access to Digital Libraries

  2022cites this paper
• Alibaba-Translate China’s Submission for WMT 2022 Quality Estimation Shared Task

  2022cites this paper
• Rethinking Round-trip Translation for Automatic Machine Translation Evaluation

  2022cites this paper
• Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data

  2022cites this paper
• Machine Translation Performance Prediction System: Optimal Prediction for Optimal Translation

  2022cites this paper
• PreQuEL: Quality Estimation of Machine Translation Outputs in Advance

  2022cites this paper
• Bias Mitigation in Machine Translation Quality Estimation

  2022cites this paper
• A Comprehensive Survey on Various Fully Automatic Machine Translation Evaluation Metrics

  2022cites this paper
• Understanding and Being Understood: User Strategies for Identifying and Recovering From Mistranslations in Machine Translation-Mediated Chat

  2022cites this paper
• Neural Models for Measuring Confidence on Interactive Machine Translation Systems

  2022cites this paper
• QUAK: A Synthetic Quality Estimation Dataset for Korean-English Neural Machine Translation

  2022cites this paper
• Perceptual Quality Dimensions of Machine-Generated Text with a Focus on Machine Translation

  2022cites this paper
• A Linguistically Motivated Test Suite to Semi-Automatically Evaluate German–English Machine Translation Output

  2022cites this paper
• Welocalize-ARC/NKUA’s Submission to the WMT 2022 Quality Estimation Shared Task

  2022cites this paper
• 利用语义关联增强的跨语言预训练模型的译文质量评估(A Cross-language Pre-trained Model with Enhanced Semantic Connection for MT Quality Estimation)

  2021cites this paper
• Ensemble Fine-tuned mBERT for Translation Quality Estimation

  2021cites this paper
• Translation Error Detection as Rationale Extraction

  2021cites this paper
• BJTU-Toshiba's Submission to CCMT 2021 QE and APE Task

  2021cites this paper
• Continual Quality Estimation with Online Bayesian Meta-Learning

  2021cites this paper
• Identifying Incorrect Classifications with Balanced Uncertainty

  2021cites this paper
• Diagnosing Transformers in Task-Oriented Semantic Parsing

  2021cites this paper
• Papago’s Submission for the WMT21 Quality Estimation Shared Task

  2021cites this paper
• IST-Unbabel 2021 Submission for the Quality Estimation Shared Task

  2021cites this paper
• Direct Exploitation of Attention Weights for Translation Quality Estimation

  2021cites this paper
• Ensemble-based Transfer Learning for Low-resource Machine Translation Quality Estimation

  2021cites this paper
• Uncertainty-Aware Machine Translation Evaluation

  2021cites this paper
• Explainable Quality Estimation: CUNI Eval4NLP Submission

  2021cites this paper
• IST-Unbabel 2021 Submission for the Explainable Quality Estimation Shared Task

  2021cites this paper
• Error Identification for Machine Translation with Metric Embedding and Attention

  2021cites this paper
• Backtranslation Feedback Improves User Confidence in MT, Not Quality

  2021cites this paper
• Transformer-based conformal predictors for paraphrase detection

  2021cites this paper
• Confidence Measures for Interactive Neural Machine Translation

  2021cites this paper
• Assessing Reference-Free Peer Evaluation for Machine Translation

  2021cites this paper
• Detecting over/under-translation errors for determining adequacy in human translations

  2021cites this paper
• Trustworthy Long-Tailed Classification

  2021cites this paper
• CLIPScore: A Reference-free Evaluation Metric for Image Captioning

  2021cites this paper
• Thought Flow Nets: From Single Predictions to Trains of Model Thought

  2021cites this paper
• The Eval4NLP Shared Task on Explainable Quality Estimation: Overview and Results

  2021cites this paper
• SynXLM-R: Syntax-Enhanced XLM-R in Translation Quality Estimation

  2021cites this paper
• Bayesian Model-Agnostic Meta-Learning with Matrix-Valued Kernels for Quality Estimation

  2021cites this paper
• Beyond Glass-Box Features: Uncertainty Quantification Enhanced Quality Estimation for Neural Machine Translation

  2021influential citation
• Three Directions for the Design of Human-Centered Machine Translation

  2021cites this paper
• QEMind: Alibaba’s Submission to the WMT21 Quality Estimation Shared Task

  2021cites this paper
• Quality Estimation without Human-labeled Data

  2021cites this paper
• Quality Estimation Using Dual Encoders with Transfer Learning

  2021cites this paper
• Artificial intelligence for classifying uncertain images by humans in determining choroidal vascular running pattern and comparisons with automated classification between artificial intelligence

  2021cites this paper
• Contrastive Learning for Machine Translation Quality Estimation

  2021cites this paper
• Fine-Grained Error Analysis on English-to-Japanese Machine Translation in the Medical Domain

  2020cites this paper
• Multimodal Quality Estimation for Machine Translation

  2020cites this paper
• Quality In, Quality Out: Learning from Actual Mistakes

  2020cites this paper
• A Post-Editing Dataset in the Legal Domain: Do we Underestimate Neural Machine Translation Quality?

  2020cites this paper
• Wat zei je? Detecting Out-of-Distribution Translations with Variational Transformers

  2020cites this paper
• A Hybrid Approach for Improved Low Resource Neural Machine Translation using Monolingual Data

  2020cites this paper
• Are we Estimating or Guesstimating Translation Quality?

  2020cites this paper
• Unsupervised Quality Estimation for Neural Machine Translation

  2020cites this paper
• Fuzzy-Match Repair Guided by Quality Estimation

  2020cites this paper
• Generating Counter Narratives against Online Hate Speech: Data and Strategies

  2020cites this paper
• DeepSubQE: Quality estimation for subtitle translations

  2020cites this paper
• Balancing Quality and Human Involvement: An Effective Approach to Interactive Neural Machine Translation

  2020cites this paper
• IST-Unbabel Participation in the WMT20 Quality Estimation Shared Task

  2020cites this paper
• The NiuTrans System for the WMT20 Quality Estimation Shared Task

  2020cites this paper
• Supervised Classification Based Machine Translation Quality Estimation

  2020cites this paper
• Confidence Estimation via Auxiliary Models

  2020cites this paper
• BERGAMOT-LATTE Submissions for the WMT20 Quality Estimation Shared Task

  2020cites this paper