Bleu: a Method for Automatic Evaluation of Machine Translation

Human evaluations of machine translation are extensive but expensive. Human evaluations can take months to finish and involve human labor that can not be reused. We propose a method of automatic machine translation evaluation that is quick, inexpensive, and language-independent, that correlates highly with human evaluation, and that has little marginal cost per run. We present this method as an automated understudy to skilled human judges which substitutes for them when there is need for quick or frequent evaluations.

• Publication year

  2002

• Venue

  Annual Meeting of the Association for Computational Linguistics

• Publication date

  2002-07-06

• Fields of study

  Linguistics, Computer Science

• Identifiers
  DOI 10.3115/1073083.1073135
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Corpus-based comprehensive and diagnostic MT evaluation: initial Arabic, Chinese, French, and Spanish results

  2002cited by this paper
• The ARPA MT Evaluation Methodologies: Evolution, Lessons, and Future Approaches

  1994cited by this paper
• Proficiency and Performance in Language Testing.

  1993cited by this paper

• A dynamic hybrid network with attention and mamba for image captioning

  2026cites this paper
• Knowledge-Augmented Adaptive Mechanism for Radiology Report Generation

  2026cites this paper
• LLM-as-a-Judge for Software Engineering: Literature Review, Vision, and the Road Ahead

  2026influential citation
• Bridging Visual Analysis and Text Generation: A Hierarchical Multi-scale Visual Feature Flow Model for Accessible Radiographic Report Automation

  2026cites this paper
• In Line with Context: Repository-Level Code Generation via Context Inlining

  2026cites this paper
• A knowledge-driven approach for automated fire safety compliance checking in operational buildings

  2026cites this paper
• Graph-guided topic modeling and multi-level context-aware framework for automated radiology report generation

  2026cites this paper
• Is more human-like better? The impact of virtual streamer language naturalness on user engagement

  2026cites this paper
• Auto-encoding clinical language for zero-shot medical report generation

  2026cites this paper
• LEAF++: Trait-Annotated Essay Feedback Dataset With Validation of Trait Scores via LLMs

  2026cites this paper
• Towards More Accurate App Review Response Generation with Fine-Grained Filtering

  2026influential citation
• STELLAR: A Search-Based Testing Framework for Large Language Model Applications

  2026cites this paper
• Question Answering for Multi-Release Systems: A Case Study at Ciena

  2026cites this paper
• How do language models handle emotional content in video game localization? A computational linguistics approach

  2026cites this paper
• Semantic Communication-Based Aerial–Maritime Energy Trade-Off for Maritime Mobile Edge Computing Networks Under Jamming

  2026cites this paper
• ERFC: Energy-Aware Reinforcement Feedback Calibration for Zero-Shot Captioning

  2026cites this paper
• Visual Evidence-Aware for Object Hallucinations Rectification in LLM-Based Video Captioning

  2026cites this paper
• Dynamic feature fusion guiding and multimodal large language model refining for medical image report generation

  2026cites this paper
• Enhancing vision-and-language transformers through two-stage generative alignment pre-training

  2026cites this paper
• Building regulation question-answering system using retrieval-augmented generation with dual-stage fine-tuned large language model

  2026cites this paper
• GMAIL: Generative Modality Alignment for generated Image Learning

  2026cites this paper
• Syntax-aware question generation through dependency relations-guided attention

  2026cites this paper
• Causal-ESC : Capture the Dynamics in Cause-and-Effect Detection for Emotional Support Conversation

  2026cites this paper
• Evolutionary-Algorithm-Based Automatic Prompt Generation for Vision-Language Model Evaluation

  2026cites this paper
• ECG-aBcDe: Overcoming model dependence, encoding ECG into a universal language for any large language model

  2026cites this paper
• JP-TL-Bench: Anchored Pairwise LLM Evaluation for Bidirectional Japanese-English Translation

  2026cites this paper
• Reinforcement-Learned Unequal Error Protection for Quantized Semantic Embeddings

  2026cites this paper
• Theory Trace Card: Theory-Driven Socio-Cognitive Evaluation of LLMs

  2026cites this paper
• The New Compiler Stack: A Survey on the Synergy of LLMs and Compilers

  2026cites this paper
• Comparative Analysis of Medical-Domain and General-Purpose Large Language Models: Evaluating Specialization in Healthcare Applications

  2026cites this paper
• COSMIC: A Novel Contextualized Orientation Similarity Metric Incorporating Consistency for NLG Assessment

  2026influential citation
• LSSF: Safety Alignment for Large Language Models through Low-Rank Safety Subspace Fusion

  2026influential citation
• Knowledge Graph-Enhanced Triplet Semantic Communication System for Low-Altitude IoT Transmission

  2026cites this paper
• Structure Observation Driven Image-Text Contrastive Learning for Computed Tomography Report Generation

  2026cites this paper
• LLM-infused multi-module transformer for emotion-aware sentiment analysis in few-shot scenarios

  2026cites this paper
• LRMTL: An efficient multitask learning network for breast tumor segmentation and medical report generation

  2026cites this paper
• Text-Conditional Visual-Language Alignment for Video Captioning

  2026cites this paper
• A Federated Adaptive Large Language Model Fine-Tuning Framework for Software Development

  2026cites this paper
• Boosting unit test generation via structure-aware fine-tuning of pre-trained model

  2026cites this paper
• An attention-enhanced deep learning model for detecting vessel anomalous behavior

  2026cites this paper
• Echo: Generating cross-modal features for unseen classes in zero-shot remote sensing image captioning

  2026cites this paper
• Progressive multi-level learning for gloss-free sign language translation

  2026cites this paper
• Automatic Detection of Unsafe Behaviors Resulting from the Improper Use of PPE during Human-Robot Collaboration

  2026cites this paper
• Evaluating Semantic Segmentation–Based Scene Descriptions for Multilane Rural Highway Point Clouds against Ground Truth

  2026cites this paper
• Multimodal language models in agriculture: A tutorial and survey

  2026cites this paper
• Cross-multi-modal seamless training for image captioning

  2026cites this paper
• PreEduAI: A Multi-Agent Collaborative Framework for Automated Preschool Curriculum Development

  2026cites this paper
• Enhancing Low-Resource Indian Language Machine Translation Using Large Language Models With Preference Optimization and Hypergeometric-Gamma Reward

  2026cites this paper
• Dual-Prompt-Aware Cross-Modal Semantic Interaction and Fusion Network for Remote Sensing Image Captioning

  2026cites this paper
• Structural Ambiguity Resolution in Indonesian–English Speech-to-Text Translation by Utilizing Prosodic Information

  2026cites this paper
• Toward Comprehensive Interactive Change Understanding in Remote Sensing: A Large-Scale Dataset and Dual-Granularity Enhanced VLM

  2026cites this paper
• Human-LLM Collaboration Framework for Translating Health Instruments

  2026influential citation
• Low Rank Comes with Low Security: Gradient Assembly Poisoning Attacks against Distributed LoRA-based LLM Systems

  2026cites this paper
• The Role of Mixed-Language Documents for Multilingual Large Language Model Pretraining

  2026cites this paper
• Parallel Universes, Parallel Languages: A Comprehensive Study on LLM-based Multilingual Counterfactual Example Generation

  2026influential citation
• Pat-DEVAL: Chain-of-Legal-Thought Evaluation for Patent Description

  2026cites this paper
• TeleDoCTR: Domain-Specific and Contextual Troubleshooting for Telecommunications

  2026cites this paper
• EmoHarbor: Evaluating Personalized Emotional Support by Simulating the User's Internal World

  2026cites this paper
• SPoRC-VIST: A Benchmark for Evaluating Generative Natural Narrative in Vision-Language Models

  2026cites this paper
• SeRe: A Security-Related Code Review Dataset Aligned with Real-World Review Activities

  2026cites this paper
• DéjàQ: Open-Ended Evolution of Diverse, Learnable and Verifiable Problems

  2026cites this paper
• Towards Automated Lexicography: Generating and Evaluating Definitions for Learner's Dictionaries

  2026influential citation
• SlowFocus: Enhancing Fine-grained Temporal Understanding in Video LLM

  2026cites this paper
• Vision-Semantics-Label: A New Two-Step Paradigm for Action Recognition With Large Language Model

  2026cites this paper
• Topic Enhanced Semantic Communication System for Reliable Semantic Recovery

  2026cites this paper
• Latent Variable Modeling for Controllable and Diverse Generation From Large Language Models

  2026cites this paper
• Generative Semantic Communications for Robust Speech-to-Text Translation

  2026cites this paper
• IGen: Redefining long-term event prediction with iterative generation and dynamic balancing

  2026cites this paper
• A Secure and Efficient Distributed Semantic Communication System for Heterogeneous Internet of Things

  2026cites this paper
• Leveraging Sentence-oriented Augmentation and Transformer-Based Architecture for Vietnamese-Bahnaric Translation

  2026cites this paper
• Data Divergence-Aware Client Selection via Knowledge Graph for Federated LLM Fine-Tuning

  2026cites this paper
• ComAI: The Convergence of Communication and Artificial Intelligence

  2026cites this paper
• Labeling-free RAG-enhanced LLM for intelligent fault diagnosis via reinforcement learning

  2026cites this paper
• Continual End-to-End Speech-to-Text translation using augmented bi-sampler

  2026cites this paper
• Automatic perception of potential safety hazards: A cross-modal multi-task framework for feature alignment, image classification and captioning

  2026cites this paper
• Eliminate-Then-Select: A human-centric reasoning framework for educational question answering with LLMs

  2026cites this paper
• Questions Beyond Pixels: Integrating Commonsense Knowledge in Visual Question Generation for Remote Sensing

  2026cites this paper
• Dynamic Persona Generation through commonsense inference in dialogues

  2026cites this paper
• Predictive typing for the Persian language: A survey

  2026cites this paper
• LBKD: Rethinking Federated Backdoors for Low-Altitude Economy via LLMs and Bidirectional Knowledge Distillation

  2026cites this paper
• Large Language Model–Based Construction Site Management for Severe Weather Preparedness

  2026cites this paper
• Simulating online political discourse using LLMs: A study on social identity and conflict

  2026cites this paper
• From hallucinations to hazards: benchmarking LLMs for hazard analysis in safety-critical systems

  2026cites this paper
• DockerFill: Automatically Completing Dockerfile Code With Syntax-Aware Multi-Task Learning

  2026cites this paper
• Using Knowledge Induction strategies: LLMs can do better in knowledge-driven dialogue tasks

  2026cites this paper
• Using LLMs to enhance code quality: A systematic literature review

  2026cites this paper
• Multi-level semantics-aware and multi-granularity knowledge-infused model for emotional support conversation

  2026cites this paper
• Assessing the potential of LLMs as crowdworkers for contextual information generation

  2026cites this paper
• "Detectors Lead, LLMs Follow": Integrating LLMs and traditional models on implicit hate speech detection to generate faithful and plausible explanations

  2026cites this paper
• An Empirical Study of Parameter-Efficient Fine-Tuning in Code Change Learning and Beyond

  2026cites this paper
• VLMAR: Maritime scene anomaly detection via retrieval-augmented vision-language models

  2026cites this paper
• DM-RRG: Radiology image report generation with disease-topic guidance and multi-grained Image-Text alignment

  2026cites this paper
• DualFocus-CapNet: A Dual-Stream Network for Real Change and Interscale Relationship-Aware Change Captioning in Remote Sensing Images

  2026cites this paper
• Pixels to prose: A comprehensive survey of image captioning techniques with deep learning and generative artificial intelligence

  2026cites this paper
• Generative Agents Navigating Digital Libraries

  2026influential citation
• SWMCC: A Lightweight Network for Remote Sensing Change Caption

  2026cites this paper
• From human and object interaction to threat detection: An interpretable threat detection method for human violence scenarios

  2026cites this paper
• HaiNougat: An academic document parser that preserves formulas and tables for high-energy physics

  2026cites this paper
• NL2ACSL : Interactively translating natural language to ANSI C specification language with large language models

  2026cites this paper
• An Empirical Study of On-Device Translation for Real-Time Live-Stream Chat on Mobile Devices

  2026influential citation