A Methodology for Extrinsic Evaluation of Text Summarization: Does ROUGE Correlate?

No abstract is available for this paper.

• Publication year

  2005

• Venue

  IEEvaluation@ACL

• Publication date

  2005-06-01

• Fields of study

  Computer Science

• Identifiers

  No identifiers available.

• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• ROUGE: A Package for Automatic Evaluation of Summaries

  2004cited by this paper
• Evaluating Content Selection in Summarization: The Pyramid Method

  2004cited by this paper
• Squibs and Discussions: The Kappa Statistic: A Second Look

  2004cited by this paper
• ORANGE: a Method for Evaluating Automatic Evaluation Metrics for Machine Translation

  2004cited by this paper
• Extrinsic Evaluation of Automatic Metrics for Summarization

  2004cited by this paper
• Examining the consensus between human summaries: initial experiments with factoid analysis

  2003cited by this paper
• Automatic Evaluation of Summaries Using N-gram Co-occurrence Statistics

  2003cited by this paper
• SUMMAC: a text summarization evaluation

  2002influential reference
• Summarization Evaluation: An Overview

  2001influential reference
• Generic summaries for indexing in information retrieval – Detailed test results

  2001influential reference
• Evaluation of Phrase-Representation Summarization based on Information Retrieval Task

  2000cited by this paper
• Generating Natural Language Summaries from Multiple On-Line Sources

  1998cited by this paper
• Advantages of query biased summaries in information retrieval

  1998cited by this paper
• Summarization Evaluation Methods: Experiments and Analysis

  1998cited by this paper
• Summarizing Similarities and Differences Among Related Documents

  1997cited by this paper
• Book Reviews: Evaluating Natural Language Processing Systems: An Analysis and Review

  1996cited by this paper
• Assessing Agreement on Classification Tasks: The Kappa Statistic

  1996influential reference
• Headline Evaluation Experiment Results

  year unknowncited by this paper

• Challenges in Creating Text Summarization Models in Malayalam: A Study

  2023cites this paper
• Is Summary Useful or Not? An Extrinsic Human Evaluation of Text Summaries on Downstream Tasks

  2023influential citation
• ConceptEVA: Concept-Based Interactive Exploration and Customization of Document Summaries

  2023cites this paper
• Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text

  2022cites this paper
• Do we still need gold standard for evaluation ?

  2022cites this paper
• What Makes a Good Summary? Investigating the Focus of Automatic Summarization in an Educational Context

  2021cites this paper
• Abstractive Text Summarization based on Language Model Conditioning and Locality Modeling

  2020cites this paper
• What Makes a Good and Useful Summary? Incorporating Users in Automatic Summarization Research

  2020cites this paper
• What Makes a Good Summary? Reconsidering the Focus of Automatic Summarization

  2020cites this paper
• Abstractive text summarization with neural sequence-to-sequence models

  2020cites this paper
• Automatic text summarization: What has been done and what has to be done

  2019cites this paper
• Summarization

  2018cites this paper
• UMMARIZATION OF S OFTWARE A RTIFACTS : A R EVIEW

  2017cites this paper
• Summarization of Software Artifacts : A Review

  2017cites this paper
• Abstractive Compression of Captions with Attentive Recurrent Neural Networks

  2016cites this paper
• Genetic Clustering Algorithm for Extractive Text Summarization

  2015cites this paper
• Finding Good Enough: A Task-Based Evaluation of Query Biased Summarization for Cross-Language Information Retrieval

  2014cites this paper
• Distributional Semantics for Robust Automatic Summarization

  2014cites this paper
• Generating explanations from support vector machines for psychological classifications

  2014cites this paper
• Syntactic Sentence Compression for Text Summarization

  2013cites this paper
• ExtraWeb - An Extrinsic Task-oriented Evaluation of Webpage Extracts

  2013cites this paper
• Semantic based content search and content summarization

  2012cites this paper
• Automatic generation of natural language nursing shift summaries in neonatal intensive care: BT-Nurse

  2012cites this paper
• utomatic generation of natural language nursing shift summaries in neonatal ntensive care : BT-Nurse

  2012cites this paper
• Methods for Mining and Summarizing Text Conversations

  2011cites this paper
• The Elements of Automatic Summarization

  2011cites this paper
• Data-driven Natural Language Generation: Making Machines Talk Like Humans Using Natural Corpora

  2010cites this paper
• Introducing Shared Tasks to NLG: The TUNA Shared Task Evaluation Challenges

  2010influential citation
• Introducing shared task evaluation to NLG : The TUNA shared task evaluation challenges

  2010cites this paper
• Textual Properties and Task-based Evaluation: Investigating the Role of Surface Properties, Structure and Content

  2010influential citation
• Text Content and Task Performance in the Evaluation of a Natural Language Generation System

  2009cites this paper
• An Investigation into the Validity of Some Metrics for Automatically Evaluating Natural Language Generation Systems

  2009cites this paper
• Do we Still Need Gold Standards for Evaluation?

  2008influential citation
• Using Speech-Specific Characteristics for Automatic Speech Summarization

  2008cites this paper
• Single-document and multi-document summarization techniques for email threads using sentence compression

  2008cites this paper
• A Cascaded Broadcast News Highlighter

  2008cites this paper
• Extrinsic summarization evaluation: A decision audit task

  2008influential citation
• Automatic summarising: The state of the art

  2007cites this paper
• Syntactic sentence compression in the biomedical domain: facilitating access to related articles

  2007cites this paper
• User-model based personalized summarization

  2007cites this paper
• TEXT SUMMARIZATION EVALUATION: CORRELATING HUMAN PERFORMANCE ON AN EXTRINSIC TASK WITH AUTOMATIC INTRINSIC METRICS

  2006cites this paper
• TLR at DUC : Tree similarity

  2005cites this paper
• Bayesian Summarization at DUC and a Suggestion for Extrinsic Evaluation

  year unknowncites this paper