Human Evaluation of Article and Noun Number Usage: Influences of Context and Construction Variability

Evaluating systems that correct errors in non-native writing is difficult because of the possibility of multiple correct answers and the variability in human agreement. This paper seeks to improve the best practice of such evaluation by analyzing the frequency of multiple correct answers and identifying factors that influence agreement levels in judging the usage of articles and noun number.

• Publication year

  2009

• Venue

  Law

• Publication date

  2009-08-06

• Fields of study

  Linguistics, Computer Science

• Identifiers
  DOI 10.3115/1698381.1698391
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Native Judgments of Non-Native Usage: Experiments in Preposition Error Detection

  2008cited by this paper
• Language Modeling for Determiner Selection

  2007cited by this paper
• Correcting ESL Errors Using Phrasal SMT Techniques

  2006cited by this paper
• Detecting errors in English article usage by non-native speakers

  2006influential reference
• Automatic grammar correction for second-language learners

  2006cited by this paper
• Correction of article errors in machine translation using Web-based model

  2005cited by this paper
• Automatic Article Restoration

  2004cited by this paper
• Memory-Based Learning for Article Generation

  2000cited by this paper
• A Maximum Entropy Model for Prepositional Phrase Attachment

  1994cited by this paper
• Automated Postediting of Documents

  1994cited by this paper
• The measurement of observer agreement for categorical data.

  1977cited by this paper

• Abstraction not Memory: BERT and the English Article System

  2022cites this paper
• Normalization in Context: Inter-Annotator Agreement for Meaning-Based Target Hypothesis Annotation

  2018cites this paper
• Capturing Pragmatic Knowledge in Article Usage Prediction using LSTMs

  2016cites this paper
• A CALL System for Learning Preposition Usage

  2016cites this paper
• Improving automatic correction of article errors

  2016cites this paper
• CityU corpus of essay drafts of English language learners: a corpus of textual revision in second language writing

  2015cites this paper
• Error detection in content word combinations

  2015cites this paper
• Book Reviews: Automated Grammatical Error Detection for Language Learners, Second Edition by Claudia Leacock, Martin Chodorow, Michael Gamon and Joel Tetreault

  2014cites this paper
• Some complexities in English article use and acquisition

  2014cites this paper
• Annotating Article Errors in Spanish Learner Texts: Design and Evaluation of an Annotation Scheme

  2014cites this paper
• Building a Large Annotated Corpus of Learner English: The NUS Corpus of Learner English

  2013cites this paper
• Correcting Serial Grammatical Errors based on N-grams and Syntax

  2013cites this paper
• Book Reviews: Automated Grammatical Error Detection for Language Learners by Claudia Leacock, Martin Chodorow, Michael Gamon, and Joel Tetreault

  2010cites this paper
• Automatic correction of grammatical errors in non-native English text

  2009cites this paper
• Mental salience and grammatical form: toward a framework for salience metrics in natural language generation

  2009cites this paper