Data-Defined Kernels for Parse Reranking Derived from Probabilistic Models

Previous research applying kernel methods to natural language parsing have focussed on proposing kernels over parse trees, which are hand-crafted based on domain knowledge and computational considerations. In this paper we propose a method for defining kernels in terms of a probabilistic model of parsing. This model is then trained, so that the parameters of the probabilistic model reflect the generalizations in the training data. The method we propose then uses these trained parameters to define a kernel for reranking parse trees. In experiments, we use a neural network based statistical parser as the probabilistic model, and use the resulting kernel with the Voted Perceptron algorithm to rerank the top 20 parses from the probabilistic model. This method achieves a significant improvement over the accuracy of the probabilistic model.

• Publication year

  2005

• Venue

  Annual Meeting of the Association for Computational Linguistics

• Publication date

  2005-06-25

• Fields of study

  Computer Science

• Identifiers
  DOI 10.3115/1219840.1219863
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Max-Margin Parsing

  2004cited by this paper
• Support vector machine learning for interdependent and structured output spaces

  2004cited by this paper
• Discriminative Training of a Neural Network Statistical Parser

  2004cited by this paper
• Flexible Margin Selection for Reranking with Full Pairwise Samples

  2004influential reference
• Incremental Parsing with the Perceptron Algorithm

  2004influential reference
• An efficient implementation of a new DOP model

  2003cited by this paper
• Using LTAG Based Features in Parse Reranking

  2003influential reference
• An SVM-based voting algorithm with application to parse reranking

  2003cited by this paper
• Inducing History Representations for Broad Coverage Statistical Parsing

  2003influential reference
• Head-Driven Statistical Models for Natural Language Parsing

  2003influential reference
• New Ranking Algorithms for Parsing and Tagging: Kernels over Discrete Structures, and the Voted Perceptron

  2002influential reference
• A New Discriminative Kernel from Probabilistic Models

  2001influential reference
• More accurate tests for the statistical significance of result differences

  2000cited by this paper
• Discriminative Reranking for Natural Language Parsing

  2000cited by this paper
• A Maximum-Entropy-Inspired Parser

  2000cited by this paper
• Learning to Parse Natural Language with Maximum Entropy Models

  1999cited by this paper
• Large Margin Classification Using the Perceptron Algorithm

  1998influential reference
• Exploiting Generative Models in Discriminative Classifiers

  1998cited by this paper
• A Maximum Entropy Model for Part-Of-Speech Tagging

  1996cited by this paper
• Building a Large Annotated Corpus of English: The Penn Treebank

  1993cited by this paper

• Adding smarter systems instead of human annotators: re-ranking for system combination

  2011cites this paper
• Learning with Annotation Noise

  2009cites this paper
• Searching for Part of Speech Tags That Improve Parsing Models

  2008cites this paper
• Exploiting non-linear probabilistic models in natural language parsing and reranking

  2008cites this paper
• Introduction to Classification: Likelihoods, Margins, Features, and Kernels

  2007cites this paper
• Constituent Parsing with Incremental Sigmoid Belief Networks

  2007cites this paper
• A Latent Variable Model for Generative Dependency Parsing

  2007influential citation
• Fast and Robust Multilingual Dependency Parsing with a Generative Latent Variable Model

  2007cites this paper
• Improving information extraction and translation using component interactions

  2007cites this paper
• HLT-NAACL-2006 Computationally Hard Problems and Joint Inference in Speech and Language Processing

  2006cites this paper
• Japanese Dependency Parsing Using Co-Occurrence Information and a Combination of Case Elements

  2006cites this paper
• Re-Ranking Algorithms for Name Tagging

  2006cites this paper
• Porting Statistical Parsers with Data-Defined Kernels

  2006influential citation
• Loss Minimization in Parse Reranking

  2006influential citation
• Bayes Risk Minimization in Natural Language Parsing

  2006influential citation
• Kernels from Probabilistic Models for Multiclass Classification and Reranking with Arbitrary Loss Function

  2005cites this paper
• Structural Bias in Inducing Representations for Probabilistic Natural Language Parsing

  2003cites this paper