A Dynamic Window Neural Network for CCG Supertagging

Combinatory Category Grammar (CCG) supertagging is a task to assign lexical categories to each word in a sentence. Almost all previous methods use fixed context window sizes to encode input tokens. However, it is obvious that different tags usually rely on different context window sizes. This motivates us to build a supertagger with a dynamic window approach, which can be treated as an attention mechanism on the local contexts. We find that applying dropout on the dynamic filters is superior to the regular dropout on word embeddings. We use this approach to demonstrate the state-of-the-art CCG supertagging performance on the standard test set.

• Publication year

  2016

• Venue

  AAAI Conference on Artificial Intelligence

• Publication date

  2016-10-10

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1609/aaai.v31i1.10992arXiv 1610.02749
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Expected F-Measure Training for Shift-Reduce Parsing with Recurrent Neural Networks

  2016influential reference
• Supertagging With LSTMs

  2016cited by this paper
• LSTM CCG Parsing

  2016cited by this paper
• Not All Contexts Are Created Equal: Better Word Representations with Variable Attention

  2015cited by this paper
• Deep Unordered Composition Rivals Syntactic Methods for Text Classification

  2015cited by this paper
• CCG Supertagging with a Recurrent Neural Network

  2015influential reference
• Semi-supervised Sequence Learning

  2015cited by this paper
• Broad-coverage CCG Semantic Parsing with AMR

  2015cited by this paper
• A Theoretically Grounded Application of Dropout in Recurrent Neural Networks

  2015cited by this paper
• Improved CCG Parsing with Semi-supervised Supertagging

  2014cited by this paper
• GloVe: Global Vectors for Word Representation

  2014cited by this paper
• Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

  2014cited by this paper
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• Dropout: a simple way to prevent neural networks from overfitting

  2014cited by this paper
• Neural Machine Translation by Jointly Learning to Align and Translate

  2014cited by this paper
• Combined Distributional and Logical Semantics

  2013cited by this paper
• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks

  2013cited by this paper
• ADADELTA: An Adaptive Learning Rate Method

  2012cited by this paper
• Natural Language Processing (Almost) from Scratch

  2011cited by this paper
• Non-Transformational Syntax: Formal and Explicit Models of Grammar

  2011cited by this paper
• Lexical Generalization in CCG Grammar Induction for Semantic Parsing

  2011cited by this paper
• Word Representations: A Simple and General Method for Semi-Supervised Learning

  2010cited by this paper
• Understanding the difficulty of training deep feedforward neural networks

  2010cited by this paper
• Inducing Probabilistic CCG Grammars from Logical Form with Higher-Order Unification

  2010cited by this paper
• Wide-Coverage Semantic Analysis with Boxer

  2008cited by this paper
• Wide-Coverage Efficient Statistical Parsing with CCG and Log-Linear Models

  2007cited by this paper
• CCGbank: A Corpus of CCG Derivations and Dependency Structures Extracted from the Penn Treebank

  2007influential reference
• Online Learning of Relaxed CCG Grammars for Parsing to Logical Form

  2007cited by this paper
• Towards Wide-Coverage Semantic Interpretation

  2005cited by this paper
• Combinatory Categorial Grammar

  2004cited by this paper
• Wide-Coverage Semantic Representations from a CCG Parser

  2004cited by this paper
• LSTM can Solve Hard Long Time Lag Problems

  1996cited by this paper
• Building a Large Annotated Corpus of English: The Penn Treebank

  1993cited by this paper
• Attractor dynamics and parallelism in a connectionist sequential machine

  1990cited by this paper
• Finding Structure in Time

  1990cited by this paper
• Experiments on Learning by Back Propagation.

  1986cited by this paper
• Syntactic Processing

  1979cited by this paper

• Span-based sliding window feature extraction and label knowledge for few-shot Named Entity Recognition

  2025cites this paper
• Learning Syllables Using Conv-LSTM Model for Swahili Word Representation and Part-of-speech Tagging

  2021cites this paper
• Generating CCG Categories

  2021cites this paper
• Supertagging the Long Tail with Tree-Structured Decoding of Complex Categories

  2020cites this paper
• Supertagging with CCG primitives

  2020cites this paper
• Wide-coverage statistical parsing with minimalist grammars

  2019cites this paper
• Deep Learning for Natural Language Processing

  2019cites this paper
• Deep Learning: Fundamentals, Theory and Applications

  2019cites this paper
• Wide-Coverage Neural A* Parsing for Minimalist Grammars

  2019cites this paper
• Sequence Labeling and Transduction with Output-Adjusted Actor-Critic Training of RNNs

  2018cites this paper
• Efficient Sequence Labeling with Actor-Critic Training

  2018cites this paper
• Constraining MGbank: Agreement, L-Selection and Supertagging in Minimalist Grammars

  2018influential citation
• Shortcut Sequence Tagging

  2017cites this paper
• An Empirical Exploration of Skip Connections for Sequential Tagging

  2016cites this paper