Improved Transition-based Parsing by Modeling Characters instead of Words with LSTMs

We present extensions to a continuousstate dependency parsing method that makes it applicable to morphologically rich languages. Starting with a highperformance transition-based parser that uses long short-term memory (LSTM) recurrent neural networks to learn representations of the parser state, we replace lookup-based word representations with representations constructed from the orthographic representations of the words, also using LSTMs. This allows statistical sharing across word forms that are similar on the surface. Experiments for morphologically rich languages show that the parsing model benefits from incorporating the character-based encodings of words.

• Publication year

  2015

• Venue

  Conference on Empirical Methods in Natural Language Processing

• Publication date

  2015-08-04

• Fields of study

  Computer Science

• Identifiers
  DOI 10.18653/v1/D15-1041arXiv 1508.00657
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Joint Learning of Character and Word Embeddings

  2015cited by this paper
• A Neural Probabilistic Structured-Prediction Model for Transition-Based Dependency Parsing

  2015cited by this paper
• Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation

  2015cited by this paper
• Structured Training for Neural Network Transition-Based Parsing

  2015cited by this paper
• Boosting Named Entity Recognition with Neural Character Embeddings

  2015cited by this paper
• LSTM: A Search Space Odyssey

  2015cited by this paper
• Transition-Based Dependency Parsing with Stack Long Short-Term Memory

  2015influential reference
• Compositional Morphology for Word Representations and Language Modelling

  2014cited by this paper
• Introducing the IMS-Wrocław-Szeged-CIS entry at the SPMRL 2014 Shared Task: Reranking and Morpho-syntax meet Unlabeled Data

  2014cited by this paper
• A Fast and Accurate Dependency Parser using Neural Networks

  2014cited by this paper
• Introducing the SPMRL 2014 Shared Task on Parsing Morphologically-rich Languages

  2014cited by this paper
• Normalizing tweets with edit scripts and recurrent neural embeddings

  2014cited by this paper
• Grounded Compositional Semantics for Finding and Describing Images with Sentences

  2014cited by this paper
• Learning Character-level Representations for Part-of-Speech Tagging

  2014cited by this paper
• Joint Morphological and Syntactic Analysis for Richly Inflected Languages

  2013cited by this paper
• Preparing Korean Data for the Shared Task on Parsing Morphologically Rich Languages

  2013influential reference
• Transition-based Dependency Parsing Using Recursive Neural Networks

  2013cited by this paper
• Training Deterministic Parsers with Non-Deterministic Oracles

  2013cited by this paper
• Chinese Parsing Exploiting Characters

  2013cited by this paper
• The Role of Syntax in Vector Space Models of Compositional Semantics

  2013cited by this paper
• Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

  2013cited by this paper
• Effective Morphological Feature Selection with MaltOptimizer at the SPMRL 2013 Shared Task

  2013cited by this paper
• Overview of the SPMRL 2013 Shared Task: A Cross-Framework Evaluation of Parsing Morphologically Rich Languages

  2013cited by this paper
• Generating Sequences With Recurrent Neural Networks

  2013cited by this paper
• Efficient Higher-Order CRFs for Morphological Tagging

  2013cited by this paper
• (Re)ranking Meets Morphosyntax: State-of-the-art Results from the SPMRL 2013 Shared Task

  2013cited by this paper
• Making Ellipses Explicit in Dependency Conversion for a German Treebank

  2012cited by this paper
• Joint Hebrew Segmentation and Parsing using a PCFGLA Lattice Parser

  2011cited by this paper
• Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection

  2011cited by this paper
• Hungarian Dependency Treebank

  2010influential reference
• Dual Decomposition for Parsing with Non-Projective Head Automata

  2010cited by this paper
• Towards a Bank of Constituent Parse Trees for Polish

  2010cited by this paper
• Non-Projective Dependency Parsing in Expected Linear Time

  2009influential reference
• Joint Word Segmentation and POS Tagging Using a Single Perceptron

  2008cited by this paper
• A Tale of Two Parsers: Investigating and Combining Graph-based and Transition-based Dependency Parsing

  2008cited by this paper
• A Single Generative Model for Joint Morphological Segmentation and Syntactic Parsing

  2008cited by this paper
• A Latent Variable Model for Generative Dependency Parsing

  2007cited by this paper
• Joint Morphological and Syntactic Disambiguation

  2007cited by this paper
• Labeled Pseudo-Projective Dependency Parsing with Support Vector Machines

  2006cited by this paper
• Generating Typed Dependency Parses from Phrase Structure Parses

  2006cited by this paper
• Integrated Morphological and Syntactic Disambiguation for Modern Hebrew

  2006cited by this paper
• CoNLL-X Shared Task on Multilingual Dependency Parsing

  2006cited by this paper
• Talbanken05: A Swedish Treebank with Phrase Structure and Dependency Annotation

  2006influential reference
• 2005 Special Issue: Framewise phoneme classification with bidirectional LSTM and other neural network architectures

  2005influential reference
• The Penn Arabic Treebank: Building a Large-Scale Annotated Arabic Corpus

  2004cited by this paper
• Incrementality in Deterministic Dependency Parsing

  2004cited by this paper
• Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network

  2003cited by this paper
• Construction of a Basque Dependency Treebank

  2003influential reference
• Learning Precise Timing with LSTM Recurrent Networks

  2003cited by this paper
• Building a Turkish Treebank

  2003cited by this paper
• Building a tree-bank of modern hebrew text

  2001influential reference
• Building a Treebank for French

  2000influential reference
• Long Short-Term Memory

  1997cited by this paper
• Building a Large Annotated Corpus of English: The Penn Treebank

  1993cited by this paper

• Fatigue limit prediction of 7050 aluminium alloy based on experimental and shallow + deep hybrid neural network

  2024cites this paper
• Toward accurate Amazigh part-of-speech tagging

  2024cites this paper
• MrT5: Dynamic Token Merging for Efficient Byte-level Language Models

  2024cites this paper
• Systematic Literature Review on Named Entity Recognition: Approach, Method, and Application

  2024cites this paper
• Using Ensemble Learning in Language Variety Identification

  2023cites this paper
• Another Dead End for Morphological Tags? Perturbed Inputs and Parsing

  2023cites this paper
• Deep Learning-based Sequence Labeling Tools for Nepali

  2023cites this paper
• Data-driven dependency parsing of Vedic Sanskrit

  2023cites this paper
• Deep Learning for Natural Language Processing: A Survey

  2023cites this paper
• Syntactic Inductive Biases for Natural Language Processing

  2022cites this paper
• Part-of-Speech and Morphological Tagging of Algerian Judeo-Arabic

  2022cites this paper
• Parsing linearizations appreciate PoS tags - but some are fussy about errors

  2022cites this paper
• Annotating the Tweebank Corpus on Named Entity Recognition and Building NLP Models for Social Media Analysis

  2022cites this paper
• Nonlinear Dynamic Soft Sensor Development with a Supervised Hybrid CNN-LSTM Network for Industrial Processes

  2022cites this paper
• Early prediction of learners at risk in self-paced education: A neural network approach

  2022cites this paper
• Deep Learning-Based Named Entity Recognition System Using Hybrid Embedding

  2022cites this paper
• A Review on Recent Advances in Recurrent Neural Networks

  2021cites this paper
• From Constituency to UD-Style Dependency: Building the First Conversion Tool of Turkish

  2021cites this paper
• Position-Aware Deep Character-Level CTR Prediction for Sponsored Search

  2021cites this paper
• Input Representations for Parsing Discourse Representation Structures: Comparing English with Chinese

  2021cites this paper
• Annotations Matter: Leveraging Multi-task Learning to Parse UD and SUD

  2021cites this paper
• Recent Trends in Named Entity Recognition (NER)

  2021cites this paper
• A Falta de Pan, Buenas Son Tortas: The Efficacy of Predicted UPOS Tags for Low Resource UD Parsing

  2021cites this paper
• What Taggers Fail to Learn, Parsers Need the Most

  2021cites this paper
• Forensic Memory Classification using Deep Recurrent Neural Networks

  2021cites this paper
• UnibucKernel: Geolocating Swiss German Jodels Using Ensemble Learning

  2021cites this paper
• Neural OCR Post-Hoc Correction of Historical Corpora

  2021cites this paper
• Embeddings in Natural Language Processing: Theory and Advances in Vector Representations of Meaning

  2020cites this paper
• Transition-Based Dependency Parsing using Perceptron Learner

  2020cites this paper
• On understanding character-level models for representing morphology

  2020cites this paper
• A Survey of Deep Learning Techniques for Neural Machine Translation

  2020cites this paper
• Testbeds and Research Infrastructures for the Development of Networks and Communications: 14th EAI International Conference, TridentCom 2019, Changsha, China, December 7-8, 2019, Proceedings

  2020cites this paper
• Weakly Supervised POS Taggers Perform Poorly on Truly Low-Resource Languages

  2020cites this paper
• An Efficient Architecture for Predicting the Case of Characters using Sequence Models

  2020cites this paper
• The use of a neural network model for the analysis of tourism development in the regions of the country

  2020cites this paper
• Machine Learning Classifiers for Twitter Surveillance of Vaping: Comparative Machine Learning Study

  2020cites this paper
• The application of tools of neural networks and artificial intelligence in the recreational sphere

  2020cites this paper
• Simple methods to overcome the limitations of general word representations in natural language processing tasks

  2020cites this paper
• On the Frailty of Universal POS Tags for Neural UD Parsers

  2020cites this paper
• Sanskrit to universal networking language EnConverter system based on deep learning and context-free grammar

  2020cites this paper
• Part-of-Speech Tagging Using Multiview Learning

  2020cites this paper
• A survey of syntactic-semantic parsing based on constituent and dependency structures

  2020cites this paper
• Combining Deep Learning and String Kernels for the Localization of Swiss German Tweets

  2020cites this paper
• The unreasonable effectiveness of machine learning in Moldavian versus Romanian dialect identification

  2020cites this paper
• Effective Connectivity Study Guiding the Neuromodulation Intervention in Figurative Language Comprehension Using Optical Neuroimaging

  2020cites this paper
• Character-level Representations Still Improve Semantic Parsing in the Age of BERT

  2020cites this paper
• Name-Nationality Classification Technology under Keras Deep Learning

  2020cites this paper
• Self Attended Stack-Pointer Networks for Learning Long Term Dependencies

  2020cites this paper
• Enhancing deep neural networks with morphological information

  2020cites this paper
• Code-mixed parse trees and how to find them

  2020cites this paper
• The Functional Expansion of Automatic Speech Recognition Output Based on Novel Prosody- and Text-Based Approaches

  2020cites this paper
• Treebank Embedding Vectors for Out-of-Domain Dependency Parsing

  2020cites this paper
• SWE2: SubWord Enriched and Significant Word Emphasized Framework for Hate Speech Detection

  2020cites this paper
• A New Method for Sentence Vector Normalization Using Word2vec

  2019cites this paper
• Morphological Knowledge Guided Mongolian Constituent Parsing

  2019cites this paper
• Authorship Attribution in Bangla literature using Character-level CNN

  2019cites this paper
• Character-based NMT with Transformer

  2019cites this paper
• Dependency Parsing as Sequence Labeling with Head-Based Encoding and Multi-Task Learning

  2019cites this paper
• Exploring Discriminative Word-Level Domain Contexts for Multi-Domain Neural Machine Translation

  2019cites this paper
• Text Representation for Machine Learning Applications

  2019cites this paper
• How Important Is POS to Dependency Parsing? Joint POS Tagging and Dependency Parsing Neural Networks

  2019cites this paper
• Multilingual POS tagging by a composite deep architecture based on character-level features and on-the-fly enriched Word Embeddings

  2019cites this paper
• Improving the Annotations in the Turkish Universal Dependency Treebank

  2019cites this paper
• JBNU at MRP 2019: Multi-level Biaffine Attention for Semantic Dependency Parsing

  2019cites this paper
• NeuMorph

  2019cites this paper
• A single-layer RNN can approximate stacked and bidirectional RNNs, and topologies in between

  2019cites this paper
• State-of-the-art Italian dependency parsers based on neural and ensemble systems

  2019cites this paper
• Hierarchical Pointer Net Parsing

  2019cites this paper
• Multi-task Learning for Low-resource Second Language Acquisition Modeling

  2019cites this paper
• Comprehensive Analysis of Aspect Term Extraction Methods using Various Text Embeddings

  2019cites this paper
• Transductive Auxiliary Task Self-Training for Neural Multi-Task Models

  2019cites this paper
• Deep Contextualized Word Embeddings in Transition-Based and Graph-Based Dependency Parsing - A Tale of Two Parsers Revisited

  2019cites this paper
• Deep Contextualized Word Embeddings for Universal Dependency Parsing

  2019cites this paper
• Aspect Detection using Word and Char Embeddings with (Bi) LSTM and CRF

  2019cites this paper
• Neural sequence-to-sequence models for low-resource morphology

  2019cites this paper
• Chinese Syntax Parsing Based on Sliding Match of Semantic String

  2019cites this paper
• Sequence Labeling Parsing by Learning across Representations

  2019cites this paper
• Neural Network-based Chinese Joint Syntactic Analysis

  2019cites this paper
• SurfCon: Synonym Discovery on Privacy-Aware Clinical Data

  2019cites this paper
• Virtual learning environment to predict withdrawal by leveraging deep learning

  2019cites this paper
• Transferring Informal Text in Arabic as Low Resource Languages: State-of-the-Art and Future Research Directions

  2019cites this paper
• Rewarding Smatch: Transition-Based AMR Parsing with Reinforcement Learning

  2019cites this paper
• Multi-Task Semantic Dependency Parsing with Policy Gradient for Learning Easy-First Strategies

  2019cites this paper
• Normalization and parsing algorithms for uncertain input

  2019cites this paper
• An Attention-based Model for Joint Extraction of Entities and Relations with Implicit Entity Features

  2019cites this paper
• From Genesis to Creole Language

  2019cites this paper
• 75 Languages, 1 Model: Parsing Universal Dependencies Universally

  2019cites this paper
• How to utilize syllable distribution patterns as the input of LSTM for Korean morphological analysis

  2019cites this paper
• Boosting Arabic Named-Entity Recognition With Multi-Attention Layer

  2019cites this paper
• Multi-Domain Gated CNN for Review Helpfulness Prediction

  2019cites this paper
• Predicting At-Risk Students Using Clickstream Data in the Virtual Learning Environment

  2019cites this paper
• Recursive Subtree Composition in LSTM-Based Dependency Parsing

  2019influential citation
• Character Decomposition for Japanese-Chinese Character-Level Neural Machine Translation

  2019cites this paper
• 利用Attentive來改善端對端中文語篇剖析遞迴類神經網路系統(Using Attentive to improve Recursive LSTM End-to-End Chinese Discourse Parsing)

  2019cites this paper
• Unsupervised learning of cross-modal mappings between speech and text

  2019cites this paper
• A Multilingual Encoding Method for Text Classification and Dialect Identification Using Convolutional Neural Network

  2019cites this paper
• Deep Learning for Natural Language Parsing

  2019cites this paper
• Power Micro-Blog Text Classification Based on Domain Dictionary and LSTM-RNN

  2019cites this paper
• Approximating Stacked and Bidirectional Recurrent Architectures with the Delayed Recurrent Neural Network

  2019cites this paper
• Machine Learning Classifiers for Twitter Surveillance of Vaping: Comparative Machine Learning Study (Preprint)

  2019cites this paper