Modelling Sentence Pairs with Tree-structured Attentive Encoder

We describe an attentive encoder that combines tree-structured recursive neural networks and sequential recurrent neural networks for modelling sentence pairs. Since existing attentive models exert attention on the sequential structure, we propose a way to incorporate attention into the tree topology. Specially, given a pair of sentences, our attentive encoder uses the representation of one sentence, which generated via an RNN, to guide the structural encoding of the other sentence on the dependency parse tree. We evaluate the proposed attentive encoder on three tasks: semantic similarity, paraphrase identification and true-false question selection. Experimental results show that our encoder outperforms all baselines and achieves state-of-the-art results on two tasks.

• Publication year

  2016

• Venue

  International Conference on Computational Linguistics

• Publication date

  2016-10-10

• Fields of study

  Computer Science

• Identifiers
  arXiv 1610.02806
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Joint Learning of Sentence Embeddings for Relevance and Entailment

  2016influential reference
• An Attentive Neural Architecture for Fine-grained Entity Type Classification

  2016cited by this paper
• Neural Responding Machine for Short-Text Conversation

  2015cited by this paper
• Multi-Perspective Sentence Similarity Modeling with Convolutional Neural Networks

  2015influential reference
• Skip-Thought Vectors

  2015cited by this paper
• Effective Approaches to Attention-based Neural Machine Translation

  2015cited by this paper
• Reasoning about Entailment with Neural Attention

  2015cited by this paper
• End-To-End Memory Networks

  2015cited by this paper
• Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks

  2015influential reference
• A Neural Attention Model for Abstractive Sentence Summarization

  2015cited by this paper
• Teaching Machines to Read and Comprehend

  2015cited by this paper
• ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs

  2015cited by this paper
• A large annotated corpus for learning natural language inference

  2015cited by this paper
• ECNU: One Stone Two Birds: Ensemble of Heterogenous Measures for Semantic Relatedness and Textual Entailment

  2014influential reference
• GloVe: Global Vectors for Word Representation

  2014influential reference
• Grounded Compositional Semantics for Finding and Describing Images with Sentences

  2014cited by this paper
• SemEval-2014 Task 1: Evaluation of Compositional Distributional Semantic Models on Full Sentences through Semantic Relatedness and Textual Entailment

  2014cited by this paper
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling

  2014cited by this paper
• A Convolutional Neural Network for Modelling Sentences

  2014cited by this paper
• Convolutional Neural Networks for Sentence Classification

  2014cited by this paper
• Neural Machine Translation by Jointly Learning to Align and Translate

  2014cited by this paper
• Discriminative Improvements to Distributional Sentence Similarity

  2013cited by this paper
• Statistical Language Models Based on Neural Networks

  2012cited by this paper
• Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection

  2011cited by this paper
• Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

  2011influential reference
• Corpus-based and Knowledge-based Measures of Text Semantic Similarity

  2006cited by this paper
• Unsupervised Construction of Large Paraphrase Corpora: Exploiting Massively Parallel News Sources

  2004influential reference
• The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions

  1998cited by this paper
• Long Short-Term Memory

  1997cited by this paper

• Sum-Product-Set Networks: Deep Tractable Models for Tree-Structured Graphs

  2024cites this paper
• Monotonicity Reasoning in the Age of Neural Foundation Models

  2023cites this paper
• Computing semantic similarity of texts by utilizing dependency graph

  2022cites this paper
• How Open Source Machine Learning Software Shapes AI

  2022cites this paper
• Preprocessing framework for scholarly big data management

  2022cites this paper
• Computing semantic similarity of texts based on deep graph learning with ability to use semantic role label information

  2022cites this paper
• Predicting Chinese Phrase-Level Sentiment Intensity in Valence-Arousal Dimensions With Linguistic Dependency Features

  2022cites this paper
• A Graph Convolutional Network With Multiple Dependency Representations for Relation Extraction

  2021influential citation
• Target Adaptive Context Aggregation for Video Scene Graph Generation

  2021cites this paper
• A Theoretical Foundation for Syntactico-Semantic Pattern Recognition

  2021cites this paper
• Attentive Tree-structured Network for Monotonicity Reasoning

  2021influential citation
• Contrasting distinct structured views to learn sentence embeddings

  2021cites this paper
• Modelling Sentence Pairs via Reinforcement Learning: An Actor-Critic Approach to Learn the Irrelevant Words

  2020cites this paper
• Make It Directly: Event Extraction Based on Tree-LSTM and Bi-GRU

  2020cites this paper
• Optimizing loss functions through multi-variate taylor polynomial parameterization

  2020cites this paper
• Evolving Loss Functions with Multivariate Taylor Polynomial Parameterizations

  2020cites this paper
• Hybrid Attention Based Neural Architecture for Text Semantics Similarity Measurement

  2020cites this paper
• Improving deep learning through loss-function evolution

  2020cites this paper
• Research on classification and similarity of patent citation based on deep learning

  2020cites this paper
• Surface Realization Using Pretrained Language Models

  2020cites this paper
• Improving Tree-LSTM with Tree Attention

  2019influential citation
• A Spark ML driven preprocessing approach for deep learning based scholarly data applications

  2019cites this paper
• Head-First Linearization with Tree-Structured Representation

  2019cites this paper
• IMSurReal: IMS at the Surface Realization Shared Task 2019

  2019influential citation
• From lexical towards contextualized meaning representation

  2019cites this paper
• 利用Attentive來改善端對端中文語篇剖析遞迴類神經網路系統(Using Attentive to improve Recursive LSTM End-to-End Chinese Discourse Parsing)

  2019cites this paper
• An Efficient Framework for Sentence Similarity Modeling

  2019cites this paper
• REGMAPR-A Recipe for Textual Matching Anonymous EMNLP submission

  2018influential citation
• An Experimental Analysis of Multi-Perspective Convolutional Neural Networks

  2018cites this paper
• Learning Semantic Textual Relatedness using Natural Deduction Proofs

  2018cites this paper
• REGMAPR - A Recipe for Textual Matching

  2018cites this paper
• REGMAPR - Text Matching Made Easy

  2018cites this paper
• Towards Bidirectional Hierarchical Representations for Attention-based Neural Machine Translation

  2017cites this paper
• An Empirical Study on End-to-End Sentence Modelling

  2017cites this paper
• Inter-Weighted Alignment Network for Sentence Pair Modeling

  2017influential citation
• Video Classification via Relational Feature Encoding Networks

  2017cites this paper
• Structured Attention Networks

  2017influential citation
• A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks

  2016cites this paper
• A Joint Many-Task Model: How to Grow a Neural Net for Multiple NLP tasks

  2016influential citation
• MIT Open Access Articles How Open Source Machine Learning Software Shapes AI

  year unknowncites this paper