Leveraging Contextual Sentences for Text Classification by Using a Neural Attention Model

We explored several approaches to incorporate context information in the deep learning framework for text classification, including designing different attention mechanisms based on different neural network and extracting some additional features from text by traditional methods as the part of representation. We propose two kinds of classification algorithms: one is based on convolutional neural network fusing context information and the other is based on bidirectional long and short time memory network. We integrate the context information into the final feature representation by designing attention structures at sentence level and word level, which increases the diversity of feature information. Our experimental results on two datasets validate the advantages of the two models in terms of time efficiency and accuracy compared to the different models with fundamental AM architectures.

• Publication year

  2019

• Venue

  Computational Intelligence and Neuroscience

• Publication date

  2019-08-01

• Fields of study

  Medicine, Computer Science

• Identifiers
  DOI 10.1155/2019/8320316PMID 31467518PMCID 6701294
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Opinion Mining and Sentiment Analysis

  2018cited by this paper
• A feature cluster taxonomy based feature selection technique

  2017cited by this paper
• Feature selection based on quality of information

  2017cited by this paper
• Neural-based Context Representation Learning for Dialog Act Classification

  2017cited by this paper
• Hierarchical Attention Networks for Document Classification

  2016cited by this paper
• Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks

  2016cited by this paper
• Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling

  2016cited by this paper
• A feature selection approach based on term distributions

  2016cited by this paper
• Bidirectional Attention Flow for Machine Comprehension

  2016cited by this paper
• Turning from TF-IDF to TF-IGM for term weighting in text classification

  2016cited by this paper
• Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification

  2016cited by this paper
• Neural Attention Models for Sequence Classification: Analysis and Application to Key Term Extraction and Dialogue Act Detection

  2016cited by this paper
• Effective Approaches to Attention-based Neural Machine Translation

  2015cited by this paper
• From Feedforward to Recurrent LSTM Neural Networks for Language Modeling

  2015cited by this paper
• Target-Dependent Sentiment Classification with Long Short Term Memory

  2015cited by this paper
• Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks

  2015cited by this paper
• ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs

  2015cited by this paper
• Multichannel Variable-Size Convolution for Sentence Classification

  2015cited by this paper
• A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification

  2015cited by this paper
• A C-LSTM Neural Network for Text Classification

  2015cited by this paper
• Recurrent Convolutional Neural Networks for Text Classification

  2015cited by this paper
• A Convolutional Neural Network for Modelling Sentences

  2014cited by this paper
• Neural Machine Translation by Jointly Learning to Align and Translate

  2014cited by this paper
• Convolutional Neural Networks for Sentence Classification

  2014cited by this paper
• Dropout: a simple way to prevent neural networks from overfitting

  2014cited by this paper
• Recurrent Convolutional Neural Networks for Discourse Compositionality

  2013cited by this paper
• Distributed Representations of Words and Phrases and their Compositionality

  2013cited by this paper
• Efficient Estimation of Word Representations in Vector Space

  2013cited by this paper
• Joint segmentation and classification of dialog acts using conditional random fields

  2009cited by this paper
• Fast logistic regression for text categorization with variable-length n-grams

  2008cited by this paper
• A neural probabilistic language model

  2003cited by this paper
• The ICSI Meeting Corpus

  2003cited by this paper
• A Neural Probabilistic Language Model

  2003cited by this paper
• Learning to classify text using support vector machines - methods, theory and algorithms

  2002cited by this paper
• Machine learning in automated text categorization

  2001cited by this paper
• Dialogue act modeling for automatic tagging and recognition of conversational speech

  2000cited by this paper
• Dialog Act Modeling for Automatic Tagging and Recognition of Conversational Speech

  2000cited by this paper
• Text Categorization with Support Vector Machines: Learning with Many Relevant Features

  1999influential reference
• Bidirectional recurrent neural networks

  1997cited by this paper
• Long Short-Term Memory

  1997cited by this paper
• Support-Vector Networks

  1995cited by this paper

• Contextual classification of clinical records with bidirectional long short‐term memory (Bi‐LSTM) and bidirectional encoder representations from transformers (BERT) model

  2024cites this paper
• Construction risk identification using a multi-sentence context-aware method

  2024cites this paper
• Predicting Patients' Satisfaction With Mental Health Drug Treatment Using Their Reviews: Unified Interchangeable Model Fusion Approach

  2023cites this paper
• A Complete Process of Text Classification System Using State-of-the-Art NLP Models

  2022cites this paper
• A Framework for Text Classification Using Evolutionary Contiguous Convolutional Neural Network and Swarm Based Deep Neural Network

  2022cites this paper
• Classification of Insincere Questions Using Deep Learning: Quora Dataset Case Study

  2021cites this paper
• Online Extremism Detection: A Systematic Literature Review With Emphasis on Datasets, Classification Techniques, Validation Methods, and Tools

  2021cites this paper
• ResNet based on feature-inspired gating strategy

  2021cites this paper
• Classification of Marathi Text Using Hierarchical Attention (HAN)-Based Encoder-Decoder Model

  2021cites this paper
• Cross-Sentence Temporal and Semantic Relations in Video Activity Localisation

  2021cites this paper
• Interactive Dual Attention Network for Text Sentiment Classification

  2020cites this paper
• Essential Sentences for Navigating Stack Overflow Answers

  2019cites this paper