Linguistic Structured Sparsity in Text Categorization

We introduce three linguistically motivated structured regularizers based on parse trees, topics, and hierarchical word clusters for text categorization. These regularizers impose linguistic bias in feature weights, enabling us to incorporate prior knowledge into conventional bagof-words models. We show that our structured regularizers consistently improve classification accuracies compared to standard regularizers that penalize features in isolation (such as lasso, ridge, and elastic net regularizers) on a range of datasets for various text prediction problems: topic classification, sentiment analysis, and forecasting.

• Publication year

  2014

• Venue

  Annual Meeting of the Association for Computational Linguistics

• Publication date

  2014-06-01

• Fields of study

  Linguistics, Computer Science

• Identifiers
  DOI 10.3115/v1/P14-1074
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Making the Most of Bag of Words: Sentence Regularization with Alternating Direction Method of Multipliers

  2014influential reference
• Structured Penalties for Log-Linear Language Models

  2013influential reference
• Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

  2013influential reference
• Textual Predictors of Bill Survival in Congressional Committees

  2012cited by this paper
• Structured Sparsity via Alternating Direction Methods

  2011cited by this paper
• Structured Sparsity in Structured Prediction

  2011influential reference
• Large Scale Text Classiflcation using Semi-supervised Multinomial

  2011cited by this paper
• Smoothing Proximal Gradient Method for General Structured Sparse Learning

  2011cited by this paper
• The Group Lasso for Stable Recovery of Block-Sparse Signal Representations

  2011cited by this paper
• Discovering Sociolinguistic Associations with Structured Sparsity

  2011influential reference
• Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach

  2011cited by this paper
• Efficient Methods for Overlapping Group Lasso

  2011cited by this paper
• Large Scale Text Classification using Semisupervised Multinomial Naive Bayes

  2011cited by this paper
• Predicting a Scientific Community’s Response to an Article

  2011influential reference
• Movie Reviews and Revenues: An Experiment in Text Regression

  2010cited by this paper
• A note on the group lasso and a sparse group lasso

  2010influential reference
• Multi-Level Structured Models for Document-Level Sentiment Classification

  2010influential reference
• Group lasso with overlap and graph lasso

  2009cited by this paper
• Predicting Risk from Financial Reports with Regression

  2009cited by this paper
• Latent Dirichlet Allocation

  2009cited by this paper
• The ACL anthology network corpus

  2009cited by this paper
• Sparsity and persistence: mixed norms provide simple signal models with dependent coefficients

  2009cited by this paper
• Structured Variable Selection with Sparsity-Inducing Norms

  2009cited by this paper
• Feature Selection via Block-Regularized Regression

  2008cited by this paper
• Improved Inference for Unlexicalized Parsing

  2007influential reference
• Get out the vote: Determining support or opposition from Congressional floor-debate transcripts

  2006cited by this paper
• Model selection and estimation in regression with grouped variables

  2006influential reference
• Sparsity and smoothness via the fused lasso

  2005cited by this paper
• Addendum: Regularization and variable selection via the elastic net

  2005cited by this paper
• Adaptive Sparseness Using Jeffreys Prior

  2001cited by this paper
• Text Classification from Labeled and Unlabeled Documents using EM

  2000cited by this paper
• A survey of smoothing techniques for ME models

  2000cited by this paper
• Ridge Regression: Biased Estimation for Nonorthogonal Problems

  2000cited by this paper
• Regression Shrinkage and Selection via the Lasso

  1996cited by this paper
• Class-Based n-gram Models of Natural Language

  1992cited by this paper
• A method for nonlinear constraints in minimization problems

  1969cited by this paper
• Multiplier and gradient methods

  1969cited by this paper

• Dual feature reduction for the sparse-group lasso and its adaptive variant

  2024cites this paper
• Encoding and Decoding Graph Representations of Natural Language

  2024cites this paper
• A dual deep neural network with phrase structure and attention mechanism for sentiment analysis

  2021cites this paper
• Location Classification Based on Tweets

  2021cites this paper
• A Densely Connected GRU Neural Network Based on Coattention Mechanism for Chinese Rice-Related Question Similarity Matching

  2021cites this paper
• Changing the Basis of Contextual Representations with Explicit Semantics

  2021cites this paper
• A History and Theory of Textual Event Detection and Recognition

  2020cites this paper
• A Multi-Channel Deep Neural Network for Relation Extraction

  2020cites this paper
• Understanding the Context of Microactions on the Web

  2020cites this paper
• Discrete Word Embedding for Logical Natural Language Understanding

  2020cites this paper
• Regularised Text Logistic Regression: Key Word Detection and Sentiment Classification for Online Reviews

  2020cites this paper
• Automatic Location Type Classification From Social-Media Posts

  2020cites this paper
• Geosocial Location Classification: Associating Type to Places Based on Geotagged Social-Media Posts

  2020cites this paper
• Structure-Based Supervised Term Weighting and Regularization for Text Classification

  2019cites this paper
• Transformation of Dense and Sparse Text Representations

  2019cites this paper
• Evaluating Discourse in Structured Text Representations

  2019cites this paper
• Sentiment and position-taking analysis of parliamentary debates: a systematic literature review

  2019cites this paper
• Orthogonal Matching Pursuit for Text Classification

  2018cites this paper
• Exploiting covariate embeddings for classification using Gaussian processes

  2018cites this paper
• Bridging CNNs, RNNs, and Weighted Finite-State Machines

  2018cites this paper
• Attention-Based Linguistically Constraints Network for Aspect-Level Sentiment

  2018cites this paper
• Feature Selection as Causal Inference: Experiments with Text Classification

  2017cites this paper
• Neural Discourse Structure for Text Categorization

  2017cites this paper
• Text Categorization Using Weighted Hyper Rectangular Keyword Extraction

  2017cites this paper
• Automated Geocoding of Textual Documents: A Survey of Current Approaches

  2017cites this paper
• A Tree Regularized Classifier - Exploiting Hierarchical Structure Information in Feature Vector for Human Action Recognition

  2017influential citation
• An iterative approach for the global estimation of sentence similarity

  2017cites this paper
• Learning Sparse Overcomplete Word Vectors Without Intermediate Dense Representations

  2017cites this paper
• Structured Sparse Methods for Imaging Genetics

  2017cites this paper
• Learning Structured Text Representations

  2017influential citation
• Exploiting Domain Knowledge via Grouped Weight Sharing with Application to Text Categorization

  2017cites this paper
• Rotated Word Vector Representations and their Interpretability

  2017cites this paper
• Arguments for Semantic Folding and Hierarchical Temporal Memory Theory Copyright

  2016cites this paper
• Regularizing Text Categorization with Clusters of Words

  2016cites this paper
• A Joint Sentiment-Target-Stance Model for Stance Classification in Tweets

  2016cites this paper
• Robust Text Classification for Sparsely Labelled Data Using Multi-level Embeddings

  2016cites this paper
• Understanding Neural Networks through Representation Erasure

  2016cites this paper
• Sparsifying Word Representations for Deep Unordered Sentence Modeling

  2016cites this paper
• Linguistically Regularized LSTM for Sentiment Classification

  2016influential citation
• Sparse models for imaging genetics

  2016cites this paper
• Human action recognition with group lasso regularized-support vector machine

  2016cites this paper
• Evaluation of Word Vector Representations by Subspace Alignment

  2015cites this paper
• Speech emotion classification using tree-structured sparse logistic regression

  2015cites this paper
• Geocoding textual documents through the usage of hierarchical classifiers

  2015cites this paper
• Parametric Maxflows for Structured Sparse Learning with Convex Relaxations of Submodular Functions

  2015cites this paper
• Sparse, Contextually Informed Models for Irony Detection: Exploiting User Communities, Entities and Sentiment

  2015cites this paper
• Compressive Document Summarization via Sparse Optimization

  2015cites this paper
• Proposal : Beyond the Distributional Hypothesis

  2015cites this paper
• Automatic Intelligibility Assessment of Dysarthric Speech Using Phonologically-Structured Sparse Linear Model

  2015cites this paper
• OmniGraph: Rich Representation and Graph Kernel Learning

  2015cites this paper
• Sparse Models of Natural Language Text

  2015cites this paper
• Multi-Layer Feature Reduction for Tree Structured Group Lasso via Hierarchical Projection

  2015cites this paper
• Reducing Lexical Features in Parsing by Word Embeddings

  2015cites this paper
• Sparse Overcomplete Word Vector Representations

  2015cites this paper
• Two-Layer Feature Reduction for Sparse-Group Lasso via Decomposition of Convex Sets

  2014cites this paper