Learning with structured sparsity

This paper investigates a new learning formulation called structured sparsity, which is a natural extension of the standard sparsity concept in statistical learning and compressive sensing. By allowing arbitrary structures on the feature set, this concept generalizes the group sparsity idea. A general theory is developed for learning with structured sparsity, based on the notion of coding complexity associated with the structure. Moreover, a structured greedy algorithm is proposed to efficiently solve the structured sparsity problem. Experiments demonstrate the advantage of structured sparsity over standard sparsity.

• Publication year

  2009

• Venue

  International Conference on Machine Learning

• Publication date

  2009-03-17

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1145/1553374.1553429arXiv 0903.3002
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Adaptive Forward-Backward Greedy Algorithm for Learning Sparse Representations

  2011cited by this paper
• Author manuscript, published in "SAMPTA'09, Marseille: France (2009)" Recovery of Clustered Sparse Signals from Compressive Measurements

  2010cited by this paper
• Grouped Orthogonal Matching Pursuit for Variable Selection and Prediction

  2009cited by this paper
• Exploiting Structure in Compressive Sensing with a JPEG Basis

  2009cited by this paper
• Structured Variable Selection with Sparsity-Inducing Norms

  2009cited by this paper
• The composite absolute penalties family for grouped and hierarchical variable selection

  2009cited by this paper
• Group lasso with overlap and graph lasso

  2009influential reference
• Structured Sparsity: from Mixed Norms to Structured Shrinkage

  2009cited by this paper
• The Benefit of Group Sparsity

  2009cited by this paper
• Exploiting Structure in Wavelet-Based Bayesian Compressive Sensing

  2009cited by this paper
• Taking Advantage of Sparsity in Multi-Task Learning

  2009cited by this paper
• Multitask Compressive Sensing

  2009cited by this paper
• Sparse Signal Recovery Using Markov Random Fields

  2008cited by this paper
• Model-Based Compressive Sensing

  2008cited by this paper
• Probing the Pareto Frontier for Basis Pursuit Solutions

  2008cited by this paper
• CoSaMP: Iterative signal recovery from incomplete and inaccurate samples

  2008cited by this paper
• A Theory for Sampling Signals From a Union of Subspaces

  2008cited by this paper
• On the asymptotic properties of the group lasso estimator for linear models

  2008cited by this paper
• Convex multi-task feature learning

  2008cited by this paper
• GROUP SPARSITY VIA LINEAR-TIME PROJECTION

  2008cited by this paper
• Sparse Recovery in Large Ensembles of Kernel Machines On-Line Learning and Bandits

  2008cited by this paper
• On the Reconstruction of Block-Sparse Signals With an Optimal Number of Measurements

  2008cited by this paper
• Union support recovery in high-dimensional multivariate regression

  2008cited by this paper
• Some theoretical results on the Grouped Variables Lasso

  2008cited by this paper
• A Wavelet Tour of Signal Processing, Third Edition: The Sparse Way

  2008cited by this paper
• Grouped and Hierarchical Model Selection through Composite Absolute Penalties

  2007cited by this paper
• Consistency of the group Lasso and multiple kernel learning

  2007cited by this paper
• An Empirical Bayesian Strategy for Solving the Simultaneous Sparse Approximation Problem

  2007cited by this paper
• Optimal Solutions for Sparse Principal Component Analysis

  2007cited by this paper
• Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit

  2007cited by this paper
• Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems

  2007cited by this paper
• Aggregation for Gaussian regression

  2007cited by this paper
• Model selection and estimation in regression with grouped variables

  2006cited by this paper
• A note on the representation of positive polynomials with structured sparsity

  2006cited by this paper
• Signal Reconstruction From Noisy Random Projections

  2006cited by this paper
• Efficient adaptive density estimation per image pixel for the task of background subtraction

  2006cited by this paper
• Elements of Information Theory

  2005cited by this paper
• Decoding by linear programming

  2005cited by this paper
• A wavelet tour of signal processing

  1998cited by this paper
• Regression Shrinkage and Selection via the Lasso

  1996cited by this paper
• Embedded image coding using zerotrees of wavelet coefficients

  1993cited by this paper
• The volume of convex bodies and Banach space geometry

  1989cited by this paper
• © Institute of Mathematical Statistics, 2004 LEAST ANGLE REGRESSION

  year unknowncited by this paper

• Shifting AI Efficiency From Model-Centric to Data-Centric Compression

  2025cites this paper
• SPP-SBL: Space-Power Prior Sparse Bayesian Learning for Block Sparse Recovery

  2025cites this paper
• Weighted Group Lasso for a static EEG problem

  2025cites this paper
• I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?

  2025cites this paper
• Smoothing gradient method for group-structured sparse feedback stabilization

  2025cites this paper
• Best Subset Selection: Optimal Pursuit for Feature Selection and Elimination

  2025cites this paper
• Sparse reconstruction of sound field using pattern-coupled Bayesian compressive sensing.

  2024cites this paper
• Adaptive Block Sparse Regularization Under Arbitrary Linear Transform

  2024cites this paper
• On Variational Block Sparse Recovery With Unknown Partition and $\ell _{0}$-Norm Constraint

  2024cites this paper
• Efficient hypothesis testing strategies for latent group lasso problem

  2024cites this paper
• Group Forward-Backward Orthogonal Matching Pursuit for General Convex Smooth Functions

  2024cites this paper
• Block Sparse Bayesian Learning: A Diversified Scheme

  2024cites this paper
• Prediction of conversion from mild cognitive impairment to Alzheimer’s disease and simultaneous feature selection and grouping using Medicaid claim data

  2024cites this paper
• Off-Grid DOA Estimation Method Based on Sparse Bayesian Learning With Clustered Structural-Aware Prior Information

  2024cites this paper
• Preprocessed Spectral Clustering with Higher Connectivity for Robustness in Real-World Applications

  2024cites this paper
• Learning with Diversification from Block Sparse Signal

  2024cites this paper
• Bayesian Structured-Sparse Modeling Using a Bernoulli–Laplacian Prior

  2023cites this paper
• A minimax optimal approach to high-dimensional double sparse linear regression

  2023cites this paper
• Interpretable Multi-Scale Neural Network for Granger Causality Discovery

  2023cites this paper
• Learning Partial Correlation based Deep Visual Representation for Image Classification

  2023influential citation
• A Novel Sparse Image Reconstruction Based on Iteratively Reweighted Least Squares Using Diagonal Regularization

  2023cites this paper
• Smoothing the Edges: A General Framework for Smooth Optimization in Sparse Regularization using Hadamard Overparametrization

  2023cites this paper
• A unified consensus-based parallel ADMM algorithm for high-dimensional regression with combined regularizations

  2023cites this paper
• Frank-Wolfe algorithm for DC optimization problem

  2023cites this paper
• Learning Sparse Group Models Through Boolean Relaxation

  2023cites this paper
• Low Rank Optimization for Efficient Deep Learning: Making A Balance between Compact Architecture and Fast Training

  2023cites this paper
• An Adaptive Half-Space Projection Method for Stochastic Optimization Problems with Group Sparse Regularization

  2023cites this paper
• A review of advances in imaging methodology in fluorescence molecular tomography

  2022cites this paper
• Datamodels: Predicting Predictions from Training Data

  2022cites this paper
• Enhanced nuclear norm based matrix regression for occluded face recognition

  2022cites this paper
• MGiaD: Multigrid in all dimensions. Efficiency and robustness by coarsening in resolution and channel dimensions

  2022cites this paper
• Block-Sparse Signal Recovery via General Total Variation Regularized Sparse Bayesian Learning

  2022cites this paper
• On Acceleration for Convex Composite Minimization with Noise-Corrupted Gradients and Approximate Proximal Mapping

  2022cites this paper
• ORKA: Object reconstruction using a K-approximation graph

  2022cites this paper
• Granger Causality using Neural Networks

  2022cites this paper
• Survey on compressed sensing reconstruction method for 3D data

  2022cites this paper
• What is the Largest Sparsity Pattern That Can Be Recovered By 1-Norm Minimization?

  2022cites this paper
• Minimum Description Length Penalization for Group and Multi- Minimum Description Length Penalization for Group and MultiTask Sparse Learning Task Sparse Learning

  2021cites this paper
• An improved tensor regression model via location smoothing

  2021cites this paper
• Neural Granger Causality

  2021cites this paper
• The Values Encoded in Machine Learning Research

  2021cites this paper
• Only Train Once: A One-Shot Neural Network Training And Pruning Framework

  2021cites this paper
• Granger Causality: A Review and Recent Advances

  2021cites this paper
• On the parameter learning for Perturb-and-MAP models

  2021cites this paper
• Robust brain MR image compressive sensing via re-weighted total variation and sparse regression.

  2021cites this paper
• Group-Sparse SVD Models via <inline-formula><tex-math notation="LaTeX">$L_1$</tex-math><alternatives><mml:math><mml:msub><mml:mi>L</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:math><inline-graphic xlink:href="zhang-ieq1-2932063.gif"/></alternatives></inline-formula>- and <inline-formula><tex-math nota

  2021cites this paper
• Lattice Pattern Analysis

  2021cites this paper
• Background subtraction for moving object detection: explorations of recent developments and challenges

  2021cites this paper
• Block-Sparse Recovery With Optimal Block Partition

  2021cites this paper
• A Convex Penalty for Block-Sparse Signals with Unknown Structures

  2021cites this paper
• Attribute Guided Sparse Tensor-Based Model for Person Re-Identification

  2021cites this paper
• An Efficient Sharing Grouped Convolution via Bayesian Learning

  2021cites this paper
• Modelling Heterogeneity Using Bayesian Structured Sparsity

  2021cites this paper
• Structural sparsity in control design of active and semi‐active systems

  2021cites this paper
• General Total Variation Regularized Sparse Bayesian Learning for Robust Block-Sparse Signal Recovery

  2021cites this paper
• Convex Latent Effect Logit Model via Sparse and Low-rank Decomposition

  2021cites this paper
• Authentication of the geographic origin of Yangshan region peaches based on hyperspectral imaging

  2021cites this paper
• What is the Largest Sparsity Pattern That Can Be Recovered by 1-Norm Minimization?

  2021cites this paper
• Vehicle Monitoring and Surveillance Through Vehicular Sensor Network

  2021cites this paper
• Exclusive Group Lasso for Structured Variable Selection

  2021cites this paper
• Structural Sparsity in Multiple Measurements

  2021cites this paper
• Correlating a Requisite Diversity of Metaphorical Patterns

  2020cites this paper
• Fluorescence Molecular Tomography Based on Group Sparsity Priori for Morphological Reconstruction of Glioma

  2020cites this paper
• Joint Sparse Channel Recovery With Quantized Feedback for Multi-User Massive MIMO Systems

  2020cites this paper
• Theoretical guarantees for graph sparse coding

  2020cites this paper
• CMCS-net: image compressed sensing with convolutional measurement via DCNN

  2020cites this paper
• Crack detection in carbide anvil using acoustic signal and deep learning with particle swarm optimisation

  2020cites this paper
• Sparse representations and compressive sampling approaches in engineering mechanics: A review of theoretical concepts and diverse applications

  2020cites this paper
• Finding Distributed Needles in Neural Haystacks

  2020cites this paper
• Learning Multiple Nonlinear Dynamical Systems with Side Information

  2020cites this paper
• A Half-Space Stochastic Projected Gradient Method for Group Sparsity Regularization

  2020cites this paper
• Half-Space Proximal Stochastic Gradient Method for Group-Sparsity Regularized Problem

  2020cites this paper
• Pruning Redundant Mappings in Transformer Models via Spectral-Normalized Identity Prior

  2020cites this paper
• Apple Variety Recognition Based on Multiview Feature Fusion

  2020cites this paper
• Making Neural Networks Interpretable with Attribution: Application to Implicit Signals Prediction

  2020cites this paper
• Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning

  2020cites this paper
• Approximation Guarantees of Local Search Algorithms via Localizability of Set Functions

  2020cites this paper
• Theoretical basis of compressive sensing

  2020cites this paper
• On change-point estimation under Sobolev sparsity

  2020cites this paper
• Calibrationless joint compressed sensing reconstruction for rapid parallel MRI

  2020cites this paper
• When Dictionary Learning Meets Deep Learning: Deep Dictionary Learning and Coding Network for Image Recognition With Limited Data

  2020cites this paper
• Machine learning tools for biomarker discovery

  2020cites this paper
• Beyond Covariance: SICE and Kernel Based Visual Feature Representation

  2020cites this paper
• Sparse and redundant signal representations for x-ray computed tomography

  2019cites this paper
• Atomic Norms In Group Sliding Sparse Recovery

  2019cites this paper
• Structured dictionary learning using mixed-norms and group-sparsity constraint

  2019cites this paper
• Graph-Based Clustering via Group Sparsity and Manifold Regularization

  2019cites this paper
• A Video Representation Method Based on Multi-View Structure Preserving Embedding for Action Retrieval

  2019cites this paper
• Greedy and IHT Algorithms for Non-convex Optimization with Monotone Costs of Non-zeros

  2019cites this paper
• Generalized Nullspace Property for Structurally Sparse Signals

  2019influential citation
• Variance State Propagation for Structured Sparse Bayesian Learning

  2019cites this paper
• Benefits of dimension reduction in penalized regression methods for high-dimensional grouped data: a case study in low sample size

  2019cites this paper
• SpaRCe: Sparse reservoir computing

  2019cites this paper
• Nonlocal Tensor Sparse Representation and Low-Rank Regularization for Hyperspectral Image Compressive Sensing Reconstruction

  2019cites this paper
• Learning Sparse Distributions using Iterative Hard Thresholding

  2019cites this paper
• Regularizing Generative Models Using Knowledge of Feature Dependence

  2019cites this paper
• DSTPCA: Double-Sparse Constrained Tensor Principal Component Analysis Method for Feature Selection

  2019cites this paper
• Radar and Communication Coexistence: An Overview: A Review of Recent Methods

  2019cites this paper
• Error Analysis on Graph Laplacian Regularized Estimator

  2019cites this paper
• Intracluster Structured Low-Rank Matrix Analysis Method for Hyperspectral Denoising

  2019cites this paper