Sparse additive models

Summary.  We present a new class of methods for high dimensional non‐parametric regression and classification called sparse additive models. Our methods combine ideas from sparse linear modelling and additive non‐parametric regression. We derive an algorithm for fitting the models that is practical and effective even when the number of covariates is larger than the sample size. Sparse additive models are essentially a functional version of the grouped lasso of Yuan and Lin. They are also closely related to the COSSO model of Lin and Zhang but decouple smoothing and sparsity, enabling the use of arbitrary non‐parametric smoothers. We give an analysis of the theoretical properties of sparse additive models and present empirical results on synthetic and real data, showing that they can be effective in fitting sparse non‐parametric models in high dimensional data.

• Publication year

  2007

• Venue

  Journal of the Royal Statistical Society: Series B (Statistical Methodology)

• Publication date

  2007-11-28

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1111/j.1467-9868.2009.00718.xarXiv 0711.4555
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition

  2009cited by this paper
• High-dimensional additive modeling

  2008cited by this paper
• High-Dimensional Graphical Model Selection Using $\ell_1$-Regularized Logistic Regression

  2008cited by this paper
• LASSO-TYPE RECOVERY OF SPARSE REPRESENTATIONS FOR HIGH-DIMENSIONAL DATA

  2008influential reference
• SpAM: Sparse Additive Models

  2007cited by this paper
• Multi-Stage Variable Selection: Screen and Clean

  2007cited by this paper
• Sparsity oracle inequalities for the Lasso

  2007cited by this paper
• Accelerated Projected Gradient Method for Linear Inverse Problems with Sparsity Constraints

  2007cited by this paper
• Nonnegative Garrote Component Selection in Functional ANOVA models

  2007cited by this paper
• High-Dimensional Graphical Model Selection Using ℓ1-Regularized Logistic Regression

  2006cited by this paper
• On Model Selection Consistency of Lasso

  2006cited by this paper
• Sharp thresholds for high-dimensional and noisy recovery of sparsity

  2006influential reference
• Model selection and estimation in regression with grouped variables

  2006influential reference
• The Adaptive Lasso and Its Oracle Properties

  2006cited by this paper
• Component selection and smoothing in multivariate nonparametric regression

  2006influential reference
• High-dimensional graphs and variable selection with the Lasso

  2006influential reference
• Nonparametric and Semiparametric Models

  2005cited by this paper
• Nonparametric Inferences for Additive Models

  2005cited by this paper
• Nonparametric and Semiparametric Models

  2004cited by this paper
• Persistence in high-dimensional linear predictor selection and the virtue of overparametrization

  2004influential reference
• An iterative thresholding algorithm for linear inverse problems with a sparsity constraint

  2003cited by this paper
• Regularization of Wavelet Approximations

  2001cited by this paper
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties

  2001cited by this paper
• Functional aggregation for nonparametric regression

  2000cited by this paper
• Variable Selection via Penalized Likelihood (cid:3)

  1999cited by this paper
• Asymptotic Statistics: Frontmatter

  1998influential reference
• Regression Shrinkage and Selection via the Lasso

  1996cited by this paper
• Emergence of simple-cell receptive field properties by learning a sparse code for natural images

  1996cited by this paper
• Probability in Banach Spaces: Isoperimetry and Processes

  1991cited by this paper
• Linear Smoothers and Additive Models

  1989influential reference
• Generalized Additive Models

  1986cited by this paper

• Variational Bayes neural networks for high-dimensional non parametric regression: Minimax optimality and adaptivity

  2026cites this paper
• Dimension Reduction in Importance Sampling: Balancing Concentration and Exploration Through Variable Selection

  2026cites this paper
• Conditional independence test in factor models via projection correlation

  2026cites this paper
• Beyond False Discovery Rate: A Stepdown Group SLOPE Approach for Grouped Variable Selection

  2026cites this paper
• Sparse Additive Model Pruning for Order-Based Causal Structure Learning

  2026cites this paper
• Bayesian Smoothing and Variable selection using Variational Automatic Relevance Determination

  2026cites this paper
• An ensemble test for ultrahigh-dimensional nonparametric models via multiple data splitting

  2025cites this paper
• Model averaging prediction for survival data with time-dependent effects

  2025cites this paper
• SplitWise regression for capturing nonlinear effects in interpretable model selection

  2025cites this paper
• Transparent and Fair Profiling in Employment Services: Evidence from Switzerland

  2025cites this paper
• Variable Selection in Nonparametric Additive Models via Data Splitting

  2025cites this paper
• Forecasting time series with constraints

  2025cites this paper
• AVERAGING ESTIMATORS OF HETEROGENEOUS TREATMENT EFFECTS UNDER ADDITIVE MODELS

  2025cites this paper
• Reluctant Interaction Inference after Additive Modeling

  2025cites this paper
• Retrieving interpretability to support vector machine regression models in dynamic system identification

  2025cites this paper
• An Interpretable Deep Learning Model for General Insurance Pricing

  2025cites this paper
• Sparse and smooth additive isotonic model in high-dimensional settings

  2025cites this paper
• When Interpretability Meets Generalization: Delta-GAM for Robust Extrapolation in Out-of-Distribution Settings

  2025cites this paper
• Simultaneous selection of knots and variables in additive models

  2025cites this paper
• Sparse Nonparametric Contextual Bandits

  2025cites this paper
• A novel variable selection method in nonlinear multivariate models using B-splines with an application to geoscience

  2025cites this paper
• Estimation and uniform inference in sparse high-dimensional additive models

  2025cites this paper
• Estimation of High-Dimensional Nonlinear Vector Autoregressive Models

  2025influential citation
• Change surface regression for nonlinear subgroup identification with application to warfarin pharmacogenomics data.

  2025cites this paper
• Sparse Deep Additive Model with Interactions: Enhancing Interpretability and Predictability

  2025cites this paper
• Representation Retrieval Learning for Heterogeneous Data Integration

  2025cites this paper
• Covariate-dependent Graphical Model Estimation via Neural Networks with Statistical Guarantees

  2025cites this paper
• A new measure of dependence: Integrated R2

  2025cites this paper
• Decoding longitudinal microbiome trajectories: an interpretable machine learning approach for biomarker discovery and prediction

  2025cites this paper
• Posterior Summarization for Variable Selection in Bayesian Tree Ensembles

  2025cites this paper
• Nonlinear Causal Discovery for Grouped Data

  2025cites this paper
• Robust variable selection in high-dimensional nonparametric additive model

  2025cites this paper
• A Feature Transformation and Selection Method to Acquire an Interpretable Model Incorporating Nonlinear Effects

  2025cites this paper
• Interpretable Meta-weighting Sparse Neural Additive Networks for Datasets with Label Noise and Class Imbalance

  2025cites this paper
• The Flexible Accumulation Model for High Density Temporal Exposures

  2025cites this paper
• Dimensionality reduction in multivariate nonparametric regression via nuclear norm penalization

  2025cites this paper
• Variable Selection for Additive Global Fr\'echet Regression

  2025cites this paper
• Optimal Distributed Model Averaging for Multivariate Additive Model

  2025cites this paper
• Supervised Predictive Modeling of High-dimensional Data with Group l0-norm Constrained Neural Networks

  2025cites this paper
• Interpretable Bilevel Additive Taylor Model for Datasets with Noisy Labels and Imbalanced Classes

  2025influential citation
• Interpretable Machine Learning for Healthcare Risk Prediction Using Sparse Interaction Models

  2025cites this paper
• An Intelligent Framework for Optimal Feature Selection of PD Signals Produced by Electrical Defects in MV Switchgear

  2025cites this paper
• Locally adaptive sparse additive quantile regression model with TV penalty

  2024cites this paper
• Bayesian Effect Selection in Additive Models with an Application to Time-to-Event Data

  2024cites this paper
• High–dimensional local linear regression under sparsity and convex losses

  2024cites this paper
• Covariate selection for optimizing balance with an innovative adaptive randomization approach

  2024cites this paper
• Inference on Large-scale Partially Functional Linear Model with Heterogeneous Errors

  2024influential citation
• Hyperparameter recommendation via automated meta-feature selection embedded with kernel group Lasso learning

  2024cites this paper
• Federated multi-task Bayesian network learning in the presence of overlapping and distinct variables

  2024influential citation
• Bayesian Smoothing and Feature Selection Using variational Automatic Relevance Determination

  2024cites this paper
• A novel high-dimensional model for identifying regional DNA methylation QTLs

  2024cites this paper
• Variable selection for additive models with missing data via multiple imputation

  2024cites this paper
• Asymptotically faster estimation of high‐dimensional additive models using subspace learning

  2024influential citation
• Efficient functional Lasso kernel smoothing for high-dimensional additive regression

  2024cites this paper
• Projection pursuit Bayesian regression for symmetric matrix predictors

  2024cites this paper
• A High-dimensional Additive Nonparametric Model

  2024cites this paper
• Generalized Sparse Additive Model with Unknown Link Function

  2024cites this paper
• Maximally informative feature selection using Information Imbalance: Application to COVID-19 severity prediction

  2024cites this paper
• Feature and functional form selection in additive models via mixed-integer optimization

  2024cites this paper
• Parallel Momentum Methods Under Biased Gradient Estimations

  2024cites this paper
• LCEN: A Nonlinear, Interpretable Feature Selection and Machine Learning Algorithm

  2024cites this paper
• Multi-Layer Kernel Machines: Fast and Optimal Nonparametric Regression with Uncertainty Quantification

  2024cites this paper
• Likelihood ratio combination of multiple biomarkers via smoothing spline estimated densities

  2024cites this paper
• Sparse and Faithful Explanations Without Sparse Models

  2024cites this paper
• Biomedical Open Source Software: Crucial Packages and Hidden Heroes

  2024cites this paper
• A high-dimensional single-index regression for interactions between treatment and covariates

  2024cites this paper
• Bayesian High-dimensional Grouped-regression using Sparse Projection-posterior

  2024cites this paper
• A Variational Spike-and-Slab Approach for Group Variable Selection

  2023influential citation
• Variable Selection and Minimax Prediction in High-dimensional Functional Linear Models

  2023cites this paper
• Joint semiparametric kernel network regression

  2023cites this paper
• A Study of Multifactor Quantitative Stock-Selection Strategies Incorporating Knockoff and Elastic Net-Logistic Regression

  2023cites this paper
• Adaptive debiased machine learning using data-driven model selection techniques

  2023cites this paper
• Consideration Set Sampling to Analyze Undecided Respondents

  2023cites this paper
• Nonparametric Additive Value Functions: Interpretable Reinforcement Learning with an Application to Surgical Recovery

  2023cites this paper
• causalAssembly: Generating Realistic Production Data for Benchmarking Causal Discovery

  2023influential citation
• Sparse Additive Machine With the Correntropy-Induced Loss

  2023cites this paper
• Sparse-Input Neural Network using Group Concave Regularization

  2023cites this paper
• Estimation of Error Variance in Genomic Selection for Ultrahigh Dimensional Data

  2023cites this paper
• How to Open a Black Box Classifier for Tabular Data

  2023cites this paper
• Semiparametric model averaging method for survival probability predictions of patients

  2023cites this paper
• End-to-end Feature Selection Approach for Learning Skinny Trees

  2023cites this paper
• Forecasting Long-Series Daily Reference Evapotranspiration Based on Best Subset Regression and Machine Learning in Egypt

  2023cites this paper
• Minimax Signal Detection in Sparse Additive Models

  2023cites this paper
• Variable selection in additive models via hierarchical sparse penalty

  2023influential citation
• Deep neural networks for nonparametric interaction models with diverging dimension

  2023cites this paper
• Robust partially linear models for automatic structure discovery

  2023cites this paper
• Choosing shape parameters for regression in reproducing kernel Hilbert space and variable selection

  2023cites this paper
• Gradient Learning With the Mode-Induced Loss: Consistency Analysis and Applications

  2023cites this paper
• Stepdown SLOPE for Controlled Feature Selection

  2023cites this paper
• Bayesian pathway selection

  2023cites this paper
• Sharper Analysis for Minibatch Stochastic Proximal Point Methods: Stability, Smoothness, and Deviation

  2023cites this paper
• A SIMPLE EXTENSION OF AZADKIA & CHATTERJEE’S RANK CORRELATION TO A VECTOR OF ENDOGENOUS VARIABLES

  2023cites this paper
• GRAND-SLAMIN' Interpretable Additive Modeling with Structural Constraints

  2023cites this paper
• The coming of age of interpretable and explainable machine learning models

  2023cites this paper
• Neural partially linear additive model

  2023cites this paper
• An Adaptive Neural Network Regression Method for Structure Identification

  2023cites this paper
• Estimation and Inference on Nonlinear and Heterogeneous Effects

  2023cites this paper
• Variable Selection Using Deep Variational Information Bottleneck with Drop-Out-One Loss

  2023cites this paper
• A RELUCTANT ADDITIVE MODEL FRAMEWORK FOR INTERPRETABLE NONLINEAR INDIVIDUALIZED TREATMENT RULES.

  2023cites this paper
• Spectral Deconfounding for High-Dimensional Sparse Additive Models

  2023cites this paper