COORDINATE DESCENT ALGORITHMS FOR NONCONVEX PENALIZED REGRESSION, WITH APPLICATIONS TO BIOLOGICAL FEATURE SELECTION.

A number of variable selection methods have been proposed involving nonconvex penalty functions. These methods, which include the smoothly clipped absolute deviation (SCAD) penalty and the minimax concave penalty (MCP), have been demonstrated to have attractive theoretical properties, but model fitting is not a straightforward task, and the resulting solutions may be unstable. Here, we demonstrate the potential of coordinate descent algorithms for fitting these models, establishing theoretical convergence properties and demonstrating that they are significantly faster than competing approaches. In addition, we demonstrate the utility of convexity diagnostics to determine regions of the parameter space in which the objective function is locally convex, even though the penalty is not. Our simulation study and data examples indicate that nonconvex penalties like MCP and SCAD are worthwhile alternatives to the lasso in many applications. In particular, our numerical results suggest that MCP is the preferred approach among the three methods.

• Publication year

  2011

• Venue

  Annals of Applied Statistics

• Publication date

  2011-04-14

• Fields of study

  Biology, Mathematics, Computer Science, Medicine

• Identifiers
  DOI 10.1214/10-AOAS388arXiv 1104.2748PMID 22081779PMCID PMC3212875
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Regularization Paths for Generalized Linear Models via Coordinate Descent.

  2010cited by this paper
• Nearly unbiased variable selection under minimax concave penalty

  2010influential reference
• One-step Sparse Estimates in Nonconcave Penalized Likelihood Models.

  2008cited by this paper
• Coordinate descent algorithms for lasso penalized regression

  2008cited by this paper
• Feature selection and molecular classification of cancer using genetic programming.

  2007cited by this paper
• L1‐regularization path algorithm for generalized linear models

  2007cited by this paper
• PATHWISE COORDINATE OPTIMIZATION

  2007cited by this paper
• Generalized Linear Models

  2002cited by this paper
• Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization

  2001cited by this paper
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties

  2001cited by this paper
• Molecular classification of cancer: class discovery and class prediction by gene expression monitoring.

  1999cited by this paper
• WAVESHRINK WITH FIRM SHRINKAGE

  1997cited by this paper
• Regression Shrinkage and Selection via the Lasso

  1996cited by this paper
• Understanding WaveShrink: Variance and bias estimation

  1996cited by this paper
• Heuristics of instability and stabilization in model selection

  1996influential reference
• Ideal spatial adaptation by wavelet shrinkage

  1994cited by this paper
• Generalized Linear Models

  1992cited by this paper
• © Institute of Mathematical Statistics, 2004 LEAST ANGLE REGRESSION

  year unknowncited by this paper

• Investigating commodity price interdependence with Granger causality networks

  2026cites this paper
• Penalized partially linear regression with deep neural networks for high-dimensional learning

  2026cites this paper
• A Flexible Empirical Bayes Approach to Generalized Linear Models, with Applications to Sparse Logistic Regression

  2026cites this paper
• Scalable Ultra-High-Dimensional Quantile Regression with Genomic Applications

  2026influential citation
• Smooth SCAD: A Raised Cosine SCAD Type Thresholding Rule for Wavelet Denoising

  2026cites this paper
• Meta‐Learning Credit Risk Prediction by Fusing Regularized Logistic Regression and Random Forest

  2026cites this paper
• Adaptive Penalized Doubly Robust Regression for Longitudinal Data

  2026influential citation
• Penalized Likelihood Estimation of Continuation Ratio Models for Ordinal Response and Its Application in CGSS Data

  2026cites this paper
• DeCAF defines clinical fibroblast subtypes and multidimensional tumor-stroma crosstalk shaping prognosis and immunotherapy response.

  2026cites this paper
• Making highways and workplaces safer: An interpretable machine learning approach to predicting recent cannabis use and impairment

  2026cites this paper
• Fast variable selection under ℓ0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\ell _0$$\end{document} regularization

  2026cites this paper
• Causal assessment of gene regulatory network in single-cell transcriptomics data based on Bayesian networks

  2025cites this paper
• High-Dimensional Covariate-Dependent Gaussian Graphical Models

  2025cites this paper
• Covariate-adjusted construction of gene regulatory networks using a combination of generalized linear model and penalized maximum likelihood

  2025cites this paper
• Echo state network with a non-convex penalty for nonlinear time series prediction

  2025cites this paper
• Scalable Subset Selection in Linear Mixed Models

  2025cites this paper
• Variable selection methods for descriptive modeling

  2025influential citation
• A Multi-Omics Framework for Survival Mediation Analysis of High-Dimensional Proteogenomic Data

  2025cites this paper
• Transcriptional modulation unique to vulnerable motor neurons predicts ALS across species and SOD1 mutations

  2025cites this paper
• Regularized Structural Equation Modeling with Both Factors and Components

  2025cites this paper
• A Stable Lasso

  2025cites this paper
• Sparse-Group Factor Analysis for High-Dimensional Time Series

  2025cites this paper
• Differential item functioning detection across multiple groups.

  2025cites this paper
• Guidelines for LASSO and derivatives use under different dependence and scale structures

  2025cites this paper
• Data-adaptive automatic threshold calibration for stability selection

  2025cites this paper
• Low-Rank Matrix Recovery Via Nonconvex Optimization Methods with Application to Errors-in-Variables Matrix Regression

  2025cites this paper
• Sparse classification with positive-confidence data in high dimensions

  2025cites this paper
• Efficient Computation of High‐Dimensional Penalized Piecewise Constant Hazard Random Effects Models

  2025influential citation
• Rethinking Hard Thresholding Pursuit: Full Adaptation and Sharp Estimation

  2025cites this paper
• HighDimMixedModels.jl: Robust high-dimensional mixed-effects models across omics data

  2025cites this paper
• Confinement and pasture-based dairy herds differ in plasma lipid profiles.

  2025cites this paper
• RNA Sequencing Reveals Inflammatory and Metabolic Changes in the Lung and Brain After Carbon Black and Naphthalene Whole Body Inhalation Exposure in a Rodent Model of Military Burn Pit Exposures

  2025influential citation
• Reconstructing Dynamic Gene Regulatory Networks Using f-Divergence from Time-Series scRNA-Seq Data

  2025cites this paper
• A powerful penalized multinomial logistic regression approach

  2025cites this paper
• The variable selection of two-part regression model for semicontinuous data

  2025cites this paper
• Formal Bayesian Transfer Learning via the Total Risk Prior

  2025cites this paper
• Research and Application of Oracle Properties and Algorithm Optimization of Non-Convex Penalties Based on High-Dimensional Lasso-Penalized Linear Regression

  2025cites this paper
• Evolutionary Feature-wise Thresholding for Binary Representation of NLP Embeddings

  2025cites this paper
• Impact of magnesium oxide nanoparticles and hydrogen enrichment on CI engine performance with Mahua oil biodiesel using machine learning

  2025cites this paper
• Enhanced Cyclic Coordinate Descent Methods for Elastic Net Penalized Linear Models

  2025cites this paper
• Variable selection and estimation of partially functional linear multiple varying coefficient spatial autoregressive model under constraint conditions

  2025cites this paper
• A Systematic Intelligent Optimization Framework for a Sustained-Release Formulation Design

  2025cites this paper
• nonprobsvy -- An R package for modern methods for non-probability surveys

  2025cites this paper
• Beyond mindfulness: the importance of body compassion in colorectal cancer distress

  2025cites this paper
• A Bayesian Spike-and-Slab Sensor Selection Approach for High-Dimensional Prognostics

  2025cites this paper
• On the greedy coordinate descent methods for solving large linear least-squares problems

  2025cites this paper
• Penalty Strategies in Semiparametric Regression Models

  2025cites this paper
• Distributed optimization for penalized regression in massive compositional data

  2025cites this paper
• Two-stage subsampling variable selection for sparse high-dimensional generalized linear models

  2025cites this paper
• Regularized Estimation of the Loading Matrix in Factor Models for High-Dimensional Time Series

  2025cites this paper
• A Newton-Type Method for ℓ0-Regularized Accelerated Failure Time Model Under the Case–Cohort Design

  2025cites this paper
• Comparing Variable Selection and Model Averaging Methods for Logistic Regression

  2025cites this paper
• Integrating Complex Selection Rules Into the Latent Overlapping Group Lasso for the Construction of Coherent Prediction Models

  2025cites this paper
• Nash: Neural Adaptive Shrinkage for Structured High-Dimensional Regression

  2025cites this paper
• Single-index logistic model for high-dimensional group testing data

  2025influential citation
• A regularized multi-state model for covariate selection with interval-censored survival data

  2025cites this paper
• Model selection for vector autoregressive processes using broken adaptive ridge

  2025cites this paper
• SDFPI: Efficient L 0 variable selection for Cox’s proportional hazards model

  2025cites this paper
• Scalable Hessian-free Proximal Conjugate Gradient Method for Nonconvex and Nonsmooth Optimization

  2025cites this paper
• Variable selection of partially functional linear spatial autoregressive model with a diverging number of parameters

  2024cites this paper
• Detecting interactions using Bayesian additive regression trees

  2024cites this paper
• Integrative subgroup analysis for high-dimensional mixed-type multi-response data

  2024cites this paper
• Characterization of Inflammatory Bowel Disease Heterogeneity Using Serum Proteomics: A Multicenter Study

  2024influential citation
• Nonconvex Dantzig selector and its parallel computing algorithm

  2024cites this paper
• Analysis of Fertility Determinants and Regional Disparities in Nigeria using Geo-Additive Regression

  2024cites this paper
• Robust variable selection methods with Cox model—a selective practical benchmark study

  2024influential citation
• Joint Design of Doppler Resilient Unimodular Discrete Phase Sequence Waveform and Receiving Filter for Multichannel Radar

  2024cites this paper
• Uncertainty quantification in high-dimensional linear models incorporating graphical structures with applications to gene set analysis

  2024cites this paper
• A high-dimensional omnibus test for set-based association analysis

  2024cites this paper
• Circulating ECM proteins decorin and alpha-L-iduronidase differentiate ATTRwt-CM from ATTRwt-negative HFpEF/HFmrEF

  2024cites this paper
• Can a Transparent Machine Learning Algorithm Predict Better than Its Black Box Counterparts? A Benchmarking Study Using 110 Data Sets

  2024cites this paper
• Machine Learning Methods in Classification of Prolonged Radiation Therapy in Oropharyngeal Cancer: National Cancer Database

  2024cites this paper
• Transcriptional regulation of transcription factor genes WRI1 and LAFL during Brassica napus seed development.

  2024cites this paper
• Variable selection and prediction performance of penalized two-part regression with community-based crime data application

  2024cites this paper
• Identification and validation of a blood- based diagnostic lipidomic signature of pediatric inflammatory bowel disease

  2024cites this paper
• A Greedy Randomized Coordinate Descent Method for Solving Large Linear Least Squares Problem

  2024cites this paper
• Advanced Algorithm for Parameters Estimation of Negative Binomial Distribution with High Dimensional Sparse Group Structure

  2024cites this paper
• Variable importance based interaction modelling with an application on initial spread of COVID-19 in China

  2024cites this paper
• Child life specialists predict successful MRI scanning in unsedated children 4 to 12 years old

  2024cites this paper
• HighDimMixedModels.jl: Robust High Dimensional Mixed Models across Omics Data

  2024cites this paper
• Methods for online calibration of Q-matrix and item parameters for polytomous responses in cognitive diagnostic computerized adaptive testing

  2024cites this paper
• Sparse Group Penalties for bi‐level variable selection

  2024cites this paper
• On high-dimensional classification by sparse generalized Bayesian logistic regression

  2024cites this paper
• Semiparametric Probit Regression Model with General Interval-Censored Failure Time Data

  2024influential citation
• Group Penalized Multinomial Logit Models and Stock Return Direction Prediction

  2024cites this paper
• MoST: model specification test by variable selection stability

  2024cites this paper
• fsemipar: an R package for SoF semiparametric regression

  2024cites this paper
• Dynamic Incremental Optimization for Best Subset Selection

  2024influential citation
• Biomarker and transcriptomics profiles of serum selenium concentrations in patients with heart failure are associated with immunoregulatory processes

  2024cites this paper
• A non-negative spike-and-slab lasso generalized linear stacking prediction modeling method for high-dimensional omics data

  2024cites this paper
• Using Regularized Methods to Validate Q-Matrix in Cognitive Diagnostic Assessment

  2024cites this paper
• PDAS: a Newton-type method for L0\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$L_0$$\end{document} regularized accel

  2024cites this paper
• Unified Transfer Learning in High-Dimensional Linear Regression

  2024cites this paper
• High-dimensional partial correlation coefficients: A survey study of estimation Methods

  2024cites this paper
• Using Regularization to Identify Measurement Bias Across Multiple Background Characteristics: A Penalized Expectation–Maximization Algorithm

  2024cites this paper
• Carbon trading price forecasting in digitalization social change era using an explainable machine learning approach: The case of China as emerging country evidence

  2024cites this paper
• Avoiding strict saddle points of nonconvex regularized problems

  2024cites this paper
• Personalized prediction of survival rate with combination of penalized Cox models in patients with colorectal cancer

  2024cites this paper
• On the Identifiability of Sparse ICA without Assuming Non-Gaussianity

  2024cites this paper
• Estimation of l0 norm penalized models: A statistical treatment

  2024cites this paper