Gradient boosting for distributional regression: faster tuning and improved variable selection via noncyclical updates

We present a new algorithm for boosting generalized additive models for location, scale and shape (GAMLSS) that allows to incorporate stability selection, an increasingly popular way to obtain stable sets of covariates while controlling the per-family error rate. The model is fitted repeatedly to subsampled data, and variables with high selection frequencies are extracted. To apply stability selection to boosted GAMLSS, we develop a new “noncyclical” fitting algorithm that incorporates an additional selection step of the best-fitting distribution parameter in each iteration. This new algorithm has the additional advantage that optimizing the tuning parameters of boosting is reduced from a multi-dimensional to a one-dimensional problem with vastly decreased complexity. The performance of the novel algorithm is evaluated in an extensive simulation study. We apply this new algorithm to a study to estimate abundance of common eider in Massachusetts, USA, featuring excess zeros, overdispersion, nonlinearity and spatiotemporal structures. Eider abundance is estimated via boosted GAMLSS, allowing both mean and overdispersion to be regressed on covariates. Stability selection is used to obtain a sparse set of stable predictors.

• Publication year

  2016

• Venue

  Statistics and computing

• Publication date

  2016-02-22

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1007/s11222-017-9754-6arXiv 1611.10171
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Spatiotemporal modelling of sea duck abundance: implications for marine spatial planning

  2017cited by this paper
• Non-homogeneous boosting for predictor selection in ensemble post-processing

  2017cited by this paper
• Boosting the discriminatory power of sparse survival models via optimization of the concordance index and stability selection

  2016cited by this paper
• Stability Selection with Error Control

  2015cited by this paper
• Model-Based Boosting

  2015cited by this paper
• gamboostLSS: An R Package for Model Building and Variable Selection in the GAMLSS Framework

  2014cited by this paper
• Model selection for ecologists: the worldviews of AIC and BIC.

  2014cited by this paper
• Extending Statistical Boosting

  2014cited by this paper
• Discussion of “The Evolution of Boosting Algorithms” and “Extending Statistical Boosting”

  2014cited by this paper
• Controlling false discoveries in high-dimensional situations: boosting with stability selection

  2014influential reference
• The Evolution of Boosting Algorithms

  2014cited by this paper
• Collinearity: a review of methods to deal with it and a simulation study evaluating their performance

  2013cited by this paper
• The Importance of Knowing When to Stop

  2012cited by this paper
• Model-based boosting in R: a hands-on tutorial using the R package mboost

  2012cited by this paper
• Gene network-based cancer prognosis analysis with sparse boosting.

  2012cited by this paper
• Generalized additive models for location, scale and shape for high dimensional data—a flexible approach based on boosting

  2012influential reference
• Non-stationary analysis of dry spells in monsoon season of Senegal River Basin using data from Regional Climate Models (RCMs)

  2012cited by this paper
• Variable selection with error control: another look at stability selection

  2011influential reference
• Decomposing environmental, spatial, and spatiotemporal components of species distributions

  2011cited by this paper
• A Framework for Unbiased Model Selection Based on Boosting

  2011cited by this paper
• Estimation and regularization techniques for regression models with multidimensional prediction functions

  2010cited by this paper
• Model-based Boosting 2.0

  2010cited by this paper
• Twin Boosting: improved feature selection and prediction

  2010cited by this paper
• Robust LogitBoost and Adaptive Base Class (ABC) LogitBoost

  2010cited by this paper
• Performance of several variable-selection methods applied to real ecological data.

  2009cited by this paper
• Boosting additive models using component-wise P-Splines

  2008cited by this paper
• Stability selection

  2008cited by this paper
• Boosting Algorithms: Regularization, Prediction and Model Fitting. Rejoinder.

  2007cited by this paper
• Sparse Boosting

  2006cited by this paper
• Generalized additive models for location, scale and shape

  2005influential reference
• Weak Hypotheses and Boosting for Generic Object Detection and Recognition

  2004cited by this paper
• Boosting With the L2 Loss

  2003cited by this paper
• Avoiding pitfalls when using information-theoretic methods

  2002cited by this paper
• Boosting with the L_2-Loss: Regression and Classification

  2001cited by this paper
• Special Invited Paper-Additive logistic regression: A statistical view of boosting

  2000cited by this paper
• Frequency of Selecting Noise Variables in Subset Regression Analysis: A Simulation Study

  1987cited by this paper
• Specification and testing of some modified count data models

  1986cited by this paper
• Generalized Additive Models

  1986cited by this paper

• Bivariate Postprocessing of Wind Vectors

  2026cites this paper
• Estimating Zero-inflated Negative Binomial GAMLSS via a Balanced Gradient Boosting Approach with an Application to Antenatal Care Data from Nigeria

  2026influential citation
• Modelling dependent censoring in time-to-event data using boosting copula regression

  2025cites this paper
• Comparative Ungulate Diversity and Biomass Change With Human Use and Drought: Implications for Community Stability and Protected Area Prioritization in African Savannas

  2025cites this paper
• A comparative analysis of generalized additive models for obesity risk prediction

  2025cites this paper
• Short-Term Forecasting of the JSE All-Share Index Using Gradient Boosting Machines

  2025cites this paper
• Gradient boosting for Dirichlet regression models

  2025influential citation
• AdaPRL: Adaptive Pairwise Regression Learning with Uncertainty Estimation for Universal Regression Tasks

  2025cites this paper
• D-vine Generalized Additive Model copula-based quantile regression with application to ensemble postprocessing

  2025cites this paper
• Improving Large‐Scale Population Estimates and Assessments of the Ecological Importance of Three Epifaunal Bivalve Species by Combining Distribution and Abundance Models

  2025influential citation
• A balanced statistical boosting approach for GAMLSS via new step lengths

  2024influential citation
• Gradient-Boosted Mixture Regression Models for Postprocessing Ensemble Weather Forecasts

  2024cites this paper
• Simple or complex statistical models: Non-traditional regression models with intuitive interpretations

  2024cites this paper
• Gradient‐Boosted Generalized Linear Models for Conditional Vine Copulas

  2024cites this paper
• Enhanced Variable Selection for Boosting Sparser and Less Complex Models in Distributional Copula Regression

  2024influential citation
• Boosting distributional copula regression for bivariate binary, discrete and mixed responses

  2024cites this paper
• Variable Selection and Allocation in Joint Models via Gradient Boosting Techniques

  2023cites this paper
• Robust gradient boosting for generalized additive models for location, scale and shape

  2023cites this paper
• Linear or smooth? Enhanced model choice in boosting via deselection of base-learners

  2023cites this paper
• Predicting Li-Ion Battery Remaining Useful Life: An XDFM-Driven Approach with Explainable AI

  2023cites this paper
• Explaining Predictive Uncertainty with Information Theoretic Shapley Values

  2023cites this paper
• Developing prediction models for symptom severity around the time of discharge from a tertiary-care program for treatment-resistant psychosis

  2023cites this paper
• The high-dimensional space of human diseases built from diagnosis records and mapped to genetic loci

  2023cites this paper
• Real-time prediction of multivariate ROP (rate of penetration) based on machine learning regression algorithms: Algorithm comparison, model evaluation and parameter analysis

  2023cites this paper
• Modeling Postoperative Mortality in Older Patients by Boosting Discrete-Time Competing Risks Models

  2023cites this paper
• A Note on Multi-Parametric Gradient Boosting Machines with Non-Life Insurance Applications

  2023cites this paper
• Scalable Estimation for Structured Additive Distributional Regression

  2023cites this paper
• Is it all bafflegab? - Linguistic and meta characteristics of research articles in prestigious economics journals

  2022cites this paper
• Adaptive step-length selection in gradient boosting for Gaussian location and scale models

  2022cites this paper
• Loss-guided stability selection

  2022cites this paper
• Boosting distributional copula regression

  2022cites this paper
• A boosting first-hitting-time model for survival analysis in high-dimensional settings

  2022cites this paper
• Boosting multivariate structured additive distributional regression models

  2022cites this paper
• On the role of benchmarking data sets and simulations in method comparison studies

  2022cites this paper
• Robust statistical boosting with quantile-based adaptive loss functions

  2022cites this paper
• Forecasting overhead distribution line failures using weather data and gradient-boosted location, scale, and shape models

  2022cites this paper
• A review of predictive uncertainty estimation with machine learning

  2022cites this paper
• Distributional regression modeling via generalized additive models for location, scale, and shape: An overview through a data set from learning analytics

  2022cites this paper
• Additive Covariance Matrix Models: Modeling Regional Electricity Net-Demand in Great Britain

  2022cites this paper
• A CASE STUDY ON ANIMAL BEHAVIOR ANALYSIS USING GAMLSS

  2021cites this paper
• Trimming stability selection increases variable selection robustness

  2021cites this paper
• Functional Additive Models on Manifolds of Planar Shapes and Forms

  2021cites this paper
• Deselection of base-learners for statistical boosting—with an application to distributional regression

  2021cites this paper
• Trends in torrential flooding in the Austrian Alps: A combination of climate change, exposure dynamics, and mitigation measures

  2021cites this paper
• Adaptive Step-Length Selection in Gradient Boosting for Generalized Additive Models for Location, Scale and Shape

  2021influential citation
• Functional additive regression on shape and form manifolds of planar curves

  2021cites this paper
• Delta Boosting Implementation of Negative Binomial Regression in Actuarial Pricing

  2020cites this paper
• Sensitivity analysis of Bayesian networks to parameters of the conditional probability model using a Beta regression approach

  2020influential citation
• CatBoostLSS - An extension of CatBoost to probabilistic forecasting

  2020cites this paper
• Prediction of COVID-19 Confirmed Cases Using Gradient Boosting Regression Method

  2020cites this paper
• Random boosting and random^2 forests - A random tree depth injection approach

  2020cites this paper
• Distributional (Single) Index Models

  2020cites this paper
• Algorithms for forecasting cotton yield based on climatic parameters in Brazil

  2020cites this paper
• Approaches to Identify Relevant Process Variables in Injection Moulding using Beta Regression and SVM

  2019cites this paper
• Significance Tests for Boosted Location and Scale Models with Linear Base-Learners

  2019cites this paper
• LASSO-type penalization in the framework of generalized additive models for location, scale and shape

  2019cites this paper
• Estimating heritability and genetic correlations from large health datasets in the absence of genetic data

  2019cites this paper
• Flexible modeling of ratio outcomes in clinical and epidemiological research

  2019cites this paper
• XGBoostLSS - An extension of XGBoost to probabilistic forecasting

  2019cites this paper
• Combining clinical and molecular data in regression prediction models: insights from a simulation study

  2019cites this paper
• NWP-based lightning prediction using flexible count data regression

  2019cites this paper
• A boosting algorithm to extend first-hitting-time models to a high-dimensional survival setting

  2019influential citation
• Modeling spatiotemporal abundance of mobile wildlife in highly variable environments using boosted GAMLSS hurdle models

  2019cites this paper
• Boosting functional response models for location, scale and shape with an application to bacterial competition

  2018cites this paper
• Extension of the Gradient Boosting Algorithm for Joint Modeling of Longitudinal and Time-to-Event data

  2018cites this paper
• The betaboost package-a software tool for modelling bounded outcome variables in potentially high-dimensional epidemiological data.

  2018cites this paper
• Adaptive Step Length Selection in Gradient Boosting for Generalized Additive Models for Location, Scale and Shape

  2018cites this paper
• Estimation, model choice and subsequent inference: methods for additive and functional regression models

  2018cites this paper
• Gradient boosting in Markov-switching generalized additive models for location, scale, and shape

  2017cites this paper
• Pruning variable selection ensembles

  2017cites this paper
• Boosting Functional Regression Models with FDboost

  2017cites this paper
• An Update on Statistical Boosting in Biomedicine

  2017cites this paper
• Boosting factor‐specific functional historical models for the detection of synchronization in bioelectrical signals

  2016influential citation