Accelerated gradient boosting

Gradient tree boosting is a prediction algorithm that sequentially produces a model in the form of linear combinations of decision trees, by solving an infinite-dimensional optimization problem. We combine gradient boosting and Nesterov’s accelerated descent to design a new algorithm, which we call AGB (for Accelerated Gradient Boosting). Substantial numerical evidence is provided on both synthetic and real-life data sets to assess the excellent performance of the method in a large variety of prediction problems. It is empirically shown that AGB is less sensitive to the shrinkage parameter and outputs predictors that are considerably more sparse in the number of trees, while retaining the exceptional performance of gradient boosting.

• Publication year

  2018

• Venue

  Machine-mediated learning

• Publication date

  2018-03-06

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1007/s10994-019-05787-1arXiv 1803.02042
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Accelerated Distributed Nesterov Gradient Descent

  2017cited by this paper
• Optimization by gradient boosting

  2017influential reference
• Accelerating Stochastic Gradient Descent

  2017cited by this paper
• XGBoost: A Scalable Tree Boosting System

  2016cited by this paper
• On the Complexity of Best-Arm Identification in Multi-Armed Bandit Models

  2014cited by this paper
• Convex Optimization: Algorithms and Complexity

  2014cited by this paper
• Introductory Lectures on Convex Optimization - A Basic Course

  2014cited by this paper
• A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights

  2014cited by this paper
• On the importance of initialization and momentum in deep learning

  2013cited by this paper
• COBRA: A combined regression strategy

  2013cited by this paper
• First-order methods of smooth convex optimization with inexact oracle

  2013cited by this paper
• Gradient methods for minimizing composite functions

  2012cited by this paper
• NESTA: A Fast and Accurate First-Order Method for Sparse Recovery

  2009cited by this paper
• A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems

  2009cited by this paper
• The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition by Trevor Hastie, Robert Tibshirani, Jerome Friedman

  2009cited by this paper
• Boosting Algorithms: Regularization, Prediction and Model Fitting. Rejoinder.

  2007cited by this paper
• Generalized Boosted Models: A guide to the gbm package

  2006cited by this paper
• Some Theory for Generalized Boosting Algorithms

  2006cited by this paper
• AdaBoost is Consistent

  2006cited by this paper
• Smooth minimization of non-smooth functions

  2005cited by this paper
• Boosting with early stopping: Convergence and consistency

  2005cited by this paper
• The elements of statistical learning: data mining, inference and prediction

  2005cited by this paper
• Complexity regularization via localized random penalties

  2004cited by this paper
• Boosting With the L2 Loss

  2003cited by this paper
• On the Rate of Convergence of Regularized Boosting Classifiers

  2003cited by this paper
• Population theory for boosting ensembles

  2003cited by this paper
• On the Bayes-risk consistency of regularized boosting methods

  2003cited by this paper
• Stochastic gradient boosting

  2002cited by this paper
• The Elements of Statistical Learning: Data Mining, Inference, and Prediction

  2001cited by this paper
• Boosting with the L_2-Loss: Regression and Classification

  2001cited by this paper
• Greedy function approximation: A gradient boosting machine.

  2001influential reference
• Random Forests

  2001cited by this paper
• Special Invited Paper-Additive logistic regression: A statistical view of boosting

  2000cited by this paper
• Functional Gradient Techniques for Combining Hypotheses

  2000cited by this paper
• SOME INFINITY THEORY FOR PREDICTOR ENSEMBLES

  2000cited by this paper
• Boosting Algorithms as Gradient Descent

  1999cited by this paper
• Prediction Games and Arcing Algorithms

  1999cited by this paper
• Arcing classifier (with discussion and a rejoinder by the author)

  1998cited by this paper
• ARCING THE EDGE

  1997cited by this paper
• A decision-theoretic generalization of on-line learning and an application to boosting

  1997cited by this paper
• Experiments with a New Boosting Algorithm

  1996cited by this paper
• Regression Shrinkage and Selection via the Lasso

  1996influential reference
• A Probabilistic Theory of Pattern Recognition

  1996cited by this paper
• Boosting a weak learning algorithm by majority

  1990cited by this paper
• The Strength of Weak Learnability

  1989cited by this paper
• A method for solving the convex programming problem with convergence rate O(1/k^2)

  1983cited by this paper

• Highly sensitive differential bilayer (BL)-MoS$$_2$$ MEMS piezoresistive pressure sensor: multi-domain design and simulation with ML-based performance analysis

  2026cites this paper
• New ideas to the design of algorithms based on derivatives

  2026cites this paper
• Insight into machine learning models to predict toxicity of organophosphorus insecticides to Photobacterium phosphoreum based on a small dataset

  2026cites this paper
• Evaluasi Ensemble Learning untuk Prediksi Nilai Matematika Siswa Sekolah Menengah

  2026cites this paper
• Ensemble learning for air quality index prediction: integrating gradient boosting, XGBoost, and stacking with SHAP-based interpretability.

  2026cites this paper
• Assessment of pro-environmental awareness and washing practices towards microfiber pollution from textile laundering in India

  2026cites this paper
• Comparison of The Performance of Machine Learning Methods for Solar Energy Power Prediction

  2025cites this paper
• Epitaxy of Emerging Materials and Advanced Heterostructures for Microelectronics and Quantum Sciences

  2025cites this paper
• Enhancing explainability in AI-based energy forecasting through clustering and data selection

  2025cites this paper
• Early detection and classification of bone marrow changes in lumbar vertebrae using machine learning techniques

  2025cites this paper
• Artificial intelligence based malignant lymphoma type prediction using enhanced super resolution image and hybrid feature extraction algorithm

  2025cites this paper
• Dijital bankacılık ve enflasyon ilişkisinin makine öğrenme algoritmaları ile tespiti

  2025cites this paper
• High-throughput screening anisotropic negative thermal expansion framework materials

  2025cites this paper
• Highly Efficient Screening of Halide Double Perovskite Optoelectronic Materials Based on Machine learning.

  2025cites this paper
• Predictive modeling of ultimate tensile strength in dissimilar friction stir welded aluminum alloys via machine learning approach

  2025cites this paper
• Foundations and Innovations in Data Fusion and Ensemble Learning for Effective Consensus

  2025cites this paper
• Predicting dry matter intake in cattle at scale using gradient boosting regression techniques and Gaussian process boosting regression with Shapley additive explanation explainable artificial intelligence, MLflow, and its containerization

  2025cites this paper
• Integrating neuroscience and artificial intelligence: EEG analysis using ensemble learning for diagnosis Alzheimer's disease and frontotemporal dementia.

  2025cites this paper
• Online Gradient Boosting Decision Tree: In-Place Updates for Efficient Adding/Deleting Data

  2025cites this paper
• Synthesizing Local Capacities, Multi-Source Remote Sensing and Meta-Learning to Optimize Forest Carbon Assessment in Data-Poor Regions

  2025cites this paper
• Accelerated Development of Vapor Deposition Technology for Efficient Perovskite Solar Cells via Accurate and Practical Machine Learning Tools

  2025cites this paper
• Insulator detection in transmission line based on Log AdaBoost

  2025cites this paper
• Wave Run-Up Distance Prediction Combined Data-Driven Method and Physical Experiments

  2025cites this paper
• Loose particle location method for sealed electronic equipment based on multi-domain joint features

  2025cites this paper
• Applying Machine Learning to Predict Software Defects

  2024cites this paper
• Adaptive Learning Rates for Gradient Boosting Machines

  2024cites this paper
• Early Detection of Coronary Heart Disease Using Exploration Data Analysis and Gradient Boosting Method

  2024cites this paper
• Predicting permeability in sandstone reservoirs from mercury injection capillary pressure data using advanced machine learning algorithms

  2024cites this paper
• Intelligent Troubleshooting of Vertical Bandsaws, Leveraging Ensemble Learning on Low-Level Data

  2024cites this paper
• Data-driven interpretation, comparison and optimization of hydrogen production from supercritical water gasification of biomass and polymer waste: Applying ensemble and differential evolution in machine learning algorithms

  2024cites this paper
• From data to harvest: Leveraging ensemble machine learning for enhanced crop yield predictions across Canada amidst climate change.

  2024cites this paper
• A random feature mapping method based on the AdaBoost algorithm and results fusion for enhancing classification performance

  2024cites this paper
• Enhancing Classification and Prediction through the Application of Hybrid Machine Learning Models

  2024cites this paper
• How to Boost Any Loss Function

  2024cites this paper
• Continuous Valence-Arousal Space Prediction and Recognition Based on Feature Fusion

  2024cites this paper
• An Ensemble Classifiers for Improved Prediction of Native–Non-Native Protein–Protein Interaction

  2024cites this paper
• Data-driven prediction of natural period for existing RC high-rise buildings using probabilistic machine learning methods

  2024cites this paper
• Application of Reinforcement Learning to Dyeing Processes for Residual Dye Reduction

  2024cites this paper
• Bond Strength between Recycled Aggregate Concrete and Rebar: Interpretable Machine Learning Modeling Approach for Performance Estimation and Engineering Design

  2024cites this paper
• Crack detection based on mel-frequency cepstral coefficients features using multiple classifiers

  2024cites this paper
• Estimation of vehicle control delay using artificial intelligence techniques for heterogeneous traffic conditions

  2024cites this paper
• Predictions of oil volume in palm fruit and estimates of their ripeness: A comparative study of machine learning algorithms

  2024cites this paper
• Kp Index Forecast Model Based on GBR Method

  2024cites this paper
• Hyper-parameter tuned yearly and seasonal models for aggregated short-term electricity demand prediction

  2024cites this paper
• Evaluation of feature selection performance for identification of best effective technical indicators on stock market price prediction

  2023cites this paper
• Estimating Daily NO2 Ground Level Concentrations Using Sentinel-5P and Ground Sensor Meteorological Measurements

  2023cites this paper
• Daily scale streamflow forecasting in multiple stream orders of Cauvery River, India: Application of advanced ensemble and deep learning models

  2023cites this paper
• 2-Copula and Simple Criterion based Algorithm for Identification of Time Delay Dynamic Systems

  2023cites this paper
• Machine Learning-Based Prediction of Flight Punctuality and Delays

  2023cites this paper
• Machine Learning for COVID-19 and Influenza Classification during Coexisting Outbreaks

  2023cites this paper
• Model of Storm Surge Maximum Water Level Increase in a Coastal Area Using Ensemble Machine Learning and Explicable Algorithm

  2023cites this paper
• Regression Forecast Model of 60-Meter Band Channel Performance Index Over Kupang Area

  2023cites this paper
• Exploring the Power of eXtreme Gradient Boosting Algorithm in Machine Learning: a Review

  2023cites this paper
• Gradient boosting Bayesian neural networks via Langevin MCMC

  2023cites this paper
• Early Identification of Risk Factors in Non-Alcoholic Fatty Liver Disease (NAFLD) Using Machine Learning

  2023cites this paper
• Predicting the occurrence of Ischemic stroke by Gradient Boost Approaches

  2023cites this paper
• Labor Accidents Classification Using Machine Learning

  2023cites this paper
• Data-driven collapse strength modelling for the screen pipes with internal corrosion defect based on finite element analysis and tree-based machine learning

  2023cites this paper
• Small data machine learning in materials science

  2023cites this paper
• Machine Learning-Based Lung Cancer Diagnosis

  2022cites this paper
• Output prediction of alpha-type Stirling engines using gradient boosted regression trees and corresponding heat recovery system optimization based on improved NSGA-II

  2022cites this paper
• Regression Machine Learning Models Used to Predict DFT-Computed NMR Parameters of Zeolites

  2022cites this paper
• Machine-Learning-Based DDoS Attack Detection Using Mutual Information and Random Forest Feature Importance Method

  2022cites this paper
• Random Forest as a Method of Predicting the Presence of Cardiovasculars Diseases

  2022cites this paper
• Validation of ERA5-Land temperature and relative humidity on four Peruvian glaciers using on-glacier observations

  2022cites this paper
• Airline ticket price-prediction model based on integrated feature extraction

  2022cites this paper
• TRBoost: a generic gradient boosting machine based on trust-region method

  2022cites this paper
• Early Detection of Parkinson's Disease using Spiral Test and Echo State Networks

  2022cites this paper
• Non-genital warts response to Cryotherapy and Immunotherapy treatment using Gradient Boosting Prediction Models

  2022cites this paper
• Transformer fault diagnosis research based on LIF technology and IAO optimization of LightGBM.

  2022cites this paper
• An IoT and Fuzzy aware e-Healthcare system using feature optimization tuned T-CNN with high dimensional data

  2022cites this paper
• A PSO-GBR Solution for Association Rule Optimization on Supermarket Sales

  2022cites this paper
• Predicting the Risk of & Time to Impairment for ALS patients

  2022cites this paper
• Solar Proton Events Short-time Forecasting Based on Ensemble Learning

  2022cites this paper
• Feature Subset Selection for Malware Detection in Smart IoT Platforms

  2021cites this paper
• A Novel Bayesian Optimization-Based Machine Learning Framework for COVID-19 Detection From Inpatient Facility Data

  2021cites this paper
• A new accelerated proximal boosting machine with convergence rate O(1/t2)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document

  2021cites this paper
• An Intelligent Multi-output Regression Model for Soil Moisture Prediction

  2021cites this paper
• Thyroid Disease Treatment prediction with machine learning approaches

  2021cites this paper
• A Framework for Predicting Placement of a Graduate Using Machine Learning Techniques

  2021cites this paper
• AUTOMATIC CHANGE DETECTION OF DIGITAL MAPS USING AERIAL IMAGES AND POINT CLOUDS

  2021cites this paper
• Bagging and Boosting Ensemble Classifiers for Classification of Multispectral, Hyperspectral and PolSAR Data: A Comparative Evaluation

  2021cites this paper
• Identifying the Main Risk Factors for Cardiovascular Diseases Prediction Using Machine Learning Algorithms

  2021cites this paper
• Classification Using Decision Tree Ensembles

  2021cites this paper
• A Comparison of Bagging and Boosting on Classification Data: Case Study on Rainfall Data in Sultan Syarif Kasim II Meteorological Station in Pekanbaru

  2021cites this paper
• Using Machine Learning Techniques to Detect Defects in Images of Metal Structures

  2021cites this paper
• Comprehending the Chaos Leveraging Text to Improve Analysis

  2021cites this paper
• Identifying the Main Risk Factors for CVD Prediction Using Machine Learning Algorithms

  2021cites this paper
• A Novel Approach for Data Analysis using Explainable AI

  2021cites this paper
• Proximal boosting and variants

  2021influential citation
• Application of gradient tree boosting regressor for the prediction of scour depth around bridge piers

  2021cites this paper
• FusionCL: a machine-learning based approach for OpenCL kernel fusion to increase system performance

  2021cites this paper
• Remote Sensing Based Yield Estimation of Rice (Oryza Sativa L.) Using Gradient Boosted Regression in India

  2021influential citation
• GBHT: Gradient Boosting Histogram Transform for Density Estimation

  2021cites this paper
• Gradient Boosted Binary Histogram Ensemble for Large-scale Regression

  2021cites this paper
• Latent Gaussian Model Boosting

  2021cites this paper
• Resilient modulus descriptive analysis and estimation for fine-grained soils using multivariate and machine learning methods

  2021cites this paper
• Classification Models for COVID-19 Test Prioritization in Brazil: Machine Learning Approach

  2021cites this paper
• Predicting Demand for Air Taxi Urban Aviation Services using Machine Learning Algorithms

  2021cites this paper
• Supercomputing Frontiers: 6th Asian Conference, SCFA 2020, Singapore, February 24–27, 2020, Proceedings

  2020cites this paper