On Theory for BART

Ensemble learning is a statistical paradigm built on the premise that many weak learners can perform exceptionally well when deployed collectively. The BART method of Chipman et al. (2010) is a prominent example of Bayesian ensemble learning, where each learner is a tree. Due to its impressive performance, BART has received a lot of attention from practitioners. Despite its wide popularity, however, theoretical studies of BART have begun emerging only very recently. Laying the foundations for the theoretical analysis of Bayesian forests, Rockova and van der Pas (2017) showed optimal posterior concentration under conditionally uniform tree priors. These priors deviate from the actual priors implemented in BART. Here, we study the exact BART prior and propose a simple modification so that it also enjoys optimality properties. To this end, we dive into branching process theory. We obtain tail bounds for the distribution of total progeny under heterogeneous Galton-Watson (GW) processes exploiting their connection to random walks. We conclude with a result stating the optimal rate of posterior convergence for BART.

• Publication year

  2018

• Venue

  International Conference on Artificial Intelligence and Statistics

• Publication date

  2018-10-01

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv 1810.00787
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Posterior Concentration for Sparse Deep Learning

  2018cited by this paper
• Bayesian Dyadic Trees and Histograms for Regression

  2017cited by this paper
• Pólya tree posterior distributions on densities

  2017cited by this paper
• Bayesian regression tree ensembles that adapt to smoothness and sparsity

  2017cited by this paper
• Posterior Concentration for Bayesian Regression Trees and their Ensembles

  2017cited by this paper
• Heteroscedastic BART Using Multiplicative Regression Trees

  2017cited by this paper
• Bayesian regression tree models for causal inference: regularization, confounding, and heterogeneous effects

  2017cited by this paper
• High-dimensional nonparametric monotone function estimation using BART

  2016cited by this paper
• Adaptive Concentration of Regression Trees, with Application to Random Forests

  2015cited by this paper
• Mondrian Forests: Efficient Online Random Forests

  2014cited by this paper
• Consistency of Random Forests

  2014cited by this paper
• Top-down particle filtering for Bayesian decision trees

  2013cited by this paper
• Variable selection for BART: An application to gene regulation

  2013influential reference
• Semiparametric Bernstein-von Mises for the error standard deviation

  2013cited by this paper
• Classification and regression trees

  2012cited by this paper
• Bayesian Nonparametric Modeling for Causal Inference

  2011cited by this paper
• Dynamic Trees for Learning and Design

  2009cited by this paper
• Which Spatial Partition Trees are Adaptive to Intrinsic Dimension?

  2009cited by this paper
• Lower bounds for posterior rates with Gaussian process priors

  2008cited by this paper
• Rates of contraction of posterior distributions based on Gaussian process priors

  2008cited by this paper
• BART: Bayesian Additive Regression Trees

  2008influential reference
• The Mondrian Process

  2008cited by this paper
• Consistency of Random Forests and Other Averaging Classifiers

  2008cited by this paper
• Bayesian Treed Gaussian Process Models With an Application to Computer Modeling

  2007cited by this paper
• Convergence rates of posterior distributions for non-i.i.d. observations

  2007cited by this paper
• Consistency of Bayes estimators of a binary regression function

  2004influential reference
• Bayesian CART Model Search

  1998influential reference
• A Bayesian CART algorithm

  1998cited by this paper
• CART AND BEST-ORTHO-BASIS: A CONNECTION'

  1997cited by this paper
• An Analysis of Bayesian Inference for Nonparametric Regression

  1993cited by this paper
• On the consistency of Bayes estimates

  1986cited by this paper
• Classification and Regression Trees (Wadsworth Statistics/Probability)

  1984influential reference
• Almost surely consistent nonparametric regression from recursive partitioning schemes

  1984cited by this paper
• Consistent nonparametric regression from recursive partitioning schemes

  1980cited by this paper
• On the extinction times of varying and random environment branching processes

  1975influential reference
• The total progeny in a branching process and a related random walk

  1969cited by this paper

• Bayesian Additive Regression Tree Copula Processes for Scalable Distributional Prediction

  2026influential citation
• BFTS: Thompson Sampling with Bayesian Additive Regression Trees

  2026influential citation
• Functional BART with Shape Priors: A Bayesian Tree Approach to Constrained Functional Regression

  2025cites this paper
• Bayesian Doubly Robust Causal Inference via Posterior Coupling

  2025cites this paper
• How do the professional players select their shot locations? An analysis of Field Goal Attempts via Bayesian Additive Regression Trees

  2025cites this paper
• Robust Semiparametric Inference for Bayesian Additive Regression Trees

  2025cites this paper
• Horseshoe Forests for High-Dimensional Causal Survival Analysis

  2025cites this paper
• Generative Regression with IQ-BART

  2025cites this paper
• Leveraging a neutrophil-derived PCD signature to predict and stratify patients with acute myocardial infarction: from AI prediction to biological interpretation

  2024influential citation
• Seemingly unrelated Bayesian additive regression trees for cost-effectiveness analyses in healthcare

  2024cites this paper
• Loss-based prior for tree topologies in BART models

  2024cites this paper
• Ranking Perspective for Tree-based Methods with Applications to Symbolic Feature Selection

  2024influential citation
• Scalable piecewise smoothing with BART

  2024influential citation
• On mixing rates for Bayesian CART

  2023cites this paper
• Bayesian Decision Trees Inspired from Evolutionary Algorithms

  2023cites this paper
• Parallel Approaches to Accelerate Bayesian Decision Trees

  2023cites this paper
• Theory of Posterior Concentration for Generalized Bayesian Additive Regression Trees

  2023influential citation
• Improving transportability of randomized controlled trial inference using robust prediction methods

  2023cites this paper
• Extrapolated Random Tree for Regression

  2023cites this paper
• Sharded Bayesian Additive Regression Trees

  2023influential citation
• Hierarchical Embedded Bayesian Additive Regression Trees

  2022cites this paper
• A Bayesian Survival Tree Partition Model Using Latent Gaussian Processes

  2022cites this paper
• Single MCMC chain parallelisation on decision trees

  2022cites this paper
• A Mixing Time Lower Bound for a Simplified Version of BART

  2022cites this paper
• Bayesian additive regression trees for probabilistic programming

  2022cites this paper
• BAST: Bayesian Additive Regression Spanning Trees for Complex Constrained Domain

  2021cites this paper
• A Bayesian Contiguous Partitioning Method for Learning Clustered Latent Variables

  2021cites this paper
• Bridging Breiman's Brook: From Algorithmic Modeling to Statistical Learning

  2021cites this paper
• Tail Forecasting with Multivariate Bayesian Additive Regression Trees

  2021cites this paper
• Bayesian Additive Regression Trees for Genotype by Environment Interaction Models

  2021cites this paper
• Ideal Bayesian Spatial Adaptation

  2021influential citation
• BNPqte: A Bayesian Nonparametric Approach to Causal Inference on Quantiles in R

  2021influential citation
• Semi-parametric Bayesian Additive Regression Trees

  2021cites this paper
• Adaptive Bayesian sum of trees model for covariate‐dependent spectral analysis

  2021cites this paper
• Shrinkage Bayesian Causal Forests for Heterogeneous Treatment Effects Estimation

  2021cites this paper
• Accounting for shared covariates in semiparametric Bayesian additive regression trees

  2021cites this paper
• The Art of BART: Minimax Optimality over Nonhomogeneous Smoothness in High Dimension

  2020influential citation
• Stochastic Tree Search for Estimating Optimal Dynamic Treatment Regimes

  2020cites this paper
• Smoothing and adaptation of shifted Pólya tree ensembles

  2020cites this paper
• The art of BART: On flexibility of Bayesian forests

  2020influential citation
• On Semi-parametric Inference for BART

  2020influential citation
• Bayesian additive regression trees with model trees

  2020cites this paper
• Bayesian Probabilistic Numerical Integration with Tree-Based Models

  2020cites this paper
• VCBART: Bayesian Trees for Varying Coefficients

  2020influential citation
• Bayesian Nonparametric Space Partitions: A Survey

  2020cites this paper
• Stochastic Tree Ensembles for Regularized Nonlinear Regression

  2020cites this paper
• Multiscale Analysis of Bayesian Cart

  2019cites this paper
• Best Split Nodes for Regression Trees

  2019cites this paper
• On Semi-parametric Bernstein-von Mises Theorems for BART

  2019cites this paper
• Bayesian additive regression trees and the General BART model

  2019cites this paper
• Uncertainty quantification for Bayesian CART

  2019cites this paper
• Variable Selection with Rigorous Uncertainty Quantification using Deep Bayesian Neural Networks: Posterior Concentration and Bernstein-von Mises Phenomenon

  2019cites this paper
• State-of-the-BART: Simple Bayesian Tree Algorithms for Prediction and Causal Inference

  2019cites this paper
• Analyzing CART.

  2019cites this paper
• Variable selection with ABC Bayesian forests

  2018cites this paper
• Posterior concentration for Bayesian regression trees and forests

  2017cites this paper
• Bayesian regression tree models for causal inference: regularization, confounding, and heterogeneous effects

  2017cites this paper
• Log-Linear Bayesian Additive Regression Trees for Multinomial Logistic and Count Regression Models

  2017cites this paper
• mBART: Multidimensional Monotone BART

  2016cites this paper