Adaptive Bayesian estimation using a Gaussian random field with inverse Gamma bandwidth

We consider nonparametric Bayesian estimation inference using a rescaled smooth Gaussian field as a prior for a multidimensional function. The rescaling is achieved using a Gamma variable and the procedure can be viewed as choosing an inverse Gamma bandwidth. The procedure is studied from a frequentist perspective in three statistical settings involving replicated observations (density estimation, regression and classification). We prove that the resulting posterior distribution shrinks to the distribution that generates the data at a speed which is minimax-optimal up to a logarithmic factor, whatever the regularity level of the data-generating distribution. Thus the hierachical Bayesian procedure, with a fixed prior, is shown to be fully adaptive.

• Publication year

  2009

• Venue

  Annals of Statistics

• Publication date

  2009-08-25

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1214/08-AOS678arXiv 0908.3556
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Reproducing kernel Hilbert spaces of Gaussian priors

  2008cited by this paper
• Rates of contraction of posterior distributions based on Gaussian process priors

  2008cited by this paper
• Nonparametric Bayesian model selection and averaging

  2008cited by this paper
• Alternative posterior consistency results in nonparametric binary regression using Gaussian process priors

  2007cited by this paper
• Posterior consistency of logistic Gaussian process priors in density estimation

  2007cited by this paper
• On universal Bayesian adaptation

  2007cited by this paper
• Bayesian inference with rescaled Gaussian process priors

  2007cited by this paper
• Nonparametric binary regression using a Gaussian process prior

  2007cited by this paper
• Convergence rates of posterior distributions for non-i.i.d. observations

  2007cited by this paper
• Posterior consistency of Gaussian process prior for nonparametric binary regression

  2006cited by this paper
• Convergence rates for posterior distributions and adaptive estimation

  2004cited by this paper
• Posterior Consistency in Nonparametric Regression Problems under Gaussian Process Priors

  2004cited by this paper
• Nonparametric Curve Estimation

  2004cited by this paper
• Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities

  2001cited by this paper
• Asymptotically Local Minimax Estimation of Infinitely Smooth Density with Censored Data

  2001cited by this paper
• Convergence rates of posterior distributions

  2000cited by this paper
• Risk bounds for model selection via penalization

  1999cited by this paper
• Approximation, metric entropy and small ball estimates for Gaussian measures

  1999influential reference
• Adaptive wavelet estimation : A block thresholding and oracle inequality approach

  1999cited by this paper
• Weak Convergence and Empirical Processes: With Applications to Statistics

  1996influential reference
• Density estimation by wavelet thresholding

  1996cited by this paper
• Wavelet Shrinkage: Asymptopia?

  1995cited by this paper
• The Gaussian measure of shifted balls

  1994cited by this paper
• Asymptotically minimax adaptive estimation. II: Schemes without optimal adaptation: adaptive estimators

  1993cited by this paper
• Metric entropy and the small ball problem for Gaussian measures

  1993cited by this paper
• Asymptotically Minimax Adaptive Estimation. I: Upper Bounds. Optimally Adaptive Estimates

  1992cited by this paper
• On a Problem of Adaptive Estimation in Gaussian White Noise

  1991cited by this paper
• Minimum complexity density estimation

  1991cited by this paper
• THE VOLUME OF CONVEX BODIES AND BANACH SPACE GEOMETRY (Cambridge Tracts in Mathematics 94)

  1991cited by this paper
• Towards a practicable Bayesian nonparametric density estimator

  1991cited by this paper
• The volume of convex bodies and Banach space geometry

  1989cited by this paper
• The Logistic Normal Distribution for Bayesian, Nonparametric, Predictive Densities

  1988cited by this paper
• Dualité des nombres d'entropie pour des opérateurs à valeurs dans un espace de Hilbert

  1987cited by this paper
• Spline Smoothing in Regression Models and Asymptotic Efficiency in $L_2$

  1985cited by this paper
• An Asymptotically Optimal Window Selection Rule for Kernel Density Estimates

  1984cited by this paper
• Probability Theory and Elements of Measure Theory

  1982cited by this paper
• Introduction to Probability and Measure

  1979cited by this paper
• Density Estimation, Stochastic Processes and Prior Information

  1978cited by this paper
• Improper Priors, Spline Smoothing and the Problem of Guarding Against Model Errors in Regression

  1978cited by this paper
• The Brunn-Minkowski inequality in Gauss space

  1975cited by this paper
• SINGULAR INTEGRALS AND DIFFERENTIABILITY PROPERTIES OF FUNCTIONS

  1973cited by this paper
• A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines

  1970cited by this paper
• Entropy and "-capacity of sets in func-tional spaces

  1961cited by this paper
• Analytic Extensions of Differentiable Functions Defined in Closed Sets

  1934cited by this paper

• Frequentist Regret Analysis of Gaussian Process Thompson Sampling via Fractional Posteriors

  2026cites this paper
• Posterior concentration in spatio-temporal Hawkes processes

  2026influential citation
• Spectral Bayesian Regression on the Sphere

  2026cites this paper
• On the contraction rate of the posterior distribution for nonlinear PDE parameter identification

  2026cites this paper
• A Nonparametric Bayesian Estimator of Copula Density with Applications to Financial Market

  2025cites this paper
• Provable test-time adaptivity and distributional robustness of in-context learning

  2025cites this paper
• Defensive Model Expansion for Robust Bayesian Inference

  2025influential citation
• Adaptive finite element type decomposition of Gaussian processes

  2025influential citation
• Prior distributions for Gaussian processes in computer model emulation and calibration

  2025cites this paper
• Hierarchical Besov-Laplace priors for spatially inhomogeneous binary classification

  2025cites this paper
• A nonparametric Bayesian analysis of independent and identically distributed observations of covariate-driven Poisson processes

  2025influential citation
• Lengthscale-informed sparse grids for kernel methods in high dimensions

  2025cites this paper
• A semiparametric Bayesian estimator of copula density

  2025cites this paper
• Bayesian Data Sketching for Varying Coefficient Regression Models

  2025cites this paper
• Spatial Dependencies in Item Response Theory: Gaussian Process Priors for Geographic and Cognitive Measurement

  2025cites this paper
• On strong posterior contraction rates for Besov-Laplace priors in the white noise model

  2024cites this paper
• Contraction rates and projection subspace estimation with Gaussian process priors in high dimension

  2024influential citation
• Deep horseshoe Gaussian processes

  2024influential citation
• Scalable Bayesian inference for heat kernel Gaussian processes on manifolds

  2024cites this paper
• Scalable Gaussian Process Regression Via Median Posterior Inference for Estimating Multi-Pollutant Mixture Health Effects

  2024cites this paper
• Improved learning theory for kernel distribution regression with two-stage sampling

  2023cites this paper
• The Bayesian Approach to Inverse Robin Problems

  2023influential citation
• A Bayesian approach for consistent reconstruction of inclusions

  2023cites this paper
• Posterior concentration for Gaussian process priors under rescaled and hierarchical Matérn and Confluent Hypergeometric covariance functions

  2023influential citation
• Posterior Contraction Rates for Matérn Gaussian Processes on Riemannian Manifolds

  2023cites this paper
• Spatiotemporal Besov Priors for Bayesian Inverse Problems

  2023cites this paper
• Adaptation Using Spatially Distributed Gaussian Processes

  2023cites this paper
• Heavy-Tailed Density Estimation

  2022cites this paper
• Sketching in High-Dimensional Regression With Big Data Using Gaussian Scale Mixture Priors

  2022cites this paper
• My title

  2022cites this paper
• Approximation in Hilbert spaces of the Gaussian and related analytic kernels

  2022cites this paper
• Reliable emulation of complex functionals by active learning with error control.

  2022cites this paper
• Bayesian Data Sketching for Spatial Regression Models

  2022cites this paper
• Asymptotic Bounds for Smoothness Parameter Estimates in Gaussian Process Interpolation

  2022cites this paper
• Approximation in Hilbert spaces of the Gaussian and other weighted power series kernels

  2022cites this paper
• Spatially aware dimension reduction for spatial transcriptomics

  2022cites this paper
• Double Soft-Thresholded Model for Multi-Group Scalar on Vector-Valued Image Regression

  2022cites this paper
• Fast Instrument Learning with Faster Rates

  2022cites this paper
• Wasserstein Distributionally Robust Gaussian Process Regression and Linear Inverse Problems

  2022cites this paper
• A note on the smoothness of densities

  2022cites this paper
• Scalable and adaptive variational Bayes methods for Hawkes processes

  2022cites this paper
• Double Robust Bayesian Inference on Average Treatment Effects

  2022cites this paper
• Optimal plug-in Gaussian processes for modeling derivatives

  2022influential citation
• BAYESIAN FIXED-DOMAIN ASYMPTOTICS FOR COVARIANCE PARAMETERS IN A GAUSSIAN PROCESS MODEL

  2022cites this paper
• Inferring differential subcellular localisation in comparative spatial proteomics using BANDLE

  2022cites this paper
• Oracle posterior contraction rates under hierarchical priors

  2021cites this paper
• Spike and slab Pólya tree posterior densities: Adaptive inference

  2021cites this paper
• Fast Statistical Leverage Score Approximation in Kernel Ridge Regression

  2021cites this paper
• Sketching in Bayesian High Dimensional Regression With Big Data Using Gaussian Scale Mixture Priors

  2021cites this paper
• A Bayesian non-parametric stochastic frontier model

  2021cites this paper
• Hierarchical Gaussian Process Models for Regression Discontinuity/Kink under Sharp and Fuzzy Designs

  2021cites this paper
• Wasserstein convergence in Bayesian deconvolution models

  2021cites this paper
• Scale estimation and rate-unbiasedness for Gaussian processes under smoothness misspecification

  2021cites this paper
• Bayesian spline smoothing with ambiguous penalties

  2021cites this paper
• Uncertainty quantification for distributed regression

  2021cites this paper
• Inferring differential subcellular localisation in comparative spatial proteomics using BANDLE

  2021cites this paper
• Contraction rates for sparse variational approximations in Gaussian process regression

  2021cites this paper
• Posterior Contraction for Deep Gaussian Process Priors

  2021cites this paper
• Posterior contraction rates for constrained deep Gaussian processes in density estimation and classification

  2021cites this paper
• Bayesian Tensor Response Regression with an Application to Brain Activation Studies

  2021cites this paper
• Convergence Guarantees for Gaussian Process Approximations Under Several Observation Models

  2020cites this paper
• SVM Learning Rates for Data with Low Intrinsic Dimension

  2020influential citation
• Privacy Preserving Efficient Computation in Bayesian High Dimensional Regression With Big Data Using Gaussian Scale Mixture Priors

  2020cites this paper
• Diffusion Based Gaussian Processes on Restricted Domains

  2020cites this paper
• A Bayesian semi-parametric model for thermal proteome profiling

  2020cites this paper
• Statistical Guarantees for Transformation Based Models with Applications to Implicit Variational Inference

  2020cites this paper
• Density estimation and modeling on symmetric spaces

  2020cites this paper
• Bayesian Fixed-domain Asymptotics: Bernstein-von Mises Theorem for Covariance Parameters in a Gaussian Process Model.

  2020cites this paper
• Equivalence of Convergence Rates of Posterior Distributions and Bayes Estimators for Functions and Nonparametric Functionals

  2020cites this paper
• Sample Complexity Result for Multi-category Classifiers of Bounded Variation

  2020influential citation
• Graph based Gaussian processes on restricted domains

  2020cites this paper
• Enriched mixtures of generalised Gaussian process experts

  2020cites this paper
• Adaptive learning rates for support vector machines working on data with low intrinsic dimension

  2020cites this paper
• Bayesian Closed Surface Fitting Through Tensor Products

  2020cites this paper
• Statistical inference in high-dimensional matrix models

  2020cites this paper
• Optimal learning with Gaussians and correntropy loss

  2019cites this paper
• Rates of contraction with respect to $L_{2}$-distance for Bayesian nonparametric regression

  2019cites this paper
• Variable selection consistency of Gaussian process regression

  2019cites this paper
• A closer look at covering number bounds for Gaussian kernels

  2019cites this paper
• Learning Temporal Evolution of Spatial Dependence with Generalized Spatiotemporal Gaussian Process Models

  2019cites this paper
• ST ] 2 6 A pr 2 01 9 A Theoretical Framework for Bayesian Nonparametric Regression

  2019cites this paper
• CBMS Conference: Gaussian Random Fields, Fractals, SPDEs, and Extremes

  2019cites this paper
• Semi-parametric analysis of efficiency and productivity using Gaussian processes

  2019cites this paper
• Optimization and statistical learning theory for piecewise smooth and switching regression. (Optimisation et théorie statistique de l'apprentissage pour la régression douce par morceaux et à commutations)

  2019cites this paper
• Bayesian nonparametric inference in mechanistic models of complex biological systems

  2019cites this paper
• Gaussian Processes with Errors in Variables: Theory and Computation

  2019cites this paper
• Consistency of Bayesian inference with Gaussian process priors in an elliptic inverse problem

  2019cites this paper
• Accurate Uncertainty Estimation and Decomposition in Ensemble Learning

  2019cites this paper
• Can We Trust Bayesian Uncertainty Quantification from Gaussian Process Priors with Squared Exponential Covariance Kernel?

  2019cites this paper
• Joint quantile regression for spatial data

  2019cites this paper
• Consistent online Gaussian process regression without the sample complexity bottleneck

  2019cites this paper
• Semi-Supervised Non-Parametric Bayesian Modelling of Spatial Proteomics.

  2019cites this paper
• Comparison theorems on large-margin learning

  2019cites this paper
• Data-driven confidence bands for distributed nonparametric regression

  2019cites this paper
• Posterior Contraction Rates for Gaussian Cox Processes with Non-identically Distributed Data

  2019cites this paper
• Adaptive Ensemble Learning of Spatiotemporal Processes with Calibrated Predictive Uncertainty: A Bayesian Nonparametric Approach

  2019cites this paper
• Spike and Slab P\'olya tree posterior distributions

  2019cites this paper
• L ] 1 3 A ug 2 01 9 Comparison theorems on large-margin learning

  2019cites this paper
• Local Function Complexity for Active Learning via Mixture of Gaussian Processes

  2019cites this paper
• Optimal Bayesian distributed methods for nonparametric regression

  2018cites this paper