Fast sampling with Gaussian scale-mixture priors in high-dimensional regression.

We propose an efficient way to sample from a class of structured multivariate Gaussian distributions. The proposed algorithm only requires matrix multiplications and linear system solutions. Its computational complexity grows linearly with the dimension, unlike existing algorithms that rely on Cholesky factorizations with cubic complexity. The algorithm is broadly applicable in settings where Gaussian scale mixture priors are used on high-dimensional parameters. Its effectiveness is illustrated through a high-dimensional regression problem with a horseshoe prior on the regression coefficients. Other potential applications are outlined.

• Publication year

  2015

• Venue

  Biometrika

• Publication date

  2015-06-15

• Fields of study

  Mathematics, Computer Science, Medicine

• Identifiers
  DOI 10.1093/biomet/asw042arXiv 1506.04778PMID 28435166PMCID PMC5400369
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Variable selection using shrinkage priors

  2015cited by this paper
• Dirichlet–Laplace Priors for Optimal Shrinkage

  2014cited by this paper
• The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors

  2014cited by this paper
• Geometric ergodicity for Bayesian shrinkage models

  2014cited by this paper
• Decoupling Shrinkage and Selection in Bayesian Linear Models: A Posterior Summary Perspective

  2014cited by this paper
• On asymptotically optimal confidence regions and tests for high-dimensional models

  2013cited by this paper
• Geometric ergodicity of the Bayesian lasso

  2013cited by this paper
• Frequentist coverage of adaptive nonparametric Bayesian credible sets

  2013cited by this paper
• Confidence intervals and hypothesis testing for high-dimensional regression

  2013cited by this paper
• Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences

  2012cited by this paper
• Locally adaptive factor processes for multivariate time series

  2012cited by this paper
• Shrink Globally, Act Locally: Sparse Bayesian Regularization and Prediction

  2012influential reference
• Posterior contraction in sparse Bayesian factor models for massive covariance matrices

  2012cited by this paper
• Bayesian Model Selection in High-Dimensional Settings

  2012cited by this paper
• Sparse Bayesian infinite factor models.

  2011cited by this paper
• GENERALIZED DOUBLE PARETO SHRINKAGE.

  2011cited by this paper
• Confidence intervals for low dimensional parameters in high dimensional linear models

  2011cited by this paper
• The Bayesian bridge

  2011cited by this paper
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem

  2010cited by this paper
• Inference with normal-gamma prior distributions in regression problems

  2010cited by this paper
• The horseshoe estimator for sparse signals

  2010cited by this paper
• Nearly unbiased variable selection under minimax concave penalty

  2010cited by this paper
• Bayesian lasso regression

  2009cited by this paper
• The Bayesian Lasso

  2008cited by this paper
• Sparse Bayesian nonparametric regression

  2008cited by this paper
• Bayesian auxiliary variable models for binary and multinomial regression

  2006cited by this paper
• Sure independence screening for ultrahigh dimensional feature space

  2006cited by this paper
• Gene selection using a two-level hierarchical Bayesian model

  2004cited by this paper
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties

  2001cited by this paper
• Sparse Bayesian Learning and the Relevance Vector Machine

  2001cited by this paper
• Fast sampling of Gaussian Markov random fields

  2000influential reference
• APPROACHES FOR BAYESIAN VARIABLE SELECTION

  1997cited by this paper
• Bayesian analysis of binary and polychotomous response data

  1993cited by this paper
• Updating the Inverse of a Matrix

  1989influential reference
• Bayesian Variable Selection in Linear Regression: Rejoinder

  1988cited by this paper
• Matrix computations

  1983cited by this paper

• Benchmarking Sparse Variable Selection Methods for Genomic Data Analyses

  2026cites this paper
• Flexible Bayesian MIDAS: Time-Variation, Group-Shrinkage and Sparsity

  2025cites this paper
• BaGGLS: A Bayesian Shrinkage Framework for Interpretable Modeling of Interactions in High-Dimensional Biological Data

  2025cites this paper
• Bayesian Modeling of Gene Regulatory Networks in Colorectal Cancer Organoids

  2025cites this paper
• Bayesian Transfer Learning for High-Dimensional Linear Regression via Adaptive Shrinkage

  2025influential citation
• An Order of Magnitude Time Complexity Reduction for Gaussian Graphical Model Posterior Sampling Using a Reverse Telescoping Block Decomposition

  2025influential citation
• A Bayesian Informative Shrinkage Approach for Large-scale Multiple Hypothesis Testing (BISHOT): with Applications in Differential Analysis of Omics Data

  2025cites this paper
• Forecasting in small open emerging economies Evidence from Thailand

  2025influential citation
• Formal Bayesian Transfer Learning via the Total Risk Prior

  2025cites this paper
• On the Posterior Computation Under the Dirichlet-Laplace Prior

  2025cites this paper
• Mhorseshoe package in R: Approximate algorithm for the horseshoe prior in Bayesian linear model

  2025cites this paper
• Joint Quantile Shrinkage: A State-Space Approach toward Non-Crossing Bayesian Quantile Models

  2025cites this paper
• Bayesian Data Sketching for Varying Coefficient Regression Models

  2025cites this paper
• Multivariate Bayesian Global–Local Shrinkage Methods for Regularisation in the High-Dimensional Linear Model

  2025cites this paper
• Monthly GDP Growth Estimates for the U.S. States

  2025cites this paper
• Dependency-Aware Shrinkage Priors for High Dimensional Regression

  2025cites this paper
• Comparing MCMC algorithms for the horseshoe prior

  2024cites this paper
• The Group R2D2 Shrinkage Prior for Sparse Linear Models with Grouped Covariates

  2024cites this paper
• Automated Bayesian variable selection methods for binary regression models with missing covariate data

  2024cites this paper
• Bayesian Scalable Precision Factor Analysis for Gaussian Graphical Models

  2024cites this paper
• Fused $L_{1/2}$ prior for large scale linear inverse problem with Gibbs bouncy particle sampler

  2024cites this paper
• Provably Efficient Posterior Sampling for Sparse Linear Regression via Measure Decomposition

  2024cites this paper
• A Bayesian Generalized Bridge Regression Approach to Covariance Estimation in the Presence of Covariates

  2024influential citation
• Bayesian Variable Shrinkage and Selection in Compositional Data Regression: Application to Oral Microbiome

  2024cites this paper
• Consistent skinny Gibbs in probit regression

  2024cites this paper
• Nowcasting with Mixed Frequency Data Using Gaussian Processes

  2024cites this paper
• Fast and order‐invariant inference in Bayesian VARs with nonparametric shocks

  2023cites this paper
• BVARs and Stochastic Volatility

  2023cites this paper
• Gibbs Sampling Using Anti-Correlation Gaussian Data Augmentation, with Applications to L1-Ball-Type Models

  2023cites this paper
• A fast asynchronous Markov chain Monte Carlo sampler for sparse Bayesian inference

  2023cites this paper
• Fast Bayesian High-Dimensional Gaussian Graphical Model Estimation

  2023cites this paper
• From Estimation to Sampling for Bayesian Linear Regression with Spike-and-Slab Prior

  2023influential citation
• Efficient Bayesian High-Dimensional Classification via Random Projection with Application to Gene Expression Data

  2023cites this paper
• Ultra-Efficient MCMC for Bayesian Longitudinal Functional Data Analysis

  2023cites this paper
• A hybrid deterministic–deterministic approach for high-dimensional Bayesian variable selection with a default prior

  2023cites this paper
• Global and local components of output gaps

  2023cites this paper
• Informative Co‐Data Learning for High‐Dimensional Horseshoe Regression

  2023cites this paper
• Variable selection for high-dimensional incomplete data using horseshoe estimation with data augmentation

  2023cites this paper
• Time-dependent shrinkage of time-varying parameter regression models

  2023cites this paper
• Regression‐based Bayesian estimation and structure learning for nonparanormal graphical models

  2022cites this paper
• Bayesian fused lasso modeling via horseshoe prior

  2022cites this paper
• Sparse multivariate modeling for stock returns predictability∗

  2022cites this paper
• Variational Bayes inference for large-scale multivariate predictive regressions∗

  2022cites this paper
• Change-Point Detection and Regularization in Time Series Cross-Sectional Data Analysis

  2022cites this paper
• Sparse Horseshoe Estimation via Expectation-Maximisation

  2022cites this paper
• Bayesian dynamic network modelling: an application to metabolic associations in cardiovascular diseases

  2022cites this paper
• Bayesian Shrinkage Priors in Zero-Inflated and Negative Binomial Regression models with Real World Data Applications of COVID-19 Vaccine, and RNA-Seq

  2022cites this paper
• MCMC Convergence for Global-Local Shrinkage Priors

  2022cites this paper
• Sketching in High-Dimensional Regression With Big Data Using Gaussian Scale Mixture Priors

  2022influential citation
• Bayesian Data Sketching for Spatial Regression Models

  2022cites this paper
• A new algorithm for structural restrictions in Bayesian vector autoregressions

  2022cites this paper
• Solving the Poisson equation using coupled Markov chains

  2022cites this paper
• Prior Knowledge Guided Ultra-high Dimensional Variable Screening with Application to Neuroimaging Data.

  2022cites this paper
• Forecasting macroeconomic data with Bayesian VARs: Sparse or dense? It depends!

  2022cites this paper
• Lossy compression of matrices by black box optimisation of mixed integer nonlinear programming

  2022cites this paper
• Shrinkage priors for high-dimensional demand estimation

  2022influential citation
• Scalable Spike-and-Slab

  2022influential citation
• Centered plug-in estimation of Wasserstein distances

  2022cites this paper
• An overview of heavy-tail extensions of multivariate Gaussian distribution and their relations

  2022cites this paper
• Synthetic Control with Time Varying Coefficients A State Space Approach with Bayesian Shrinkage

  2022cites this paper
• Two-step mixed-type multivariate Bayesian sparse variable selection with shrinkage priors

  2022influential citation
• Sparse linear mixed model selection via streamlined variational Bayes

  2021cites this paper
• Bayesian Tensor Response Regression with an Application to Brain Activation Studies

  2021cites this paper
• Bayesian inference in high-dimensional models

  2021cites this paper
• Geometric ergodicity of Gibbs samplers for the Horseshoe and its regularized variants

  2021cites this paper
• Applications of Bayesian shrinkage prior models in clinical research with categorical responses

  2021cites this paper
• Group Inverse-Gamma Gamma Shrinkage for Sparse Regression with Block-Correlated Predictors

  2021cites this paper
• Integrated transcriptomic and neuroimaging brain model decodes biological mechanisms in aging and Alzheimer’s disease

  2021cites this paper
• Joint mean-covariance estimation via the horseshoe

  2021cites this paper
• Variational Inference in high-dimensional linear regression

  2021cites this paper
• Inference with gamma and inverse gamma prior densities in left-censored regression

  2021cites this paper
• Circadian Gene Selection for Time-to-event Phenotype by Integrating CNV and RNAseq Data.

  2021cites this paper
• Sampling Local‐Scale Parameters in High‐Dimensional Regression Models

  2021cites this paper
• Bayesian global-local shrinkage methods for regularisation in the high dimension linear model

  2021influential citation
• A comparative study on high-dimensional bayesian regression with binary predictors

  2021cites this paper
• Sparse Modeling for Cross-Industry Returns Predictability

  2021influential citation
• Sketching in Bayesian High Dimensional Regression With Big Data Using Gaussian Scale Mixture Priors

  2021influential citation
• Structural damage identification with limited modal measurements and ultra‐sparse Bayesian regression

  2021cites this paper
• Robust Bayesian nonparametric variable selection for linear regression

  2021cites this paper
• Bayesian Inference for Gamma Models

  2021cites this paper
• Decoupling Shrinkage and Selection for the Bayesian Quantile Regression

  2021influential citation
• Bayesian Precision Factor Analysis for High-dimensional Sparse Gaussian Graphical Models

  2021cites this paper
• Subset selection for linear mixed models

  2021cites this paper
• Computational Statistics and Data Science in the Twenty‐First Century

  2021cites this paper
• Bayesian L1/2 Regression

  2021cites this paper
• A fast asynchronous MCMC sampler for sparse Bayesian inference

  2021cites this paper
• Bayesian Scalable Precision Factor Analysis for Massive Sparse Gaussian Graphical Models

  2021cites this paper
• Flexible Mixture Priors for Large Time-varying Parameter Models

  2021cites this paper
• The Time-Varying Evolution of Inflation Risks

  2021cites this paper
• Strathclyde

  2021influential citation
• An approach of Bayesian variable selection for ultrahigh‐dimensional multivariate regression

  2021cites this paper
• Bounding Wasserstein Distance with Couplings

  2021influential citation
• Group Inverse-Gamma Gamma Shrinkage for Sparse Linear Models with Block-Correlated Regressors

  2021cites this paper
• Bayesian group regularization in generalized linear models with a continuous spike-and-slab prior

  2020cites this paper
• Bayesian estimation of bidding process and bidder’s preference under shape restrictions

  2020cites this paper
• Drug sensitivity prediction with normal inverse Gaussian shrinkage informed by external data

  2020influential citation
• Bayesian estimation of bidding process and bidder’s preference under shape restrictions

  2020cites this paper
• Privacy Preserving Efficient Computation in Bayesian High Dimensional Regression With Big Data Using Gaussian Scale Mixture Priors

  2020cites this paper
• Bayesian model selection for high-dimensional data

  2020cites this paper
• Horseshoe Regularisation for Machine Learning in Complex and Deep Models 1

  2020cites this paper