Block Hyper-g Priors in Bayesian Regression

The development of prior distributions for Bayesian regression has traditionally been driven by the goal of achieving sensible model selection and parameter estimation. The formalization of properties that characterize good performance has led to the development and popularization of thick tailed mixtures of g priors such as the Zellner--Siow and hyper-g priors. The properties of a particular prior are typically illuminated under limits on the likelihood or the prior. In this paper we introduce a new, conditional information asymptotic that is motivated by the common data analysis setting where at least one regression coefficient is much larger than others. We analyze existing mixtures of g priors under this limit and reveal two new behaviors, Essentially Least Squares (ELS) estimation and the Conditional Lindley's Paradox (CLP), and argue that these behaviors are, in general, undesirable. As the driver behind both of these behaviors is the use of a single, latent scale parameter that is common to all coefficients, we propose a block hyper-g prior, defined by first partitioning the covariates into groups and then placing independent hyper-g priors on the corresponding blocks of coefficients. We provide conditions under which ELS and the CLP are avoided by the new class of priors, and provide consistency results under traditional sample size asymptotics.

• Publication year

  2014

• Venue

  arXiv: Statistics Theory

• Publication date

  2014-06-25

• Fields of study

  Mathematics

• Identifiers
  arXiv 1406.6419
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• BAYES FACTORS AND MARGINAL DISTRIBUTIONS IN INVARIANT SITUATIONS

  2016cited by this paper
• Paradoxes and Priors in Bayesian Regression

  2014influential reference
• Dirichlet–Laplace Priors for Optimal Shrinkage

  2014cited by this paper
• Criteria for Bayesian model choice with application to variable selection

  2012influential reference
• Objective Bayesian inference for stress-strength models and Bayesian ANOVA

  2012cited by this paper
• Default Bayes factors for ANOVA designs

  2012cited by this paper
• Shrink Globally, Act Locally: Sparse Bayesian Regularization and Prediction

  2012influential reference
• Structuring shrinkage: some correlated priors for regression

  2012cited by this paper
• BAYESIAN HYPER‐LASSOS WITH NON‐CONVEX PENALIZATION

  2011cited by this paper
• Bayesian Adaptive Sampling for Variable Selection and Model Averaging

  2011cited by this paper
• On the half-cauchy prior for a global scale parameter

  2011cited by this paper
• GENERALIZED DOUBLE PARETO SHRINKAGE.

  2011cited by this paper
• Elastic Net Regression Modeling With the Orthant Normal Prior

  2011cited by this paper
• Local shrinkage rules, Lévy processes and regularized regression

  2010cited by this paper
• The horseshoe estimator for sparse signals

  2010cited by this paper
• Robust Bayesian variable selection with sub-harmonic priors

  2010cited by this paper
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem

  2010cited by this paper
• Inference with normal-gamma prior distributions in regression problems

  2010influential reference
• Evolutionary Stochastic Search for Bayesian model exploration

  2010cited by this paper
• Bayesian lasso regression

  2009cited by this paper
• The Bayesian Lasso

  2008influential reference
• Mixtures of g Priors for Bayesian Variable Selection

  2008influential reference
• Feature-Inclusion Stochastic Search for Gaussian Graphical Models

  2008cited by this paper
• Fully Bayes factors with a generalized g-prior

  2008influential reference
• Shotgun Stochastic Search for “Large p” Regression

  2007influential reference
• An exploration of aspects of Bayesian multiple testing

  2006cited by this paper
• Posterior model probabilities via path‐based pairwise priors

  2005cited by this paper
• Spike and slab variable selection: Frequentist and Bayesian strategies

  2005cited by this paper
• Alternative prior distributions for variable selection with very many more variables than observations

  2005cited by this paper
• Testing Statistical Hypotheses

  2005cited by this paper
• Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences

  2004cited by this paper
• Bayesian factor regression models in the''large p

  2003cited by this paper
• Computational approaches to motor learning by imitation.

  2003cited by this paper
• Objective Bayesian Methods for Model Selection: Introduction and Comparison

  2001cited by this paper
• Model Selection and the Principle of Minimum Description Length

  2001cited by this paper
• Benchmark Priors for Bayesian Model Averaging

  2001influential reference
• Ridge Regression: Biased Estimation for Nonorthogonal Problems

  2000cited by this paper
• Calibration and empirical Bayes variable selection

  2000influential reference
• APPROACHES FOR BAYESIAN VARIABLE SELECTION

  1997cited by this paper
• A Reference Bayesian Test for Nested Hypotheses and its Relationship to the Schwarz Criterion

  1995cited by this paper
• The risk inflation criterion for multiple regression

  1994cited by this paper
• Statistical Decision Theory and Bayesian Analysis, Second Edition

  1993influential reference
• Bayesian Variable Selection in Linear Regression

  1988cited by this paper
• On scale mixtures of normal distributions

  1987cited by this paper
• A Robust Generalized Bayes Estimator and Confidence Region for a Multivariate Normal Mean

  1980cited by this paper
• Posterior odds ratios for selected regression hypotheses

  1980influential reference
• Proper Bayes Minimax Estimators of the Multivariate Normal Mean

  1971cited by this paper
• A comment on D. V. Lindley's statistical paradox

  1957cited by this paper
• A STATISTICAL PARADOX

  1957influential reference
• Theory of Probability

  year unknowninfluential reference

• Bayesian Elastic Net Regression with Structured Prior Dependence

  2025cites this paper
• Bayesian hierarchical models with calibrated mixtures of g-priors for assessing treatment effect moderation in meta-analysis

  2024cites this paper
• Empirical Bayes Model Averaging with Influential Observations: Tuning Zellner’s g Prior for Predictive Robustness

  2021cites this paper
• Intuitive Joint Priors for Variance Parameters

  2019cites this paper
• Economic variable selection

  2019cites this paper
• Prior Distributions for Objective Bayesian Analysis

  2018cites this paper
• Some Themes in High-Dimensional Statistics

  2016cites this paper
• On the Computation of Gauss Hypergeometric Functions

  2015cites this paper