An Exact Bayesian Model for Meta-Analysis of the Standardized Mean Difference with Its Simultaneous Credible Intervals

Abstract While Bayesian methodology is increasingly favored in behavioral research for its clear probabilistic inference and model structure, its widespread acceptance as a standard meta-analysis approach remains limited. Although some conventional Bayesian hierarchical models are frequently used for analysis, their performance has not been thoroughly examined. This study evaluates two commonly used Bayesian models for meta-analysis of standardized mean difference and identifies significant issues with these models. In response, we introduce a new Bayesian model equipped with novel features that address existing model concerns and a broader limitation of the current Bayesian meta-analysis. Furthermore, we introduce a simple computational approach to construct simultaneous credible intervals for the summary effect and between-study heterogeneity, based on their joint posterior samples. This fully captures the joint uncertainty in these parameters, a task that is challenging or impractical with frequentist models. Through simulation studies rooted in a joint Bayesian/frequentist paradigm, we compare our model’s performance against existing ones under conditions that mirror realistic research scenarios. The results reveal that our new model outperforms others and shows enhanced statistical properties. We also demonstrate the practicality of our models using real-world examples, highlighting how our approach strengthens the robustness of inferences regarding the summary effect.

• Publication year

  2024

• Venue

  Multivariate Behavioral Research

• Publication date

  2024-07-23

• Fields of study

  Medicine, Psychology

• Identifiers
  DOI 10.1080/00273171.2024.2358233PMID 39042102
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Sample Size in Behavioral Research: A Systematic Review of JEAB and JABA from 2009 to 2018

  2022cited by this paper
• Systematically Defined Informative Priors in Bayesian Estimation: An Empirical Application on the Transmission of Internalizing Symptoms Through Mother-Adolescent Interaction Behavior

  2021cited by this paper
• On weakly informative prior distributions for the heterogeneity parameter in Bayesian random‐effects meta‐analysis

  2020cited by this paper
• Video-game based instruction for vocabulary acquisition with English language learners: A Bayesian meta-analysis

  2020influential reference
• A comparison of Bayesian synthesis approaches for studies comparing two means: A tutorial

  2019influential reference
• Comparing Objective and Subjective Bayes Factors for the Two-Sample Comparison: The Classification Theorem in Action

  2019cited by this paper
• Analysing data and undertaking meta‐analyses

  2019cited by this paper
• Auditory Distraction During Reading: A Bayesian Meta-Analysis of a Continuing Controversy

  2018cited by this paper
• Bayesian Meta-Analysis with Weakly Informative Prior Distributions

  2018cited by this paper
• Computational Modeling of Cognition and Behavior

  2018cited by this paper
• A comparison of heterogeneity variance estimators in simulated random‐effects meta‐analyses

  2018cited by this paper
• Some Natural Solutions to the p-Value Communication Problem—and Why They Won’t Work

  2017cited by this paper
• The need for Bayesian hypothesis testing in psychological science

  2017cited by this paper
• The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective

  2017cited by this paper
• Estimates of Between-Study Heterogeneity for 705 Meta-Analyses Reported in Psychological Bulletin From 1990-2013

  2017cited by this paper
• A Bayesian Power Analysis Procedure Considering Uncertainty in Effect Size Estimates from a Meta-analysis

  2016cited by this paper
• The ASA Statement on p-Values: Context, Process, and Purpose

  2016cited by this paper
• Bayesian hypothesis testing for selected regression coefficients

  2016cited by this paper
• Is the call to abandon p-values the red herring of the replicability crisis?

  2015cited by this paper
• The fallacy of placing confidence in confidence intervals

  2015cited by this paper
• Methods to estimate the between‐study variance and its uncertainty in meta‐analysis†

  2015cited by this paper
• Estimating the reproducibility of psychological science

  2015cited by this paper
• Bayesian Methods: A Social and Behavioral Sciences Approach

  2014cited by this paper
• Data from Paper “False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant”

  2014cited by this paper
• Bayesian estimation supersedes the t test.

  2013cited by this paper
• Robust misinterpretation of confidence intervals

  2013cited by this paper
• Bayesian data analysis, third edition

  2013cited by this paper
• Effect size estimates: current use, calculations, and interpretation.

  2012influential reference
• The Time Has Come

  2012cited by this paper
• The Time Has Come : Bayesian Methods for Data Analysis in the Organizational Sciences

  2012cited by this paper
• Default Bayes Factors for Model Selection in Regression

  2012cited by this paper
• Sample Size in Psychological Research over the Past 30 Years

  2011cited by this paper
• Bayesian methods

  2011cited by this paper
• On the half-cauchy prior for a global scale parameter

  2011cited by this paper
• False-Positive Psychology

  2011cited by this paper
• Bayesian mediation analysis.

  2009cited by this paper
• Bayesian t tests for accepting and rejecting the null hypothesis

  2009cited by this paper
• THE FORMAL DEFINITION OF REFERENCE PRIORS

  2009cited by this paper
• New Effect Size Rules of Thumb

  2009influential reference
• Introduction to Meta‐Analysis

  2009influential reference
• A re-evaluation of the ‘quantile approximation method’ for random effects meta-analysis

  2008cited by this paper
• Confidence intervals for the overall effect size in random-effects meta-analysis.

  2008cited by this paper
• Statistical Methods for Meta‐Analysis

  2008influential reference
• Statistical Meta-Analysis with Applications

  2008cited by this paper
• A practical solution to the pervasive problems ofp values

  2007cited by this paper
• Confidence intervals for the amount of heterogeneity in meta‐analysis

  2007cited by this paper
• Hypothesis tests for population heterogeneity in meta-analysis.

  2007cited by this paper
• A comparison of heterogeneity variance estimators in combining results of studies

  2007cited by this paper
• Relational and Arelational Confidence Intervals

  2005cited by this paper
• Simple heterogeneity variance estimation for meta‐analysis

  2005cited by this paper
• Mindless statistics

  2004cited by this paper
• Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper)

  2004cited by this paper
• Measuring inconsistency in meta-analyses

  2003cited by this paper
• Hypothesis testing and theory evaluation at the boundaries: surprising insights from Bayes's theorem.

  2003cited by this paper
• Quantifying heterogeneity in a meta‐analysis

  2002cited by this paper
• Bayesian methods in meta-analysis and evidence synthesis.

  2001cited by this paper
• A refined method for the meta‐analysis of controlled clinical trials with binary outcome

  2001cited by this paper
• On tests of the overall treatment effect in meta‐analysis with normally distributed responses

  2001cited by this paper
• Bayesian Theory

  2001cited by this paper
• Bayesian methods in meta-analysis and evidence synthesis

  2001influential reference
• The proof of the pudding: an illustration of the relative strengths of null hypothesis, meta-analysis, and Bayesian analysis.

  2000cited by this paper
• Null hypothesis significance testing: a review of an old and continuing controversy.

  2000cited by this paper
• Approximate Bayesian inference for random effects meta-analysis.

  1998cited by this paper
• Markov Chain Monte Carlo in Practice: A Roundtable Discussion

  1998cited by this paper
• Bayesian Computation and Stochastic Systems

  1995cited by this paper
• Statistical Decision Theory and Bayesian Analysis, Second Edition

  1993cited by this paper
• Methods of Meta-Analysis

  1989cited by this paper
• Statistical Power Analysis for the Behavioral Sciences (2nd ed.)

  1989influential reference
• Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence

  1987cited by this paper
• Distribution Theory for Glass's Estimator of Effect size and Related Estimators

  1981cited by this paper
• The file drawer problem and tolerance for null results

  1979cited by this paper
• Frequentist probability and frequentist statistics

  1977cited by this paper
• © Institute of Mathematical Statistics, 2004 The Interplay of Bayesian and Frequentist Analysis

  year unknowncited by this paper

• No citing papers are available for this paper.