Bamboozled by Bonferroni

Abstract When many statistical hypotheses are evaluated simultaneously, statisticians often recommend adjusting (or “correcting”) standard hypothesis tests. In this article, I (1) distinguish two senses of adjustment, (2) investigate the prudential and epistemic goals that adjustment might achieve, and (3) identify conditions under which a researcher should not adjust for multiplicity in the two senses I identify. I tentatively conclude that the goals of scientists and the public may be misaligned with the decision criteria used to evaluate multiple-testing regimes.

• Publication year

  2024

• Venue

  Philosophia Scientiæ

• Publication date

  2024-04-22

• Fields of study

  Not labeled

• Identifiers
  DOI 10.1017/psa.2024.13
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• When to adjust alpha during multiple testing: a consideration of disjunction, conjunction, and individual testing

  2021cited by this paper
• Optimal and maximin procedures for multiple testing problems

  2018cited by this paper
• Statistical Inference as Severe Testing

  2018cited by this paper
• Multiple Studies and Evidential Defeat

  2013cited by this paper
• The Laws of Belief - Ranking Theory and Its Philosophical Applications

  2012cited by this paper
• Philosophy and the practice of Bayesian statistics.

  2010cited by this paper
• Genome-wide association study of 14,000 cases of seven common diseases and 3,000 shared controls

  2007cited by this paper
• Subjective Bayesian Analysis: Principles and Practice

  2006cited by this paper
• An exploration of aspects of Bayesian multiple testing

  2006cited by this paper
• FDR and Bayesian Multiple Comparisons Rules

  2006cited by this paper
• The case for objective Bayesian analysis

  2006cited by this paper
• Characterization of Bayes procedures for multiple endpoint problems and inadmissibility of the step-up procedure

  2005cited by this paper
• A foundation for statistics

  2003cited by this paper
• Science, Truth, and Democracy

  2002cited by this paper
• Bayesian perspectives on multiple comparisons

  1999cited by this paper
• Statistical Evidence: A Likelihood Paradigm

  1998cited by this paper
• Controlling the false discovery rate: a practical and powerful approach to multiple testing

  1995cited by this paper
• Science as Social Knowledge. Values and Objectivity in Scientific Inquiry

  1990cited by this paper
• Upper and Lower Probabilities Generated by a Random Closed Interval

  1968cited by this paper

• No citing papers are available for this paper.