p-value peeking and estimating extrema.

A pervasive issue in statistical hypothesis testing is that the reported $p$-values are biased downward by data "peeking" -- the practice of reporting only progressively extreme values of the test statistic as more data samples are collected. We develop principled mechanisms to estimate such running extrema of test statistics, which directly address the effect of peeking in some general scenarios.

• Publication year

  2020

• Venue

  arXiv: Statistics Theory

• Publication date

  2020-11-02

• Fields of study

  Mathematics

• Identifiers
  arXiv 2011.01343
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Foundations of Modern Probability

  2021cited by this paper
• Nonparametric Iterated-Logarithm Extensions of the Sequential Generalized Likelihood Ratio Test

  2020cited by this paper
• Admissible anytime-valid sequential inference must rely on nonnegative martingales.

  2020cited by this paper
• Universal inference

  2019cited by this paper
• A simple proof of a theorem of Blackwell and Dubins on the maximum of a uniformly integrable martingale

  2019influential reference
• Safe Testing

  2019cited by this paper
• Time-uniform, nonparametric, nonasymptotic confidence sequences

  2018cited by this paper
• Uniform, nonparametric, non-asymptotic confidence sequences

  2018cited by this paper
• Sequential Nonparametric Testing with the Law of the Iterated Logarithm

  2015cited by this paper
• An Ethical Approach to Peeking at Data

  2014cited by this paper
• Sharp Finite-Time Iterated-Logarithm Martingale Concentration

  2014influential reference
• Random variables, monotone relations, and convex analysis

  2014influential reference
• A reading guide for last passage times with financial applications in view

  2013cited by this paper
• False-Positive Psychology

  2011cited by this paper
• Test Martingales, Bayes Factors and p-Values

  2009cited by this paper
• On Az\'ema-Yor processes, their optimal properties and the Bachelier-drawdown equation

  2009influential reference
• Why Most Discovered True Associations Are Inflated

  2008cited by this paper
• MAX-PLUS DECOMPOSITION OF SUPERMARTINGALES AND CONVEX ORDER. APPLICATION TO AMERICAN OPTIONS AND PORTFOLIO INSURANCE

  2008cited by this paper
• Bayesian Statistical Inference for Psychological Research

  2008cited by this paper
• A practical solution to the pervasive problems ofp values

  2007cited by this paper
• Doob's maximal identity, multiplicative decompositions and enlargements of filtrations

  2005cited by this paper
• Non-stopping times and stopping theorems

  2005cited by this paper
• When (not) to stop a clinical trial for benefit.

  2005cited by this paper
• A DEFINITION AND SOME CHARACTERISTIC PROPERTIES OF PSEUDO-STOPPING TIMES

  2005cited by this paper
• The Skorokhod embedding problem and its offspring

  2004cited by this paper
• Robust Hedging of Barrier Options

  2001cited by this paper
• Null hypothesis significance testing: a review of an old and continuing controversy.

  2000cited by this paper
• Optimization of conditional value-at risk

  2000cited by this paper
• Unified frequentist and Bayesian testing of a precise hypothesis

  1997cited by this paper
• Probability: Theory and Examples

  1993cited by this paper
• A logic of probability, with application to the foundations of statistics

  1993cited by this paper
• Une solution simple au probleme de Skorokhod

  1979influential reference
• On the distribution of maxima of martingales

  1978cited by this paper
• Group sequential methods in the design and analysis of clinical trials

  1977cited by this paper
• Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II. analysis and examples.

  1977cited by this paper
• Théorie générale des processus et retournement du temps

  1973cited by this paper
• Boundary Crossing Probabilities for the Wiener Process and Sample Sums

  1970influential reference
• Repeated Significance Tests on Accumulating Data

  1969cited by this paper
• Some nonparametric sequential tests with power one.

  1968cited by this paper
• A converse to the dominated convergence theorem

  1963cited by this paper
• Fixed-Sample-Size Analysis of Sequential Observations

  1954cited by this paper
• Some aspects of the sequential design of experiments

  1952cited by this paper
• Optimum Character of the Sequential Probability Ratio Test

  1948cited by this paper
• Étude critique de la notion de collectif

  1939cited by this paper
• Théorie des probabilités continues

  year unknowncited by this paper

• No citing papers are available for this paper.