Is most published research really false?

There has been an increasing concern that most published medical findings are false. But what does it mean to be false? Here we describe the range of definitions of false discoveries in the scientific literature. We summarize the philosophical, statistical, and experimental evidence for each type of false discovery. We discuss common underpinning problems with the scientific and data analytic practices and point to tools and behaviors that can be implemented to reduce the problems with published scientific results.

• Publication year

  2016

• Venue

  bioRxiv

• Publication date

  2016-04-27

• Fields of study

  Biology, Computer Science, Philosophy, Psychology, Medicine

• Identifiers
  DOI 10.1101/050575
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Investigating Variation in Replicability: A “Many Labs” Replication Project

  2022cited by this paper
• Ten Simple Rules for Effective Statistical Practice

  2016cited by this paper
• Dynamic Documents for R

  2016cited by this paper
• What Should Researchers Expect When They Replicate Studies? A Statistical View of Replicability in Psychological Science

  2016cited by this paper
• Peer review: Troubled from the start

  2016cited by this paper
• The ASA Statement on p-Values: Context, Process, and Purpose

  2016cited by this paper
• Statistics: P values are just the tip of the iceberg

  2015cited by this paper
• Promoting an open research culture : Author guidelines for journals could help to promote transparency , openness , and reproducibility

  2015cited by this paper
• Estimating the reproducibility of psychological science

  2015cited by this paper
• Promoting an open research culture

  2015cited by this paper
• Growth in a Time of Debt

  2015cited by this paper
• A glass half full interpretation of the replicability of psychological science

  2015cited by this paper
• Dynamic Documents with R and knitr

  2015cited by this paper
• Data Carpentry: Workshops to Increase Data Literacy for Researchers

  2015cited by this paper
• NIH plans to enhance reproducibility

  2014cited by this paper
• How to Make More Published Research True

  2014cited by this paper
• Discussion: Difficulties in making inferences about scientific truth from distributions of published p-values.

  2014cited by this paper
• Changing our culture.

  2014cited by this paper
• Policy: NIH plans to enhance reproducibility

  2014cited by this paper
• An estimate of the science-wise false discovery rate and application to the top medical literature.

  2014cited by this paper
• Power failure: why small sample size undermines the reliability of neuroscience

  2013cited by this paper
• Massive Open Online Courses in Public Health

  2013cited by this paper
• Toward Reproducible Computational Research: An Empirical Analysis of Data and Code Policy Adoption by Journals

  2013cited by this paper
• A Survey on Data Reproducibility in Cancer Research Provides Insights into Our Limited Ability to Translate Findings from the Laboratory to the Clinic

  2013cited by this paper
• Evolution of Translational Omics: Lessons Learned and the Path Forward

  2013cited by this paper
• The Open Science Framework

  2013cited by this paper
• Next-generation sequencing data interpretation: enhancing reproducibility and accessibility

  2012cited by this paper
• Drug development: Raise standards for preclinical cancer research

  2012cited by this paper
• Public Availability of Published Research Data in High-Impact Journals

  2011cited by this paper
• Case studies in reproducibility

  2011cited by this paper
• Reproducible Research in Computational Science

  2011cited by this paper
• Believe it or not: how much can we rely on published data on potential drug targets?

  2011cited by this paper
• Disclose all data in publications.

  2010cited by this paper
• Galaxy: a comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life sciences

  2010cited by this paper
• Repeatability of published microarray gene expression analyses

  2009cited by this paper
• Reproducible research and Biostatistics.

  2009cited by this paper
• Deriving chemosensitivity from cell lines: Forensic bioinformatics and reproducible research in high-throughput biology

  2009cited by this paper
• An Introduction to the Dataverse Network as an Infrastructure for Data Sharing

  2007cited by this paper
• IPython: A System for Interactive Scientific Computing

  2007cited by this paper
• Genomic signatures to guide the use of chemotherapeutics

  2006cited by this paper
• Software Carpentry: Getting Scientists to Write Better Code by Making Them More Productive

  2006cited by this paper
• Software Carpentry

  2006cited by this paper
• Reproducible epidemiologic research.

  2006cited by this paper
• Why Most Published Research Findings Are False

  2005cited by this paper
• Contradicted and initially stronger effects in highly cited clinical research.

  2005cited by this paper
• The question.

  2000cited by this paper
• WaveLab and Reproducible Research

  1995cited by this paper

• The Controversy behind the Controversies: Scientific Discourse in the Twenty-First Century

  2025cites this paper
• Replicability of bulk RNA-Seq differential expression and enrichment analysis results for small cohort sizes

  2025cites this paper
• Replicability: Terminology, Measuring Success, and Strategy

  2025cites this paper
• Raising awareness of uncertain choices in empirical data analysis: A teaching concept toward replicable research practices

  2024cites this paper
• Characterising reproducibility debt in scientific software: A systematic literature review

  2024cites this paper
• Challenges in Lipidomics Biomarker Identification: Avoiding the Pitfalls and Improving Reproducibility

  2024cites this paper
• Detailed Implementation of a Reproducible Machine Learning-Enabled Workflow

  2024cites this paper
• Replicability of bulk RNA-Seq differential expression and enrichment analysis results in cancer research

  2023cites this paper
• A Study in Reproducibility: The Congruent Matching Cells Algorithm and cmcR Package

  2023cites this paper
• Editorial: Special Issue on Reproducibility and Replicability

  2023cites this paper
• A Response to Rice and Lumley

  2023cites this paper
• The use of the phrase “data not shown” in dental research

  2022cites this paper
• Estimating the replicability of highly cited clinical research (2004–2018)

  2022cites this paper
• The State of Play of Reproducibility in Statistics: An Empirical Analysis

  2022cites this paper
• Publication Bias in Asset Pricing Research

  2022cites this paper
• Comparison of text preprocessing methods

  2022cites this paper
• The Scholarly Knowledge Ecosystem: Challenges and Opportunities for the Field of Information

  2021cites this paper
• A field-wide assessment of differential expression profiling by high-throughput sequencing reveals widespread bias

  2021cites this paper
• r-cubed: Guiding the overwhelmed scientist from random wrangling to Reproducible Research in R

  2021cites this paper
• Most claimed statistical findings in cross-sectional return predictability are likely true

  2021cites this paper
• Complementing the P-value from null-hypothesis significance testing with a Bayes factor from null-hypothesis Bayesian testing.

  2020cites this paper
• Assessing Consistency in Single-Case Data Features Using Modified Brinley Plots

  2020cites this paper
• Improving reproducibility probability estimation and preserving RP-testing

  2020cites this paper
• Improving reproducibility probability estimation and preserving RP-testing

  2020cites this paper
• Best uses of p-values and complementary measures in medical research: Recent developments in the frequentist and Bayesian frameworks

  2020cites this paper
• Modernizing the Methods and Analytics Curricula for Health Science Doctoral Programs

  2020cites this paper
• ТИРАЖИРУЕМЫЕ УСЛУГИ ДЛЯ ВОСПРОИЗВОДИМЫХ ИССЛЕДОВАНИЙ: МОДЕЛЬ ДЛЯ НАУЧНЫХ БИБЛИОТЕК

  2020cites this paper
• The case for formal methodology in scientific reform

  2020cites this paper
• Assessing the evidence of perspective taking on stereotyping and negative evaluations: A p-curve analysis

  2020cites this paper
• Estimating publication bias in meta‐analyses of peer‐reviewed studies: A meta‐meta‐analysis across disciplines and journal tiers

  2020cites this paper
• YARD: A Tool for Curating Research Outputs

  2020cites this paper
• High-throughput analysis suggests differences in journal false discovery rate by subject area and impact factor but not open access status

  2020cites this paper
• If journals embraced conditional equivalence testing, would research be better?

  2019cites this paper
• Replicable Services for Reproducible Research: A Model for Academic Libraries

  2019cites this paper
• Factors Affecting the Growth of Pseudokirchneriella subcapitata in Single‐Species Tests: Lessons for the Experimental Design and the Reproducibility of a Multitrophic Laboratory Microcosm

  2019cites this paper
• An Overview of Scientific Reproducibility: Consideration of Relevant Issues for Behavior Science/Analysis

  2019influential citation
• qPortal: A platform for data-driven biomedical research

  2018cites this paper
• The GalNAc-T Activation (GALA) Pathway: Drivers and markers

  2018cites this paper
• The Reproducibility Crisis and Academic Libraries

  2018cites this paper
• Bibliography

  2018cites this paper
• References

  2018cites this paper
• Computational reproducibility in geoscientific papers: Insights from a series of studies with geoscientists and a reproduction study

  2018cites this paper
• Analysis validation has been neglected in the Age of Reproducibility

  2018cites this paper
• Forecasting, uncertainty and risk; perspectives on clinical decision-making in preventive and curative medicine

  2018cites this paper
• Analysis of >30,000 abstracts suggests higher false discovery rates for oncology journals, especially those with low impact factors

  2018cites this paper
• The World of Research Has Gone Berserk: Modeling the Consequences of Requiring “Greater Statistical Stringency” for Scientific Publication

  2018cites this paper
• Reliability issues in imaging genetics

  2017cites this paper
• The earth is flat (p > 0.05): significance thresholds and the crisis of unreplicable research

  2017cites this paper
• Analyzing dynamic data: A tutorial

  2017cites this paper
• The decline effect in evaluations of the impact of the Strengthening Families Program for Youth 10-14 (SFP 10-14) on adolescent substance use

  2017cites this paper
• Trust in numbers

  2017cites this paper
• The earth is flat (p>0.05): Significance thresholds and the crisis of unreplicable research

  2017cites this paper
• The earth is flat ( p > 0 . 05 ) : Significance thresholds and the crisis of unreplicable research ( # 16890 ) 1

  2017cites this paper
• Conditional equivalence testing: An alternative remedy for publication bias

  2017cites this paper
• If these data could talk

  2017cites this paper
• A critical reexamination of doing arithmetic nonconsciously

  2017cites this paper
• Science with no fiction: measuring the veracity of scientific reports by citation analysis

  2017cites this paper
• A statistical definition for reproducibility and replicability

  2016cites this paper
• Irreproducibility of published bioscience research: Diagnosis, pathogenesis and therapy

  2016influential citation
• Electronic

  1991cites this paper