A guide to megastudies

Abstract How can behavioral insights best be leveraged to solve pressing policy challenges? Because research studies are typically designed to test the validity of a particular idea, surprisingly little is known about the relative efficacy of different approaches to changing behavior in any given policy context. We discuss megastudies as a research approach that can surmount this and other obstacles to developing optimal behaviorally informed policy interventions. We define a megastudy as “a massive field experiment in which many different treatments are tested synchronously in one large sample using a common, objectively measured outcome.” We summarize this apples-to-apples approach to research and lay out recommendations, limitations, and promising future directions for scholars who might want to conduct or evaluate megastudies.

• Publication year

  2022

• Venue

  PNAS Nexus

• Publication date

  2022-10-07

• Fields of study

  Medicine, Economics, Political Science

• Identifiers
  DOI 10.1093/pnasnexus/pgac214PMID 36712333PMCID 9802435
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• A 680,000-person megastudy of nudges to encourage vaccination in pharmacies

  2022cited by this paper
• No reason to expect large and consistent effects of nudge interventions

  2022cited by this paper
• No evidence for nudging after adjusting for publication bias

  2022cited by this paper
• Left-truncated effects and overestimated meta-analytic means

  2022cited by this paper
• Behavioural nudges increase COVID-19 vaccinations

  2021cited by this paper
• The effectiveness of nudging: A meta-analysis of choice architecture interventions across behavioral domains

  2021cited by this paper
• Megastudies improve the impact of applied behavioural science

  2021cited by this paper
• A megastudy of text-based nudges encouraging patients to get vaccinated at an upcoming doctor’s appointment

  2021cited by this paper
• The distributional effects of nudges

  2021cited by this paper
• Task complexity moderates group synergy

  2021cited by this paper
• Behavioral nudges increase COVID-19 vaccinations: Two randomized controlled trials

  2021cited by this paper
• Global citation inequality is on the rise

  2021cited by this paper
• Personalized nudging

  2020cited by this paper
• False Discovery in A/B Testing

  2020cited by this paper
• A/B Testing with Fat Tails

  2020cited by this paper
• RCTs to Scale: Comprehensive Evidence from Two Nudge Units

  2020cited by this paper
• How to undertake great cross-disciplinary research

  2020cited by this paper
• Nudging: Progress to date and future directions.

  2020cited by this paper
• A large-scale field experiment shows giving advice improves academic outcomes for the advisor

  2019cited by this paper
• How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses.

  2019cited by this paper
• The preregistration revolution

  2018cited by this paper
• What Motivates Effort?: Evidence and Expert Forecasts

  2018cited by this paper
• The Matthew effect in science funding

  2018cited by this paper
• Answers to 18 Questions About Open Science Practices

  2018cited by this paper
• Should Governments Invest More in Nudging?

  2017cited by this paper
• 50 Years of Data Science

  2017cited by this paper
• Should social science be more solution-oriented?

  2017cited by this paper
• Using Design Thinking to Improve Psychological Interventions: The Case of the Growth Mindset During the Transition to High School.

  2016cited by this paper
• Publication bias in the social sciences: Unlocking the file drawer

  2014cited by this paper
• Psychological Strategies for Winning a Geopolitical Forecasting Tournament

  2014cited by this paper
• The short-run and long-run effects of behavioral interventions: Experimental evidence from energy conservation

  2014cited by this paper
• Particle swarm optimization with deliberate loss of information

  2012cited by this paper
• Cryptic multiple hypotheses testing in linear models: overestimated effect sizes and the winner's curse

  2010cited by this paper
• The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): new methods for more potent eHealth interventions.

  2007cited by this paper
• Overcoming the winner's curse: estimating penetrance parameters from case-control data.

  2007cited by this paper
• Controlling the false discovery rate: a practical and powerful approach to multiple testing

  1995cited by this paper
• Estimation with Quadratic Loss

  1992cited by this paper
• Field experiments

  1924cited by this paper

• A registered report megastudy on the persuasiveness of the most-cited climate messages

  2026cites this paper
• A megastudy of behavioral interventions to catalyze public, political, and financial climate advocacy

  2026cites this paper
• From one, many: How can nudges be personalized?

  2026cites this paper
• A national megastudy shows that email nudges to elementary school teachers boost student math achievement, particularly when personalized

  2025cites this paper
• On the evolution of psychological theory: Advancing from empirical effects to single‐process explanations to multi‐process models

  2025cites this paper
• Can science-based interventions tamp down polarization?

  2025cites this paper
• Incentives and the replication crisis in social sciences: A critical review of open science practices

  2025cites this paper
• Combating vaccine hesitancy: The case of HPV vaccination.

  2025cites this paper
• Can reminder emails compel Americans to save? A two-million-person megastudy

  2025cites this paper
• Effectiveness of ex ante honesty oaths in reducing dishonesty depends on content

  2024influential citation
• How to design and conduct a megastudy

  2024cites this paper
• Leveraging LLMs for predictive insights in food policy and behavioral interventions

  2024cites this paper
• Agenda for psychological and behavioural science of transformative behavioural change

  2024cites this paper
• Using survey experiment pretesting to support future pandemic response

  2024cites this paper
• Addressing climate change with behavioral science: A global intervention tournament in 63 countries

  2024cites this paper
• Chilling results: how explicit warm glow appeals fail to boost pro-environmental behaviour

  2024cites this paper
• Field testing the transferability of behavioural science knowledge on promoting vaccinations

  2024cites this paper
• Toolbox of individual-level interventions against online misinformation

  2024cites this paper
• Behavioral public policy for global challenges 

  2024cites this paper
• Megastudy shows that reminders boost vaccination but adding free rides does not

  2024cites this paper
• The Whys and Hows of Theory—Comparing Cognitive Science and Economics

  2024cites this paper
• Boosting: Empowering Citizens with Behavioral Science.

  2024cites this paper
• Megastudies: A New Approach to Reducing Vaccine Hesitation Worldwide

  2023cites this paper
• Psychological inoculation strategies to fight climate disinformation across 12 countries

  2023cites this paper
• Crowdsourcing as a tool for creating effective nudges: An example for financial oversubscription

  2023cites this paper
• AI, Behavioural Science, and Consumer Welfare

  2023cites this paper
• The Opportunities and Costs of AI in Behavioural Science

  2023cites this paper
• How Do Behavioral Approaches to Increase Savings Compare? Evidence from Multiple Interventions in the U.S. Army

  2022cites this paper
• Grand Challenges for Research on Privacy Documents

  year unknowncites this paper