The Current Landscape in Biostatistics of Real-World Data and Evidence: Clinical Study Design and Analysis

Abstract Real-world data (RWD), such as electronic health records, reimbursement requests as adjudicated by health insurance companies, and health survey data as collected by government agencies or other research organizations, are increasingly used in drug development. Regulatory agencies, public-private partnerships, and professional organizations have initiated major programs and released guidance or guidelines to address challenges in the use of real-world evidence (RWE) to inform regulatory decision making. Since statistical research and considerations in the design, analysis, and interpretation of RWE studies is still a nascent area, the ASA Biopharmaceutical Section established a scientific working group to survey the existing practices and identify opportunities in the area. This article reviews the current landscape of using external data such as RWD/RWE to inform clinical study design and analysis. Two companion articles review the landscape of using RWE in medical product label expansion and of causal inference frameworks to generate RWE.

• Publication year

  2021

• Venue

  Statistics in Biopharmaceutical Research

• Publication date

  2021-03-15

• Fields of study

  Biology, Medicine, Business

• Identifiers
  DOI 10.1080/19466315.2021.1883474
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• The Current Landscape in Biostatistics of Real-World Data and Evidence: Causal Inference Frameworks for Study Design and Analysis

  2021cited by this paper
• Use of a rare disease registry for establishing phenotypic classification of previously unassigned GLA variants: a consensus classification system by a multispecialty Fabry disease genotype–phenotype workgroup

  2020cited by this paper
• A roadmap to using historical controls in clinical trials – by Drug Information Association Adaptive Design Scientific Working Group (DIA-ADSWG)

  2020cited by this paper
• Synthetic and External Controls in Clinical Trials – A Primer for Researchers

  2020cited by this paper
• Beyond Randomized Clinical Trials: Use of External Controls

  2019cited by this paper
• Synthesizing existing evidence to design future trials: survey of methodologists from European institutions

  2019influential reference
• Design of randomized controlled confirmatory trials using historical control data to augment sample size for concurrent controls

  2019cited by this paper
• The dark side of the force: Multiplicity issues in network meta‐analysis and how to address them

  2019influential reference
• Power gains by using external information in clinical trials are typically not possible when requiring strict type I error control

  2019cited by this paper
• Principles of confounder selection

  2019cited by this paper
• Characteristics of non-randomised studies using comparisons with external controls submitted for regulatory approval in the USA and Europe: a systematic review

  2019cited by this paper
• Analysing data and undertaking meta‐analyses

  2019cited by this paper
• REQUIREMENTS FOR PHARMACEUTICALS FOR HUMAN USE

  2019cited by this paper
• Borrowing From Historical Control Data in Cancer Drug Development: A Cautionary Tale and Practical Guidelines

  2019cited by this paper
• Incorporating single‐arm evidence into a network meta‐analysis using aggregate level matching: Assessing the impact

  2019cited by this paper
• Use of Health Care Databases to Support Supplemental Indications of Approved Medications

  2018cited by this paper
• On the propensity score weighting analysis with survival outcome: Estimands, estimation, and inference

  2018cited by this paper
• Covariate selection with group lasso and doubly robust estimation of causal effects

  2018cited by this paper
• An overview of methods for network meta-analysis using individual participant data: when do benefits arise?

  2018cited by this paper
• Use of a historical control group in a noninferiority trial assessing a new antibacterial treatment: A case study and discussion of practical implementation aspects

  2018cited by this paper
• Power and commensurate priors for synthesizing aggregate and individual patient level data in network meta‐analysis

  2018cited by this paper
• Improving the efficiency of clinical trial designs by using historical control data or adding a treatment arm to an ongoing trial

  2018influential reference
• Methods for the inclusion of real-world evidence in network meta-analysis

  2018cited by this paper
• DELTA2 guidance on choosing the target difference and undertaking and reporting the sample size calculation for a randomised controlled trial

  2018cited by this paper
• Rare disease registries: potential applications towards impact on development of new drug treatments

  2018cited by this paper
• The inclusion of real world evidence in clinical development planning

  2018cited by this paper
• Minimizing Patient Burden Through the Use of Historical Subject-Level Data in Innovative Confirmatory Clinical Trials: Review of Methods and Opportunities

  2018cited by this paper
• When and How Can Real World Data Analyses Substitute for Randomized Controlled Trials?

  2017cited by this paper
• Good practices for real‐world data studies of treatment and/or comparative effectiveness: Recommendations from the joint ISPOR‐ISPE Special Task Force on real‐world evidence in health care decision making

  2017cited by this paper
• Work Package 4 Methodological guidance , recommendations and illustrative case studies for ( network ) meta-analysis and modelling to predict real-world effectiveness using individual participant and / or aggregate data

  2017cited by this paper
• Multivariate and network meta-analysis of multiple outcomes and multiple treatments: rationale, concepts, and examples

  2017influential reference
• Illustrating Informed Presence Bias in Electronic Health Records Data: How Patient Interactions with a Health System Can Impact Inference

  2017cited by this paper
• Outcome‐adaptive lasso: Variable selection for causal inference

  2017cited by this paper
• Combining randomized and non‐randomized evidence in network meta‐analysis

  2017influential reference
• GetReal in network meta‐analysis: a review of the methodology

  2016influential reference
• Regulatory approval of pharmaceuticals without a randomised controlled study: analysis of EMA and FDA approvals 1999–2014

  2016cited by this paper
• “Threshold‐crossing”: A Useful Way to Establish the Counterfactual in Clinical Trials?

  2016influential reference
• Sample size considerations for historical control studies with survival outcomes

  2016cited by this paper
• Association of Pathologic Complete Response to Neoadjuvant Therapy in HER2-Positive Breast Cancer With Long-Term Outcomes: A Meta-Analysis.

  2016cited by this paper
• Propensity score and proximity matching using random forest.

  2016cited by this paper
• Using Big Data to Emulate a Target Trial When a Randomized Trial Is Not Available.

  2016cited by this paper
• THE USE OF REAL WORLD DATA FOR THE ESTIMATION OF TREATMENT EFFECTS IN NICE DECISION MAKING

  2016influential reference
• Meta-analysis in clinical trials revisited.

  2015cited by this paper
• Innovative Interventional and Imaging Registries: Precision Medicine in Cerebrovascular Disorders

  2015cited by this paper
• Get real in individual participant data (IPD) meta‐analysis: a review of the methodology

  2015influential reference
• The PRISMA Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-analyses of Health Care Interventions: Checklist and Explanations

  2015cited by this paper
• Meta-analysis of randomized phase II trials to inform subsequent phase III decisions

  2014cited by this paper
• Evaluating the Quality of Evidence from a Network Meta-Analysis.

  2014influential reference
• A comparison of national guidelines for network meta-analysis.

  2014cited by this paper
• Robust meta‐analytic‐predictive priors in clinical trials with historical control information

  2014cited by this paper
• Designing Premarket Observational Comparative Studies Using Existing Data as Controls: Challenges and Opportunities

  2014influential reference
• Use of historical control data for assessing treatment effects in clinical trials

  2014influential reference
• A Distribution-based Method for Assessing The Differences between Clinical Trial Target Populations and Patient Populations in Electronic Health Records

  2014cited by this paper
• Evaluating the Quality of Evidence from a Network Meta-Analysis

  2014influential reference
• Balancing Covariates via Propensity Score Weighting

  2014cited by this paper
• Evidence Synthesis for Decision Making 4

  2013cited by this paper
• Evaluating the impact of database heterogeneity on observational study results.

  2013cited by this paper
• Evidence Synthesis for Decision Making 2

  2013cited by this paper
• Prognostic score-based balance measures can be a useful diagnostic for propensity score methods in comparative effectiveness research.

  2013cited by this paper
• Evidence Synthesis for Decision Making 6

  2013cited by this paper
• Evidence Synthesis for Decision Making 7

  2013cited by this paper
• Combining individual patient data and aggregate data in mixed treatment comparison meta‐analysis: Individual patient data may be beneficial if only for a subset of trials

  2013influential reference
• Evidence Synthesis for Decision Making 3

  2013cited by this paper
• Matching and Propensity Scores

  2013cited by this paper
• Evidence Synthesis for Decision Making 1

  2013cited by this paper
• Adaptive adjustment of the randomization ratio using historical control data

  2013cited by this paper
• Prospective observational studies to assess comparative effectiveness: the ISPOR good research practices task force report.

  2012cited by this paper
• Network meta‐analysis of individual and aggregate level data

  2012influential reference
• Conducting quantitative synthesis when comparing medical interventions: AHRQ and the Effective Health Care Program.

  2011cited by this paper
• Causal Diagrams

  2011cited by this paper
• Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 2.

  2011influential reference
• Hierarchical Commensurate and Power Prior Models for Adaptive Incorporation of Historical Information in Clinical Trials

  2011cited by this paper
• Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1.

  2011cited by this paper
• A Bayesian model for repeated measures zero-inflated count data with application to outpatient psychiatric service use

  2010cited by this paper
• Matching methods for causal inference: A review and a look forward.

  2010influential reference
• Summarizing historical information on controls in clinical trials

  2010cited by this paper
• Cochrane Handbook for Systematic Reviews of Interventions

  2010cited by this paper
• Calculating sample size in trials using historical controls

  2010cited by this paper
• Causal inference in statistics: An overview

  2009cited by this paper
• Good research practices for comparative effectiveness research: defining, reporting and interpreting nonrandomized studies of treatment effects using secondary data sources: the ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report--Part I.

  2009cited by this paper
• Patient Registries of Acute Coronary Syndrome: Assessing or Biasing the Clinical Real World Data?

  2009cited by this paper
• Good research practices for comparative effectiveness research: approaches to mitigate bias and confounding in the design of nonrandomized studies of treatment effects using secondary data sources: the International Society for Pharmacoeconomics and Outcomes Research Good Research Practices for Retr

  2009cited by this paper
• A note on the power prior

  2009cited by this paper
• Good research practices for comparative effectiveness research: analytic methods to improve causal inference from nonrandomized studies of treatment effects using secondary data sources: the ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report--Part III.

  2009cited by this paper
• For objective causal inference, design trumps analysis

  2008influential reference
• Matching With Multiple Control Groups With Adjustment for Group Differences

  2008cited by this paper
• The prognostic analogue of the propensity score

  2008cited by this paper
• Misunderstandings between experimentalists and observationalists about causal inference

  2008cited by this paper
• The design versus the analysis of observational studies for causal effects: parallels with the design of randomized trials

  2007cited by this paper
• Evidence‐based sample size calculations based upon updated meta‐analysis

  2007cited by this paper
• Variable selection for propensity score models.

  2006cited by this paper
• Moving the Goalposts: Addressing Limited Overlap in Estimation of Average Treatment Effects by Changing the Estimand

  2006cited by this paper
• Using power priors to improve the binomial test of water quality

  2006cited by this paper
• Incorporating Bayesian Ideas into Health-Care Evaluation

  2004influential reference
• Combination of direct and indirect evidence in mixed treatment comparisons

  2004cited by this paper
• Full Matching in an Observational Study of Coaching for the SAT

  2004cited by this paper
• Retrospective data collection and analytical techniques for patient safety studies

  2003cited by this paper
• Explaining the sex difference in coronary heart disease mortality among patients with type 2 diabetes mellitus: a meta-analysis.

  2002cited by this paper
• Sample size calculations for randomized controlled trials.

  2002influential reference
• Substantial evidence of effect

  2002cited by this paper

• Additional Considerations for Single-Arm Trials to Support Accelerated Approval of Oncology Drugs

  2025cites this paper
• Real-World Data and Causal Machine Learning to Enhance Drug Development

  2025cites this paper
• rdborrow: an R package for causal inference incorporating external controls in randomized controlled trials with longitudinal outcomes

  2025cites this paper
• The Current Landscape of HTA Framework and Key Challenges

  2025cites this paper
• Use of Real‐World Data and Real‐World Evidence in Rare Disease Drug Development: A Statistical Perspective

  2024cites this paper
• Robust transfer learning for individualized treatment rules in the presence of missing data.

  2024cites this paper
• Causal estimators for incorporating external controls in randomized trials with longitudinal outcomes

  2024cites this paper
• Considerations for Single-Arm Trials to Support Accelerated Approval of Oncology Drugs

  2024cites this paper
• A Causal Inference Methodology to Support Research on Osteopenia for Breast Cancer Patients

  2024cites this paper
• Research on the Application of Computer Technology in Biostatistics

  2024cites this paper
• Leveraging Real-World Data in COVID-19 Response

  2023cites this paper
• Current Perspectives for External Control Arms in Oncology Clinical Trials: Analysis of EMA approvals 2016-2021.

  2023cites this paper
• Editor’s Note: Special Section on a Collection of Articles on Opportunities and Challenges in Utilizing Real-World Data for Clinical Trials and Medical Product Development

  2023cites this paper
• Considerations for master protocols using external controls

  2023cites this paper
• Considerations for pooling real-world data as a comparator cohort to a single arm trial: a simulation study on assessment of heterogeneity

  2023cites this paper
• Estimating treatment effects from single-arm trials via latent-variable modeling

  2023cites this paper
• Comparative effectiveness of tocilizumab vs standard care in patients with severe COVID-19-related pneumonia: a retrospective cohort study utilizing registry data as a synthetic control

  2023cites this paper
• Rejoinder to the Commentaries on “The Current Landscape in Biostatistics of Real-World Data and Evidence: Label Expansion”

  2022cites this paper
• Clinical Trials with External Control: Beyond Propensity Score Matching

  2022cites this paper
• Exploring the Potential of External Control Arms created from Patient Level Data: A case study in non-small cell lung cancer

  2022cites this paper
• Increasing patient participation in oncology clinical trials

  2022cites this paper
• Applying Quantitative Approaches in the Use of RWE in Clinical Development and Life-Cycle Management

  2021cites this paper
• The Current Landscape in Biostatistics of Real-World Data and Evidence: Causal Inference Frameworks for Study Design and Analysis

  2021cites this paper
• Propensity‐score‐based meta‐analytic predictive prior for incorporating real‐world and historical data

  2021influential citation
• Some Biostatistical Considerations About Real-World Data and Evidence in Clinical Studies, Especially for Regulatory Purposes

  2021influential citation
• Comment on “Biostatistical Considerations When Using RWD and RWE in Clinical Studies for Regulatory Purposes: A Landscape Assessment”

  2021cites this paper
• Comment on “Biostatistical Considerations When Using RWD and RWE in Clinical Studies for Regulatory Purposes: A Landscape Assessment”

  2021influential citation