Continual updating and monitoring of clinical prediction models: time for dynamic prediction systems?

Clinical prediction models (CPMs) have become fundamental for risk stratification across healthcare. The CPM pipeline (development, validation, deployment, and impact assessment) is commonly viewed as a one-time activity, with model updating rarely considered and done in a somewhat ad hoc manner. This fails to address the fact that the performance of a CPM worsens over time as natural changes in populations and care pathways occur. CPMs need constant surveillance to maintain adequate predictive performance. Rather than reactively updating a developed CPM once evidence of deteriorated performance accumulates, it is possible to proactively adapt CPMs whenever new data becomes available. Approaches for validation then need to be changed accordingly, making validation a continuous rather than a discrete effort. As such, “living” (dynamic) CPMs represent a paradigm shift, where the analytical methods dynamically generate updated versions of a model through time; one then needs to validate the system rather than each subsequent model revision.

• Publication year

  2021

• Venue

  Diagnostic and Prognostic Research

• Publication date

  2021-01-11

• Fields of study

  Medicine

• Identifiers
  DOI 10.1186/s41512-020-00090-3PMID 33431065PMCID 7797885
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal

  2020cited by this paper
• Temporal recalibration for improving prognostic model development and risk predictions in settings where survival is improving over time

  2020cited by this paper
• Validation and Utility Testing of Clinical Prediction Models: Time to Change the Approach.

  2020cited by this paper
• Performance of prediction models for COVID-19: the Caudine Forks of the external validation

  2020cited by this paper
• Changing predictor measurement procedures affected the performance of prediction models in clinical examples.

  2019cited by this paper
• Prognostic models will be victims of their own success, unless

  2019cited by this paper
• A review of statistical updating methods for clinical prediction models

  2018cited by this paper
• Dynamic models to predict health outcomes: current status and methodological challenges

  2018cited by this paper
• Calibration drift in regression and machine learning models for acute kidney injury

  2017cited by this paper
• Toward an Information Infrastructure for Global Health Improvement

  2017cited by this paper
• A guide to systematic review and meta-analysis of prediction model performance

  2017cited by this paper
• Development and validation of QRISK3 risk prediction algorithms to estimate future risk of cardiovascular disease: prospective cohort study

  2017cited by this paper
• A closed testing procedure to select an appropriate method for updating prediction models

  2017cited by this paper
• Clinical prediction in defined populations: a simulation study investigating when and how to aggregate existing models

  2017cited by this paper
• A multiple‐model generalisation of updating clinical prediction models

  2017cited by this paper
• External validation of clinical prediction models using big datasets from e-health records or IPD meta-analysis: opportunities and challenges

  2016cited by this paper
• Individual Participant Data (IPD) Meta-analyses of Diagnostic and Prognostic Modeling Studies: Guidance on Their Use

  2015cited by this paper
• Updated prognostic model for predicting overall survival in first-line chemotherapy for patients with metastatic castration-resistant prostate cancer.

  2014cited by this paper
• Dynamic Prediction Modeling Approaches for Cardiac Surgery

  2013cited by this paper
• Dynamic trends in cardiac surgery: why the logistic EuroSCORE is no longer suitable for contemporary cardiac surgery and implications for future risk models.

  2013cited by this paper
• Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research

  2013cited by this paper
• Prognosis research strategy (PROGRESS) 1: A framework for researching clinical outcomes

  2013cited by this paper
• Dynamic Logistic Regression and Dynamic Model Averaging for Binary Classification

  2012cited by this paper
• Aggregating published prediction models with individual participant data: a comparison of different approaches

  2012cited by this paper
• EuroSCORE II.

  2012cited by this paper
• External validity of risk models: Use of benchmark values to disentangle a case-mix effect from incorrect coefficients.

  2010cited by this paper
• Online Prediction Under Model Uncertainty via Dynamic Model Averaging: Application to a Cold Rolling Mill

  2010cited by this paper
• The Leicester Risk Assessment score for detecting undiagnosed Type 2 diabetes and impaired glucose regulation for use in a multiethnic UK setting

  2010cited by this paper
• Updating methods improved the performance of a clinical prediction model in new patients.

  2008cited by this paper
• Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2

  2008cited by this paper
• Statistical Methods with Varying Coefficient Models.

  2008cited by this paper
• Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study

  2007cited by this paper
• Improving Accessibility and Usability of Geo-referenced Statistical Data

  2003cited by this paper
• What do we mean by validating a prognostic model?

  2000cited by this paper
• European system for cardiac operative risk evaluation (EuroSCORE).

  1999cited by this paper
• Nonparametric smoothing estimates of time-varying coefficient models with longitudinal data

  1998cited by this paper
• Construction, validation and updating of a prognostic model for kidney graft survival.

  1995cited by this paper
• Present position and potential developments: some personal views

  1984cited by this paper

• Performance of the Leicester risk assessment and Leicester practice risk scores for assessing the risk of undiagnosed type 2 diabetes or prediabetes in diverse populations: protocol for a systematic review of published validations and updates

  2026cites this paper
• Artificial intelligence in functional gastrointestinal disorders: From precision diagnosis to preventive healthcare

  2026cites this paper
• A moving target in AI-assisted decision-making: dataset shift, model updating, and the problem of update opacity

  2025cites this paper
• Dynamic updating of psychosis prediction models in individuals at ultra high-risk of psychosis.

  2025cites this paper
• Predictive performance of risk prediction models for lung cancer incidence in Western and Asian countries: a systematic review and meta-analysis

  2025cites this paper
• Predictive modeling of methadone poisoning outcomes in children ≤ 5 years: utilizing machine learning and the National Poison Data System for improved clinical decision-making

  2025cites this paper
• Adverse drug reactions and events in an Ageing PopulaTion risk Prediction (ADAPTiP) tool: the development and validation of a model for predicting adverse drug reactions and events in older patients

  2025cites this paper
• Holding Ourselves to a Higher Standard: Dynamic Prediction Modeling in Cardiac Surgery.

  2025cites this paper
• Mortality Prediction Performance Under Geographical, Temporal, and COVID-19 Pandemic Dataset Shift: External Validation of the Global Open-Source Severity of Illness Score Model

  2025cites this paper
• The Efficacy of Bayesian Analysis in Predicting Stock Prices

  2025cites this paper
• Early Physiological and Neurological Monitoring after Traumatic Brain Injury as Foundations for Outcome Prediction: A Narrative Review

  2025cites this paper
• Evaluation of performance measures in predictive artificial intelligence models to support medical decisions: overview and guidance.

  2025cites this paper
• Enhancing bronchopulmonary dysplasia prediction in preterm infants using artificial intelligence and multimodal data integration

  2025cites this paper
• From Models to Management: A Framework for Predictive Analytics in Health Systems

  2025cites this paper
• Developing and validating a clinical prediction model to predict epilepsy-related emergency department attendance, hospital admission, or death: A cohort study protocol

  2025cites this paper
• Remaining Useful Life Prediction for Bearings Based on Generative Pretrained Transformer

  2025cites this paper
• Reversible Chemistry for Cancer Therapy and Diagnosis.

  2025cites this paper
• Development and evaluation of a lymph node invasion risk prediction model in intermediate- and high-risk prostate cancer patients

  2025cites this paper
• Improving lung cancer screening: the role and challenges of risk prediction models

  2025cites this paper
• External Validation of the NELA Prognostic Model in Emergency Abdominal Surgery

  2025cites this paper
• Hidden risks of predictive models in healthcare

  2025cites this paper
• Radiomics Quality Score 2.0: towards radiomics readiness levels and clinical translation for personalized medicine

  2025cites this paper
• Beyond One-Time Validation: A Framework for Adaptive Validation of Prognostic and Diagnostic AI-based Medical Devices

  2024cites this paper
• Risk Prediction Models as an Emerging Trend for Managing Cancer-Related Fatigue: A Systematic Review.

  2024cites this paper
• AI in Healthcare: Bridging the Gap between Research and Clinical Implementation

  2024cites this paper
• Nationwide study on development and validation of a risk prediction model for CIN3+ and cervical cancer in Estonia

  2024cites this paper
• Multicentre Validation of the RESECT-90 Prediction Model for 90-Day Mortality After Lung Resection.

  2024cites this paper
• MoCab: A framework for the deployment of machine learning models across health information systems

  2024cites this paper
• Evaluation of clinical prediction models (part 1): from development to external validation

  2024cites this paper
• CEL: A Continual Learning Model for Disease Outbreak Prediction by Leveraging Domain Adaptation via Elastic Weight Consolidation

  2024cites this paper
• External validation of six COVID-19 prognostic models for predicting mortality risk in older populations in a hospital, primary care, and nursing home setting.

  2024cites this paper
• Ethical considerations when planning, implementing and releasing health economic model software: a new proposal

  2024cites this paper
• External validation and update of the International Medical Prevention Registry on Venous Thromboembolism bleeding risk score for predicting bleeding in acutely ill hospitalized medical patients: a retrospective single-center cohort study in Japan

  2024cites this paper
• Computable phenotype for real-world, data-driven retrospective identification of relapse in ANCA-associated vasculitis

  2024cites this paper
• Digital Mental Health for Schizophrenia and Other Severe Mental Illnesses: An International Consensus on Current Challenges and Potential Solutions

  2024cites this paper
• Determining Outcomes in Nonpenetrating Trauma With qSOFA.

  2024cites this paper
• Enhancing Postmarketing Surveillance of Medical Products With Large Language Models.

  2024cites this paper
• The PROgnostic ModEl for chronic lung disease (PRO-MEL): development and temporal validation

  2024cites this paper
• Clinician perspectives and recommendations regarding design of clinical prediction models for deteriorating patients in acute care

  2024cites this paper
• Developing clinical prediction models: a step-by-step guide

  2024cites this paper
• Seeing the random forest through the decision trees. Supporting learning health systems from histopathology with machine learning models: Challenges and opportunities

  2023cites this paper
• Severity of Illness Scores and Biomarkers for Prognosis of Patients with Coronavirus Disease 2019

  2023cites this paper
• Predicted probabilities of live birth following assisted reproductive technology using United States national surveillance data from 2016-2018.

  2023cites this paper
• Taking shared decision making for prostate cancer to the next level: Requirements for a Dutch treatment decision aid with personalized risks on side effects

  2023cites this paper
• An integrated approach to geographic validation helped scrutinize prediction model performance and its variability.

  2023cites this paper
• Risk factors for severe respiratory syncytial virus infection during the first year of life: development and validation of a clinical prediction model

  2023cites this paper
• There is no such thing as a validated prediction model

  2023cites this paper
• Group B Streptococcus Early-Onset Disease: New Preventive and Diagnostic Tools to Decrease the Burden of Antibiotic Use

  2023cites this paper
• Into the trees: random forests for predicting Fusarium head blight epidemics of wheat in the United States.

  2023cites this paper
• Dynamic Updating of Clinical Survival Prediction Models in a Rapidly Changing Environment

  2023influential citation
• A prediction risk score for HIV among adolescent girls and young women in South Africa: identifying those in need of HIV pre-exposure prophylaxis

  2023cites this paper
• Six Externally Validated Prognostic Models Have Potential Clinical Value to Predict Patient Health Outcomes in the Rehabilitation of Musculoskeletal Conditions: A Systematic Review.

  2023cites this paper
• Subacute prognosis of oral nutrition (SPOON): Development of a multivariable prognostic model for complete oral intake in tube-fed subjects with acquired brain injury.

  2023cites this paper
• Updating Clinical Risk Stratification Models Using Rank-Based Compatibility: Approaches for Evaluating and Optimizing Clinician-Model Team Performance

  2023cites this paper
• Ethics framework for predictive clinical AI model updating

  2023cites this paper
• Using temporal recalibration to improve the calibration of risk prediction models in competing risk settings when there are trends in survival over time

  2023cites this paper
• Implications of the Use of Artificial Intelligence Predictive Models in Health Care Settings

  2023cites this paper
• Personalized dynamic super learning: an application in predicting hemodiafiltration convection volumes

  2023cites this paper
• Automatic correction of performance drift under acquisition shift in medical image classification

  2023cites this paper
• Opportunities and Challenges of Prognostic Models for Extremely Preterm Infants

  2023cites this paper
• Fixing the Leaky Pipe: How to Improve the Uptake of Patient-Reported Outcomes–Based Prognostic and Predictive Models in Cancer Clinical Practice

  2023cites this paper
• An external validation of the Kidney Donor Risk Index in the UK transplant population in the presence of semi-competing events

  2023cites this paper
• Dynamic updating of clinical survival prediction models in a changing environment

  2023cites this paper
• Validation of the Trauma and Injury Severity Score for Prediction of Mortality in a Greek Trauma Population

  2022cites this paper
• Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review

  2022cites this paper
• Methodological guidance for the evaluation and updating of clinical prediction models: a systematic review

  2022cites this paper
• Prognostic models for COVID-19 needed updating to warrant transportability over time and space

  2022cites this paper
• Identification of clinical factors related to prediction of alcohol use disorder from electronic health records using feature selection methods

  2022cites this paper
• Development and validation of self-monitoring auto-updating prognostic models of survival for hospitalized COVID-19 patients

  2022cites this paper
• Building the Model.

  2022cites this paper
• Can the application of machine learning to electronic health records guide antibiotic prescribing decisions for suspected urinary tract infection in the Emergency Department?

  2022cites this paper
• A hybrid adaptive approach for instance transfer learning with dynamic and imbalanced data

  2022cites this paper
• Open questions and research gaps for monitoring and updating AI-enabled tools in clinical settings

  2022cites this paper
• Risk scores in anaesthesia: the future is hard to predict

  2022influential citation
• Predicting outcome of patients with prolonged disorders of consciousness using machine learning models based on medical complexity

  2022cites this paper
• Is your clinical prediction model past its sell by date?

  2022influential citation
• Clinical prediction models for mortality in patients with covid-19: external validation and individual participant data meta-analysis

  2022cites this paper
• Clinical and radiomics prediction of complete response in rectal cancer pre-chemoradiotherapy

  2022cites this paper
• Developing, validating, updating and judging the impact of prognostic models for respiratory diseases

  2022cites this paper
• Challenges in realising the potential of wastewater-based epidemiology to quantitatively monitor and predict the spread of disease.

  2022cites this paper
• ACCEPT 2·0: Recalibrating and externally validating the Acute COPD exacerbation prediction tool (ACCEPT)

  2022cites this paper
• Rethinking Autism Intervention Science: A Dynamic Perspective

  2022cites this paper
• Integrating landmark modeling framework and machine learning algorithms for dynamic prediction of tuberculosis treatment outcomes

  2022cites this paper
• Big data and predictive modelling for the opioid crisis: existing research and future potential.

  2021cites this paper
• Advanced cardiovascular risk prediction in the emergency department: updating a clinical prediction model – a large database study protocol

  2021cites this paper
• Validation and comparison of 28 risk prediction models for coronary artery disease.

  2021cites this paper
• Evaluation of Prognostic and Predictive Models in the Oncology Clinic.

  2021cites this paper
• Dynamic prediction models improved the risk classification of type 2 diabetes compared with classical static models.

  2021cites this paper
• A prototype software framework for transparent, reusable and updatable computational health economic models

  year unknowncites this paper
• EDITORIAL TEAM

  year unknowncites this paper