Uncertainty quantification for White Matter Hyperintensity segmentation detects silent failures and improves automated Fazekas quantification

White Matter Hyperintensities (WMH) are key neuroradiological markers of small vessel disease present in brain MRI. Assessment of WMH is important in research and clinics. However, WMH are challenging to segment due to their high variability in shape, location, size, poorly defined borders, and similar intensity profile to other pathologies (e.g stroke lesions) and artefacts (e.g head motion). In this work, we assess the utility and semantic properties of the most effective techniques for uncertainty quantification (UQ) in segmentation for the WMH segmentation task across multiple test-time data distributions. We find UQ techniques reduce 'silent failure' by identifying in UQ maps small WMH clusters in the deep white matter that are unsegmented by the model. A combination of Stochastic Segmentation Networks with Deep Ensembles also yields the highest Dice and lowest Absolute Volume Difference % (AVD) score and can highlight areas where there is ambiguity between WMH and stroke lesions. We further demonstrate the downstream utility of UQ, proposing a novel method for classification of the clinical Fazekas score using spatial features extracted from voxelwise WMH probability and UQ maps. We show that incorporating WMH uncertainty information improves Fazekas classification performance and calibration. Our model with (UQ and spatial WMH features)/(spatial WMH features)/(WMH volume only) achieves a balanced accuracy score of 0.74/0.67/0.62, and root brier score (↓) of 0.65/0.72/0.74 in the Deep WMH and balanced accuracy of 0.74/0.73/0.71 and root brier score of 0.64/0.66/0.68 in the Periventricular region. We further demonstrate that stochastic UQ techniques with high sample diversity can improve the detection of poor quality segmentations.

• Publication year

  2024

• Venue

  Medical Image Anal.

• Publication date

  2024-11-26

• Fields of study

  Medicine, Computer Science, Engineering

• Identifiers
  DOI 10.1016/j.media.2025.103697arXiv 2411.17571PMID 40680569
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Stochastic Uncertainty Quantification Techniques Fail to Account for Inter-analyst Variability in White Matter Hyperintensity Segmentation

  2024influential reference
• Variational Bayesian Last Layers

  2024cited by this paper
• Denoising and uncertainty estimation in parameter mapping with approximate Bayesian deep image priors

  2023cited by this paper
• Artificial intelligence for diagnostic and prognostic neuroimaging in dementia: A systematic review

  2023cited by this paper
• Challenges of implementing computer-aided diagnostic models for neuroimages in a clinical setting

  2023cited by this paper
• Robust T-Loss for Medical Image Segmentation

  2023cited by this paper
• Deep Anti-Regularized Ensembles provide reliable out-of-distribution uncertainty quantification

  2023cited by this paper
• Implicit Anatomical Rendering for Medical Image Segmentation with Stochastic Experts

  2023cited by this paper
• Deep learning-based grading of white matter hyperintensities enables identification of potential markers in multi-sequence MRI data

  2023cited by this paper
• A Review of Uncertainty Estimation and its Application in Medical Imaging

  2023cited by this paper
• EvidenceCap: Towards trustworthy medical image segmentation via evidential identity cap

  2023cited by this paper
• Structural-Based Uncertainty in Deep Learning Across Anatomical Scales: Analysis in White Matter Lesion Segmentation

  2023cited by this paper
• Proper Scoring Loss Functions Are Simple and Effective for Uncertainty Quantification of White Matter Hyperintensities

  2023cited by this paper
• A real-world clinical validation for AI-based MRI monitoring in multiple sclerosis

  2023cited by this paper
• Application of uncertainty quantification to artificial intelligence in healthcare: A review of last decade (2013-2023)

  2023cited by this paper
• Neuroimaging in the Era of Artificial Intelligence: Current Applications.

  2022cited by this paper
• Impact of different white matter hyperintensities patterns on cognition: A cross-sectional and longitudinal study

  2022cited by this paper
• Deep Ensembles Work, But Are They Necessary?

  2022cited by this paper
• Volumetric White Matter Hyperintensity Ranges Correspond to Fazekas Scores on Brain MRI.

  2022cited by this paper
• Effects of white matter hyperintensities distribution and clustering on late-life cognitive impairment

  2022cited by this paper
• COVID-19 Infection Segmentation from Chest CT Images Based on Scale Uncertainty

  2022cited by this paper
• Improving robustness and calibration in ensembles with diversity regularization

  2022cited by this paper
• Brain tumour segmentation with incomplete imaging data

  2022cited by this paper
• Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysis

  2022cited by this paper
• Diagnostic performance of deep learning-based automatic white matter hyperintensity segmentation for classification of the Fazekas scale and differentiation of subcortical vascular dementia

  2022cited by this paper
• Ensemble learning via supervision augmentation for white matter hyperintensity segmentation

  2022cited by this paper
• Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?

  2022cited by this paper
• Periventricular rather than deep white matter hyperintensities mediate effects of hypertension on cognitive performance in the population-based 1000BRAINS study

  2022cited by this paper
• Advances, challenges and opportunities in creating data for trustworthy AI

  2022cited by this paper
• Region-based evidential deep learning to quantify uncertainty and improve robustness of brain tumor segmentation

  2022cited by this paper
• Generalized Probabilistic U-Net for medical image segementation

  2022cited by this paper
• TBraTS: Trusted Brain Tumor Segmentation

  2022cited by this paper
• Disentangling Epistemic and Aleatoric Uncertainty in Reinforcement Learning

  2022cited by this paper
• A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awareness

  2022cited by this paper
• Trusted Multi-View Classification With Dynamic Evidential Fusion

  2022cited by this paper
• Artificial Intelligence in Neuroimaging: Clinical Applications

  2022cited by this paper
• Deep Deterministic Uncertainty for Semantic Segmentation

  2021cited by this paper
• Exploring the effect of image enhancement techniques on COVID-19 detection using chest X-ray images

  2021cited by this paper
• Analyzing Epistemic and Aleatoric Uncertainty for Drusen Segmentation in Optical Coherence Tomography Images

  2021cited by this paper
• Robust White Matter Hyperintensity Segmentation On Unseen Domain

  2021cited by this paper
• Is segmentation uncertainty useful?

  2021influential reference
• Uncertainty Reduction for Model Adaptation in Semantic Segmentation

  2021cited by this paper
• Ensemble deep learning: A review

  2021cited by this paper
• White matter hyperintensities segmentation using the ensemble U-Net with multi-scale highlighting foregrounds

  2021influential reference
• Deterministic Neural Networks with Inductive Biases Capture Epistemic and Aleatoric Uncertainty

  2021cited by this paper
• On Stein Variational Neural Network Ensembles

  2021cited by this paper
• Improving Aleatoric Uncertainty Quantification in Multi-Annotated Medical ImageSegmentation with Normalizing Flows

  2021cited by this paper
• Deep Bayesian networks for uncertainty estimation and adversarial resistance of white matter hyperintensity segmentation

  2021cited by this paper
• Trustworthy AI: From Principles to Practices

  2021cited by this paper
• A Survey on Evidential Deep Learning For Single-Pass Uncertainty Estimation

  2021cited by this paper
• Calibrating the Dice Loss to Handle Neural Network Overconfidence for Biomedical Image Segmentation

  2021cited by this paper
• A National Network of Safe Havens: Scottish Perspective

  2021cited by this paper
• Deep Deterministic Uncertainty: A New Simple Baseline

  2021cited by this paper
• SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining

  2021cited by this paper
• Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation

  2021cited by this paper
• Predicting Fazekas scores from automatic segmentations of white matter signal abnormalities

  2020cited by this paper
• Machine intelligence in healthcare—perspectives on trustworthiness, explainability, usability, and transparency

  2020cited by this paper
• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges

  2020influential reference
• Automatic segmentation of white matter hyperintensities from brain magnetic resonance images in the era of deep learning and big data - A systematic review

  2020cited by this paper
• Uncertainty modelling in deep learning for safer neuroimage enhancement: Demonstration in diffusion MRI

  2020cited by this paper
• Uncertainty Estimation in Medical Image Denoising with Bayesian Deep Image Prior

  2020influential reference
• Neural Ensemble Search for Uncertainty Estimation and Dataset Shift

  2020cited by this paper
• Rationale and design of a longitudinal study of cerebral small vessel diseases, clinical and imaging outcomes in patients presenting with mild ischaemic stroke: Mild Stroke Study 3

  2020influential reference
• Stochastic Segmentation Networks: Modelling Spatially Correlated Aleatoric Uncertainty

  2020influential reference
• Analyzing the Quality and Challenges of Uncertainty Estimations for Brain Tumor Segmentation

  2020cited by this paper
• nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation

  2020cited by this paper
• Supervised Uncertainty Quantification for Segmentation with Multiple Annotations

  2019cited by this paper
• An improved algorithm of white matter hyperintensity detection in elderly adults

  2019cited by this paper
• A Hierarchical Probabilistic U-Net for Modeling Multi-Scale Ambiguities

  2019cited by this paper
• Deep Evidential Regression

  2019cited by this paper
• Automatic spatial estimation of white matter hyperintensities evolution in brain MRI using disease evolution predictor deep neural networks

  2019cited by this paper
• Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge

  2019cited by this paper
• The need for uncertainty quantification in machine-assisted medical decision making

  2019cited by this paper
• Building Trust in Deep Learning System towards Automated Disease Detection

  2019cited by this paper
• Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift

  2019cited by this paper
• Evaluating the Impact of Intensity Normalization on MR Image Synthesis

  2018cited by this paper
• Do 2-year changes in superior frontal gyrus and global brain atrophy affect cognition?

  2018cited by this paper
• Considerations on accuracy, pattern and possible underlying factors of brain microbleed progression in older adults with absence or mild presence of vascular pathology

  2018cited by this paper
• Evaluating Bayesian Deep Learning Methods for Semantic Segmentation

  2018cited by this paper
• Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks

  2018cited by this paper
• A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks

  2018cited by this paper
• Combo Loss: Handling Input and Output Imbalance in Multi-Organ Segmentation

  2018cited by this paper
• A Probabilistic U-Net for Segmentation of Ambiguous Images

  2018cited by this paper
• Evidential Deep Learning to Quantify Classification Uncertainty

  2018cited by this paper
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?

  2017cited by this paper
• Classification and characterization of periventricular and deep white matter hyperintensities on MRI: A study in older adults

  2017cited by this paper
• A survey on deep learning in medical image analysis

  2017cited by this paper
• Variational Inference: A Review for Statisticians

  2016cited by this paper
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles

  2016cited by this paper
• Weight Uncertainty in Neural Network

  2015cited by this paper
• Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networks

  2015cited by this paper
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

  2015cited by this paper
• Dropout: a simple way to prevent neural networks from overfitting

  2014cited by this paper
• Morphologic, Distributional, Volumetric, and Intensity Characterization of Periventricular Hyperintensities

  2014cited by this paper
• Neuroimaging standards for research into small vessel disease and its contribution to ageing and neurodegeneration

  2013cited by this paper
• Human neuroimaging as a “Big Data” science

  2013cited by this paper
• White Matter Lesion Assessment in Patients with Cognitive Impairment and Healthy Controls: Reliability Comparisons between Visual Rating, a Manual, and an Automatic Volumetrical MRI Method—The Gothenburg MCI Study

  2013cited by this paper
• Energy statistics: A class of statistics based on distances

  2013cited by this paper
• The Design of SimpleITK

  2013cited by this paper
• Close Correlation between Quantitative and Qualitative Assessments of White Matter Lesions

  2012cited by this paper

• No citing papers are available for this paper.