Trust Me, I Know the Way: Predictive Uncertainty in the Presence of Shortcut Learning

The correct way to quantify predictive uncertainty in neural networks remains a topic of active discussion. In particular, it is unclear whether the state-of-the art entropy decomposition leads to a meaningful representation of model, or epistemic, uncertainty (EU) in the light of a debate that pits ignorance against disagreement perspectives. We aim to reconcile the conflicting viewpoints by arguing that both are valid but arise from different learning situations. Notably, we show that the presence of shortcuts is decisive for EU manifesting as disagreement.

• Publication year

  2025

• Venue

  arXiv.org

• Publication date

  2025-02-13

• Fields of study

  Physics, Philosophy, Computer Science

• Identifiers
  DOI 10.48550/arXiv.2502.09137arXiv 2502.09137
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Partial Transportability for Domain Generalization

  2025cited by this paper
• Rethinking Aleatoric and Epistemic Uncertainty

  2024cited by this paper
• Implicitly Bayesian Prediction Rules in Deep Learning

  2024cited by this paper
• The Epistemic Uncertainty Hole: an issue of Bayesian Neural Networks

  2024cited by this paper
• Label-wise Aleatoric and Epistemic Uncertainty Quantification

  2024cited by this paper
• Mitigating Overconfidence in Out-of-Distribution Detection by Capturing Extreme Activations

  2024cited by this paper
• Robust agents learn causal world models

  2024cited by this paper
• Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks

  2024cited by this paper
• A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methods

  2023cited by this paper
• Mitigating Shortcut Learning with Diffusion Counterfactuals and Diverse Ensembles

  2023cited by this paper
• Spuriosity Didn't Kill the Classifier: Using Invariant Predictions to Harness Spurious Features

  2023cited by this paper
• Prediction-Oriented Bayesian Active Learning

  2023cited by this paper
• Monitoring Shortcut Learning using Mutual Information

  2022cited by this paper
• A Typology for Exploring the Mitigation of Shortcut Behavior

  2022cited by this paper
• Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?

  2022cited by this paper
• Agree to Disagree: Diversity through Disagreement for Better Transferability

  2022cited by this paper
• On Calibration and Out-of-domain Generalization

  2021cited by this paper
• Towards Causal Representation Learning

  2021cited by this paper
• Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization

  2021cited by this paper
• A survey of uncertainty in deep neural networks

  2021cited by this paper
• Shortcut learning in deep neural networks

  2020cited by this paper
• Understanding the Failure Modes of Out-of-Distribution Generalization

  2020cited by this paper
• Bayesian Deep Learning and a Probabilistic Perspective of Generalization

  2020influential reference
• Communicating uncertainty about facts, numbers and science

  2019cited by this paper
• Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods

  2019cited by this paper
• PyTorch: An Imperative Style, High-Performance Deep Learning Library

  2019cited by this paper
• On Calibration of Modern Neural Networks

  2017cited by this paper
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?

  2017cited by this paper
• Uncertainty, information, and disagreement of economic forecasters

  2017cited by this paper
• Decoupled Weight Decay Regularization

  2017cited by this paper
• Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning

  2017cited by this paper
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles

  2016cited by this paper
• Visual Causal Feature Learning

  2014cited by this paper
• Bayesian Active Learning for Classification and Preference Learning

  2011cited by this paper
• Assessment and Propagation of Model Uncertainty

  2011cited by this paper
• A Mathematical Theory of Communication

  2006cited by this paper
• An Introduction to MCMC for Machine Learning

  2004cited by this paper
• Bayesian model averaging is not model combination

  2002cited by this paper
• Gradient-based learning applied to document recognition

  1998cited by this paper
• Representing partial ignorance

  1996cited by this paper
• Quantifying Aleatoric and Epistemic Uncertainty: A Credal Approach

  year unknowncited by this paper

• Invariance Pair-Guided Learning: Enhancing Robustness in Neural Networks

  2025cites this paper