High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach

This paper considers the generation of prediction intervals (PIs) by neural networks for quantifying uncertainty in regression tasks. It is axiomatic that high-quality PIs should be as narrow as possible, whilst capturing a specified portion of data. We derive a loss function directly from this axiom that requires no distributional assumption. We show how its form derives from a likelihood principle, that it can be used with gradient descent, and that model uncertainty is accounted for in ensembled form. Benchmark experiments show the method outperforms current state-of-the-art uncertainty quantification methods, reducing average PI width by over 10%.

• Publication year

  2018

• Venue

  International Conference on Machine Learning

• Publication date

  2018-02-20

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv 1802.07167
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Wind Power Forecasting Using Multi-Objective Evolutionary Algorithms for Wavelet Neural Network-Optimized Prediction Intervals

  2018influential reference
• Prediction Interval Construction for Byproduct Gas Flow Forecasting Using Optimized Twin Extreme Learning Machine

  2017influential reference
• Wind Power Interval Prediction Based on Improved PSO and BP Neural Network

  2017cited by this paper
• Incorporating Uncertainty into Deep Learning for Spoken Language Assessment

  2017cited by this paper
• Multi-objective evolutionary optimization of prediction intervals for solar energy forecasting with neural networks

  2017influential reference
• Uncertainty in Deep Learning

  2016cited by this paper
• Landslide Displacement Prediction With Uncertainty Based on Neural Networks With Random Hidden Weights

  2016influential reference
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles

  2016influential reference
• Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networks

  2015cited by this paper
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

  2015cited by this paper
• Weight Uncertainty in Neural Networks

  2015cited by this paper
• Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks

  2015cited by this paper
• Uncertainty handling using neural network-based prediction intervals for electrical load forecasting

  2014cited by this paper
• Optimal Prediction Intervals of Wind Power Generation

  2014cited by this paper
• Multi-objective Genetic Algorithm Optimization of a Neural Network for Estimating Wind Speed Prediction Intervals

  2013influential reference
• NSGA-II-trained neural network approach to the estimation of prediction intervals of scale deposition rate in oil & gas equipment

  2013cited by this paper
• Standard Particle Swarm Optimisation

  2012cited by this paper
• Practical Variational Inference for Neural Networks

  2011cited by this paper
• Comprehensive Review of Neural Network-Based Prediction Intervals and New Advances

  2011influential reference
• Lower Upper Bound Estimation Method for Construction of Neural Network-Based Prediction Intervals

  2011influential reference
• Prediction intervals to account for uncertainties in neural network predictions: Methodology and application in bus travel time prediction

  2011cited by this paper
• Characteristic Kernels on Structured Domains Excel in Robotics and Human Action Recognition

  2010cited by this paper
• A tutorial on conformal prediction

  2007cited by this paper
• COMPARISON OF GENETIC ALGORITHM AND PARTICLE SWARM OPTIMISATION

  2005cited by this paper
• Predicting prostate cancer recurrence via maximizing the concordance index

  2004cited by this paper
• Confidence estimation methods for neural networks : a practical comparison

  2001influential reference
• A Comparison of Some Error Estimates for Neural Network Models

  1996influential reference
• Practical Confidence and Prediction Intervals

  1996influential reference
• Estimating the mean and variance of the target probability distribution

  1994cited by this paper
• A Practical Bayesian Framework for Backpropagation Networks

  1992cited by this paper

• A multi-objective framework for predicting public opinion trends on infectious diseases using NSGA-II and interval predictions

  2026cites this paper
• Probabilistic Photovoltaic Power Forecasting with Reliable Uncertainty Quantification via Multi-Scale Temporal–Spatial Attention and Conformalized Quantile Regression

  2026cites this paper
• Cost-Efficient Sensing With Self-Improving Soft Sensor Based on Predictive Uncertainty Estimation in IoT Environments

  2026cites this paper
• A novel interval prediction model for buffeting response of a long-span bridge based on machine learning

  2026cites this paper
• Optical losses in photovoltaic solar panels: Mechanisms, modeling approaches, and mitigation strategies

  2026cites this paper
• Co-optimization for Adaptive Conformal Prediction

  2026cites this paper
• Improving monte carlo dropout uncertainty estimation with stable output layers

  2025cites this paper
• An explainable Bayesian gated recurrent unit model for multi-step streamflow forecasting

  2025cites this paper
• Introducing Interval Neural Networks for Uncertainty-Aware System Identification

  2025cites this paper
• Prospect certainty for data-driven models

  2025cites this paper
• Deterministic Uncertainty Estimation for Multi-Modal Regression With Deep Neural Networks

  2025influential citation
• Prediction Interval in Renewable Energy Forecasting: A Comprehensive Review of Uncertainty Quantification Methods

  2025cites this paper
• Towards trustworthy AI in radiotherapy: a comprehensive review of uncertainty-aware techniques

  2025cites this paper
• ProbeMDE: Uncertainty-Guided Active Proprioception for Monocular Depth Estimation in Surgical Robotics

  2025cites this paper
• Extreme Conformal Prediction: Reliable Intervals for High-Impact Events

  2025cites this paper
• Evaluation of Bayesian Deep Learning Methods for Quantifying Uncertainties in Forecasting Deaths from COVID-19

  2025cites this paper
• Predicting moment-rotation behaviour of beam to column extended endplate bolted connections: An explainable machine learning approach

  2025cites this paper
• Neural network-based prediction interval estimation with large width penalization for renewable energy forecasting and system applications

  2025cites this paper
• Multi-K KNN regression with bootstrap aggregation: Accurate predictions and alternative prediction intervals

  2025cites this paper
• Clinical Bedside Benchmarking Test for Measuring the Total Hemoglobin Concentration

  2025cites this paper
• Towards An Efficient and Effective En Route Travel Time Estimation Framework

  2025cites this paper
• Fuzzy Logic Strikes Back: Fuzzy ODEs for Dynamic Modeling and Uncertainty Quantification

  2025cites this paper
• A Review of Uncertainty Representation and Quantification in Neural Networks

  2025influential citation
• Deterministic Forecasts and Prediction Intervals for Wind Speed Using Enhanced Multi-Quantile Loss Based Dilated Causal Convolutions

  2025cites this paper
• Odyssey of Interval Type-2 Fuzzy Logic Systems: Learning Strategies for Uncertainty Quantification

  2025cites this paper
• A Quasi-Newton Method for a Mean Variance Estimation Network

  2025cites this paper
• Multi-objective optimization empowered spatiotemporal parallel network for online PM2.5\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \

  2025cites this paper
• Streamflow prediction with aleatoric uncertainty quantification in the Yala river basin via Fokker–Planck equation-based frameworks

  2025cites this paper
• CLEAR: Calibrated Learning for Epistemic and Aleatoric Risk

  2025cites this paper
• Deep Learning Framework for Collaborative Variable Time Delay Estimation and Uncertainty Quantification in Industrial Quality Prediction

  2025influential citation
• Calibration Prediction Interval for Non-parametric Regression and Neural Networks

  2025cites this paper
• Uncertainty-aware probabilistic travel demand prediction for mobility-on-demand services

  2025cites this paper
• Guided Uncertainty Learning Using a Post-Hoc Evidential Meta-Model

  2025cites this paper
• Self-adaptive hybrid uncertainty integration method via deterministic analytic formula

  2025cites this paper
• Token-Level Uncertainty-Aware Objective for Language Model Post-Training

  2025cites this paper
• A generic quality and accuracy driven uncertainty quantification framework for reliability analysis

  2025cites this paper
• CONTINA: Confidence Interval for Traffic Demand Prediction with Coverage Guarantee

  2025cites this paper
• A review of predictive uncertainty modeling techniques and evaluation metrics in probabilistic wind speed and wind power forecasting

  2025cites this paper
• Pharmacophore-oriented 3D molecular generation toward efficient feature-customized drug discovery

  2025cites this paper
• Tube Loss based Deep Networks For Improving the Probabilistic Forecasting of Wind Speed

  2025cites this paper
• Robust Adversarial Quantification via Conflict-Aware Evidential Deep Learning

  2025cites this paper
• Evaluating the impact of climate change on hurricane wind risk: A machine learning approach

  2025cites this paper
• Towards reliable control: Uncertainty-aware domain preserving stacked auto-encoder for data-driven modeling in large-scale industrial systems

  2025cites this paper
• Uncertainty Quantification in SVM prediction

  2025influential citation
• Adapting GT2-FLS for Uncertainty Quantification: A Blueprint Calibration Strategy

  2025cites this paper
• E-PINNs: Epistemic Physics-Informed Neural Networks

  2025cites this paper
• Conformal Prediction and Trustworthy AI

  2025cites this paper
• Multi-step ahead streamflow and uncertainty forecasting using a HyMoLAP rainfall-runoff model-based framework integrated with Bayesian neural networks in the Ouémé river basin, Benin

  2025cites this paper
• An Improved Uncertainty Calibration Approach for Reliable Satellite Temperature Interval Prediction

  2025cites this paper
• Early Failure Detection in Autonomous Surgical Soft-Tissue Manipulation via Uncertainty Quantification

  2025cites this paper
• A distribution-free interval estimation method for lithium-ion battery state of health

  2025cites this paper
• Shapley value-driven superior subset selection algorithm for carbon price interval forecast combination

  2025cites this paper
• A novel hybrid interval prediction framework integrating multiobjective optimization and quantile deep learning for copper price prediction

  2025cites this paper
• Probabilistic Uncertainty Consideration in Regionalization and Prediction of Groundwater Nitrate Concentration

  2024cites this paper
• A Contextual Bandit Approach for Value-oriented Prediction Interval Forecasting

  2024cites this paper
• SDE-HNN: Accurate and Well-Calibrated Forecasting Using Stochastic Differential Equations

  2024cites this paper
• Research Directions on AI and Nuclear

  2024cites this paper
• Evaluation methodology for disentangled uncertainty quantification on regression models

  2024cites this paper
• Interval Forecasts for Gas Prices in the Face of Structural Breaks - Statistical Models vs. Neural Networks

  2024cites this paper
• UTrack: Multi-Object Tracking with Uncertain Detections

  2024cites this paper
• Creating Model-Agnostic Prediction Intervals

  2024cites this paper
• Multi-Objective Estimation of Optimal Prediction Intervals for Wind Power Forecasting

  2024influential citation
• Estimation of Prediction Intervals for Performance Assessment of Building Using Machine Learning

  2024influential citation
• SEF: A Method for Computing Prediction Intervals by Shifting the Error Function in Neural Networks

  2024influential citation
• Relaxed Quantile Regression: Prediction Intervals for Asymmetric Noise

  2024influential citation
• SEMF: Supervised Expectation-Maximization Framework for Predicting Intervals

  2024cites this paper
• Beyond the Norms: Detecting Prediction Errors in Regression Models

  2024cites this paper
• SAUC: Sparsity-Aware Uncertainty Calibration for Spatiotemporal Prediction with Graph Neural Networks

  2024cites this paper
• Evidential Uncertainty Sets in Deep Classifiers Using Conformal Prediction

  2024cites this paper
• Looks Too Good To Be True: An Information-Theoretic Analysis of Hallucinations in Generative Restoration Models

  2024cites this paper
• Regularized artificial neural network based patent value interval prediction model

  2024cites this paper
• Coverage-based Data-centric Approaches for Responsible and Trustworthy AI

  2024cites this paper
• A hybrid graph attention network based method for interval prediction of shipboard solar irradiation

  2024cites this paper
• Uncertainty Quantification of Data-driven Quality Prediction Model For Realizing the Active Sampling Inspection of Mechanical Properties in Steel Production

  2024cites this paper
• Continuous optimization for construction of neural network-based prediction intervals

  2024cites this paper
• Quantification of uncertainty in robot pose errors and calibration of reliable compensation values

  2024cites this paper
• A novel interval estimation framework for wind power forecasting using multi-objective gradient descent optimization

  2024cites this paper
• Zadeh's Type-2 Fuzzy Logic Systems: Precision and High-Quality Prediction Intervals

  2024cites this paper
• Uncertainty-Aware Temporal Graph Convolutional Network for Traffic Speed Forecasting

  2024cites this paper
• A neural network approach for the mortality analysis of multiple populations: a case study on data of the Italian population

  2024cites this paper
• Uncertainty Quantification of Capacitor Switching Transient Location Using Machine Learning

  2024cites this paper
• Estimating Epistemic and Aleatoric Uncertainty with a Single Model

  2024cites this paper
• Probabilistic Calibration by Design for Neural Network Regression

  2024cites this paper
• Expected Pinball Loss For Quantile Regression And Inverse CDF Estimation

  2024cites this paper
• Enhancing Interval Type-2 Fuzzy Logic Systems: Learning for Precision and Prediction Intervals

  2024cites this paper
• Reliability-based journey time prediction via two-stream deep learning with multi-source data

  2024influential citation
• A survey on uncertainty quantification in deep learning for financial time series prediction

  2024cites this paper
• CreINNs: Credal-Set Interval Neural Networks for Uncertainty Estimation in Classification Tasks

  2024cites this paper
• Inferring Data Preconditions from Deep Learning Models for Trustworthy Prediction in Deployment

  2024cites this paper
• Uncertainty‐informed regional deformation diagnosis of arch dams

  2024cites this paper
• A Conformal Approach to Feature-based Newsvendor under Model Misspecification

  2024cites this paper
• Tube Loss: A Novel Approach for Prediction Interval Estimation and probabilistic forecasting

  2024influential citation
• High-speed rail passenger flow prediction based on crossformer and quantile regression: A deep learning approach assisted by internet search indices

  2024cites this paper
• High-dimensional Bayesian optimization with a combination of Kriging models

  2024cites this paper
• Constructing prediction intervals to explore uncertainty based on deep neural networks

  2024influential citation
• Probabilistic-based electricity demand forecasting with hybrid convolutional neural network-extreme learning machine model

  2024cites this paper
• Operational SYM‐H Forecasting With Confidence Intervals Using Deep Neural Networks

  2024cites this paper
• Large width penalization for neural network-based prediction interval estimation

  2024influential citation
• Particle Swarm Training of a Neural Network for the Lower Upper Bound Estimation of the Prediction Intervals of Time Series

  2023cites this paper
• Multitask learning of health state assessment and remaining useful life prediction for sensor-equipped machines

  2023cites this paper