Selective Classification for Deep Neural Networks

Selective classification techniques (also known as reject option) have not yet been considered in the context of deep neural networks (DNNs). These techniques can potentially significantly improve DNNs prediction performance by trading-off coverage. In this paper we propose a method to construct a selective classifier given a trained neural network. Our method allows a user to set a desired risk level. At test time, the classifier rejects instances as needed, to grant the desired risk (with high probability). Empirical results over CIFAR and ImageNet convincingly demonstrate the viability of our method, which opens up possibilities to operate DNNs in mission-critical applications. For example, using our method an unprecedented 2% error in top-5 ImageNet classification can be guaranteed with probability 99.9%, and almost 60% test coverage.

• Publication year

  2017

• Venue

  Neural Information Processing Systems

• Publication date

  2017-05-01

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv 1705.08500
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Boosting with Abstention

  2016cited by this paper
• Wide Residual Networks

  2016influential reference
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning

  2015influential reference
• Deep Residual Learning for Image Recognition

  2015cited by this paper
• Very deep convolutional neural network based image classification using small training sample size

  2015cited by this paper
• Very Deep Convolutional Networks for Large-Scale Image Recognition

  2014cited by this paper
• ImageNet Large Scale Visual Recognition Challenge

  2014cited by this paper
• Active Learning via Perfect Selective Classification

  2012cited by this paper
• Active learning via perfect selective classification

  2012cited by this paper
• A risk bound for ensemble classification with a reject option

  2011cited by this paper
• On the Foundations of Noise-free Selective Classification

  2010influential reference
• Learning Multiple Layers of Features from Tiny Images

  2009cited by this paper
• Generalization bounds for averaged classifiers

  2004cited by this paper
• Support Vector Machines with Embedded Reject Option

  2002cited by this paper
• To reject or not to reject: that is the question-an answer in case of neural classifiers

  2000cited by this paper
• A method for improving classification reliability of multilayer perceptrons

  1995cited by this paper
• Distribution-free performance bounds with the resubstitution error estimate

  1992influential reference
• The Nearest Neighbor Classification Rule with a Reject Option

  1970cited by this paper
• An optimum character recognition system using decision functions

  1957cited by this paper

• Operationalizing Large Language Models for Clinical Research Data Extraction: Methods, Quality Control, and Governance

  2026cites this paper
• PEER: Towards reliable and efficient inference via Patience-Based Early Exiting with Rejection.

  2026cites this paper
• Variance-Gated Ensembles: An Epistemic-Aware Framework for Uncertainty Estimation

  2026cites this paper
• A Physics-Informed Neuro-Fuzzy Framework for Quantum Error Attribution

  2026influential citation
• Leveraging Data to Say No: Memory Augmented Plug-and-Play Selective Prediction

  2026cites this paper
• A Framework for Optimizing Human-Machine Interaction in Classification Systems

  2026cites this paper
• Evaluating Prediction Uncertainty Estimates from BatchEnsemble

  2026cites this paper
• Temperature Scaling Attack Disrupting Model Confidence in Federated Learning

  2026cites this paper
• Not Just How Much, But Where: Decomposing Epistemic Uncertainty into Per-Class Contributions

  2026cites this paper
• Support Tokens, Stability Margins, and a New Foundation for Robust LLMs

  2026cites this paper
• Multi-Level Strategic Classification: Incentivizing Improvement through Promotion and Relegation Dynamics

  2026cites this paper
• Provably Protecting Fine-Tuned LLMs from Training Data Extraction

  2026cites this paper
• Predicting When to Trust Vision-Language Models for Spatial Reasoning

  2026cites this paper
• When to Act: Calibrated Confidence for Reliable Human Intention Prediction in Assistive Robotics

  2026cites this paper
• ReaMIL: Reasoning- and Evidence-Aware Multiple Instance Learning for Whole-Slide Histopathology

  2026cites this paper
• CascadeMind at SemEval-2026 Task 4: A Hybrid Neuro-Symbolic Cascade for Narrative Similarity

  2026cites this paper
• SafeGround: Know When to Trust GUI Grounding Models via Uncertainty Calibration

  2026cites this paper
• MOSAIC: A Structured Multi-level Framework for Probabilistic and Interpretable Cell-type Annotation

  2026influential citation
• Knowing When Not to Answer: Abstention-Aware Scientific Reasoning

  2026cites this paper
• Why Agent Caching Fails and How to Fix It: Structured Intent Canonicalization with Few-Shot Learning

  2026cites this paper
• An Empirical Analysis of Calibration and Selective Prediction in Multimodal Clinical Condition Classification

  2026cites this paper
• LOCUS: A Distribution-Free Loss-Quantile Score for Risk-Aware Predictions

  2026cites this paper
• On Calibration of Large Language Models: From Response To Capability

  2026cites this paper
• Selective Prior Synchronization via SYNC Loss

  2026influential citation
• Bounded-Abstention Multi-horizon Time-series Forecasting

  2026cites this paper
• PRISM: Festina Lente Proactivity -- Risk-Sensitive, Uncertainty-Aware Deliberation for Proactive Agents

  2026cites this paper
• Epistemic Uncertainty Quantification for Pre-trained VLMs via Riemannian Flow Matching

  2026cites this paper
• SpatialBench-UC: Uncertainty-Aware Evaluation of Spatial Prompt Following in Text-to-Image Generation

  2026cites this paper
• An Explainable Agentic AI Framework for Uncertainty-Aware and Abstention-Enabled Acute Ischemic Stroke Imaging Decisions

  2026cites this paper
• Trustworthy Data-Driven Wildfire Risk Prediction and Understanding in Western Canada

  2026cites this paper
• BaseCal: Unsupervised Confidence Calibration via Base Model Signals

  2026cites this paper
• LatentRefusal: Latent-Signal Refusal for Unanswerable Text-to-SQL Queries

  2026cites this paper
• Shapelets-Enriched Selective Forecasting using Time Series Foundation Models

  2026cites this paper
• Should I Have Expressed a Different Intent? Counterfactual Generation for LLM-Based Autonomous Control

  2026cites this paper
• Learning to Defer in Non-Stationary Time Series via Switching State-Space Models

  2026cites this paper
• Chance-Constrained Inference for Hallucination Risk Control in Large Language Models

  2026cites this paper
• SAFE-KD: Risk-Controlled Early-Exit Distillation for Vision Backbones

  2026cites this paper
• Knowing When to Answer: Adaptive Confidence Refinement for Reliable Audio-Visual Question Answering

  2026influential citation
• TRACER: Trajectory Risk Aggregation for Critical Episodes in Agentic Reasoning

  2026cites this paper
• Towards Selection as Power: Bounding Decision Authority in Autonomous Agents

  2026cites this paper
• MMA: Multimodal Memory Agent

  2026cites this paper
• MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs

  2026cites this paper
• Entropy-Based Confidence for Speech-to-Text Recognition

  2026cites this paper
• Monotropic Artificial Intelligence: Toward a Cognitive Taxonomy of Domain-Specialized Language Models

  2026cites this paper
• Refined Generation-Based Framework for Consistent and Reliable Visual Question Answering

  2026cites this paper
• From Misclassifications to Outliers: Joint Reliability Assessment in Classification

  2026cites this paper
• DAVE: A Policy-Enforcing LLM Spokesperson for Secure Multi-Document Data Sharing

  2026cites this paper
• When to Trust the Cheap Check: Weak and Strong Verification for Reasoning

  2026cites this paper
• Evaluating the Impact of Post-Training Quantization on Reliable VQA with Multimodal LLMs

  2026cites this paper
• SCOPE: Selective Conformal Optimized Pairwise LLM Judging

  2026cites this paper
• Reinforcement Inference: Leveraging Uncertainty for Self-Correcting Language Model Reasoning

  2026cites this paper
• A layer-resolved diagnostic identifies bias-driven decisions in deep neural networks

  2026cites this paper
• DoubleTake: Contrastive Reasoning for Faithful Decision-Making in Medical Imaging

  2026cites this paper
• Adaptive Evidence Weighting for Audio-Spatiotemporal Fusion

  2026cites this paper
• Concrete crack reasoning: Explainable defect diagnosis incorporating generative Pretrained transformer 4 and multimodal nondestructive testing data

  2026cites this paper
• Fail closed trust gated synthetic augmentation governs tail risk under subject shift in EEG

  2026cites this paper
• ClinNet: Evidential Ordinal Regression with Bilateral Asymmetry and Prototype Memory for Knee Osteoarthritis Grading

  2026influential citation
• Fostering effective hybrid human-LLM reasoning and decision making

  2025cites this paper
• Few-Shot Adaptation of Foundation Vision Models for PCB Defect Inspection

  2025cites this paper
• INQUIRE-Search: A Framework for Interactive Discovery in Large-Scale Biodiversity Databases

  2025cites this paper
• SETS: Leveraging Self-Verification and Self-Correction for Improved Test-Time Scaling

  2025cites this paper
• On Risk Assessment for Out-of-Distribution Detection

  2025cites this paper
• MedBayes-Lite: Bayesian Uncertainty Quantification for Safe Clinical Decision Support

  2025cites this paper
• Trustworthy AI framework for Salmon disease detection using replay-enhanced selective network with incremental learning

  2025cites this paper
• Confidence-Aware Neural Decoding of Overt Speech from EEG: Toward Robust Brain-Computer Interfaces

  2025cites this paper
• The predictability of the effectiveness of chains of classifiers in the out-of-domain detection

  2025cites this paper
• Honesty over Accuracy: Trustworthy Language Models through Reinforced Hesitation

  2025cites this paper
• Optimizing Uncertainty-Aware Deep Learning for On-the-Edge Murmur Detection in Low-Resource Settings

  2025cites this paper
• Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees

  2025cites this paper
• Catching Contamination Before Generation: Spectral Kill Switches for Agents

  2025cites this paper
• Learning to Defer with Better Features: A Case for Going Beyond the Final Layer

  2025cites this paper
• GALA: A GlobAl-LocAl Approach for Multi-Source Active Domain Adaptation

  2025cites this paper
• The Gray Zone of Faithfulness: Taming Ambiguity in Unfaithfulness Detection

  2025cites this paper
• Budgeted Multiple-Expert Deferral

  2025cites this paper
• What Does It Take to Build a Performant Selective Classifier?

  2025cites this paper
• Cold-Start Sales Forecasting in Fashion via Feature-Based Ensembles

  2025cites this paper
• Towards Explainable and Reliable AI in Finance

  2025cites this paper
• Evaluating Medical LLMs by Levels of Autonomy: A Survey Moving from Benchmarks to Applications

  2025cites this paper
• Model-agnostic Selective Labeling with Provable Statistical Guarantees

  2025cites this paper
• Policy Learning with Abstention

  2025cites this paper
• Deep Learning With Width-Wise Early Exiting and Rejection for Computational Efficient and Trustworthy Modulation Classification

  2025cites this paper
• Revisiting Reweighted Risk for Calibration: AURC, Focal, and Inverse Focal Loss

  2025cites this paper
• When In Doubt, Abstain: The Impact of Abstention on Strategic Classification

  2025cites this paper
• Systematic Evaluation of Uncertainty Estimation Methods in Large Language Models

  2025cites this paper
• One-Stage Top-k Learning-to-Defer: Score-Based Surrogates with Theoretical Guarantees

  2025cites this paper
• Advancing Image Classification with Discrete Diffusion Classification Modeling

  2025cites this paper
• A Trustworthy Model With Uncertainty Management for Predicting Vascular Access Dysfunction in Hemodialysis Patients

  2025cites this paper
• Selective Risk Certification for LLM Outputs via Information-Lift Statistics: PAC-Bayes, Robustness, and Skeleton Design

  2025cites this paper
• Pseudo-D: Informing Multi-View Uncertainty Estimation with Calibrated Neural Training Dynamics

  2025cites this paper
• Similarity-Distance-Magnitude Activations

  2025cites this paper
• Progressive semi-supervised learning for specific emitter identification: An iterative clustering and pseudo-label refinement approach

  2025cites this paper
• Calibrating MLLM-as-a-judge via Multimodal Bayesian Prompt Ensembles

  2025cites this paper
• Trustworthy Fault Diagnosis Paradigm: Learning Models to Abstain From Uncertain Diagnosis

  2025cites this paper
• Conformal validation: A deferral policy using uncertainty quantification with a human-in-the-loop for model validation

  2025cites this paper
• Learning to Partially Defer for Sequences

  2025cites this paper
• Can Molecular Foundation Models Know What They Don't Know? A Simple Remedy with Preference Optimization

  2025cites this paper
• Pitfalls of Conformal Predictions for Medical Image Classification

  2025cites this paper
• Anytime-Valid Answer Sufficiency Certificates for LLM Generation via Sequential Information Lift

  2025cites this paper
• Interpretable and Fair Mechanisms for Abstaining Classifiers

  2025cites this paper
• Energy Landscapes Enable Reliable Abstention in Retrieval-Augmented Large Language Models for Healthcare

  2025cites this paper