Identity Mappings in Deep Residual Networks

Deep residual networks have emerged as a family of extremely deep architectures showing compelling accuracy and nice convergence behaviors. In this paper, we analyze the propagation formulations behind the residual building blocks, which suggest that the forward and backward signals can be directly propagated from one block to any other block, when using identity mappings as the skip connections and after-addition activation. A series of ablation experiments support the importance of these identity mappings. This motivates us to propose a new residual unit, which makes training easier and improves generalization. We report improved results using a 1001-layer ResNet on CIFAR-10 (4.62 % error) and CIFAR-100, and a 200-layer ResNet on ImageNet. Code is available at: https://github.com/KaimingHe/resnet-1k-layers.

• Publication year

  2016

• Venue

  European Conference on Computer Vision

• Publication date

  2016-03-16

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1007/978-3-319-46493-0_38arXiv 1603.05027
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning

  2016cited by this paper
• Training Very Deep Networks

  2015influential reference
• Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

  2015cited by this paper
• All you need is a good init

  2015influential reference
• Highway Networks

  2015influential reference
• Rethinking the Inception Architecture for Computer Vision

  2015cited by this paper
• Deep Residual Learning for Image Recognition

  2015influential reference
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

  2015influential reference
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)

  2015cited by this paper
• Microsoft COCO: Common Objects in Context

  2014cited by this paper
• Deeply-Supervised Nets

  2014cited by this paper
• FitNets: Hints for Thin Deep Nets

  2014influential reference
• ImageNet Large Scale Visual Recognition Challenge

  2014influential reference
• Striving for Simplicity: The All Convolutional Net

  2014cited by this paper
• On the complexity of shallow and deep neural network classifiers

  2014cited by this paper
• Very Deep Convolutional Networks for Large-Scale Image Recognition

  2014cited by this paper
• Fractional Max-Pooling

  2014cited by this paper
• Going deeper with convolutions

  2014cited by this paper
• Network In Network

  2013influential reference
• Improving neural networks by preventing co-adaptation of feature detectors

  2012cited by this paper
• Rectified Linear Units Improve Restricted Boltzmann Machines

  2010cited by this paper
• Learning Multiple Layers of Features from Tiny Images

  2009influential reference
• Long Short-Term Memory

  1997cited by this paper
• Backpropagation Applied to Handwritten Zip Code Recognition

  1989cited by this paper

• Learning Better Certified Models from Empirically-Robust Teachers

  2026cites this paper
• GAF-ResNet-MHSA: A novel transfer learning method for soil nutrient prediction in small sample datasets

  2026cites this paper
• Egocentric Hand Activity Video Dataset and Bidirectional Motion-Priors for Hand Action Recognition

  2026cites this paper
• Morphe: High-Fidelity Generative Video Streaming with Vision Foundation Model

  2026cites this paper
• Deep Learning for Classification of Internal Defects in Fused Filament Fabrication Using Optical Coherence Tomography

  2026cites this paper
• Mitigating Instance Entanglement in Instance-Dependent Partial Label Learning

  2026cites this paper
• Improving Adversarial Transferability on Vision Transformers via Dual-Flow Adversarial Attack

  2026cites this paper
• Spatiotemporal Prediction of Ground Surface Deformation Using TPE-Optimized Deep Learning

  2026cites this paper
• GlassCurtainCrackAdversarialDefense: A convolutional and attention-based adversarial defense network for glass curtain wall crack detection

  2026cites this paper
• Revisiting Generalization Measures Beyond IID: An Empirical Study under Distributional Shift

  2026cites this paper
• Transformer Based on Multi-Domain Feature Fusion for AI-Generated Image Detection

  2026cites this paper
• A Lightweight Drone Vision System for Autonomous Inspection with Real-Time Processing

  2026cites this paper
• Goal-Oriented Influence-Maximizing Data Acquisition for Learning and Optimization

  2026cites this paper
• Analysis of Human Brain Tumor Lesions Based on an Enhanced ResNet-18 Model

  2026cites this paper
• Domain-augmented large language models for automated root cause classification of offshore process incidents

  2026cites this paper
• A comprehensive review and ensemble CNN approach for Bharatanatyam single-hand gesture classification

  2026cites this paper
• Essentially No Energy Barrier Between Independent Fermionic Neural Quantum State Minima

  2026cites this paper
• Breaking the Limits of Open-Weight CLIP: An Optimization Framework for Self-supervised Fine-tuning of CLIP

  2026cites this paper
• Quadratic Upper Bound for Boosting Robustness

  2026cites this paper
• Compilation of a Nationwide River Image Dataset for Identifying River Channels and River Rapids via Deep Learning

  2026cites this paper
• A temporal-aware generative network for cross-modal video universal adversarial perturbation generation

  2026cites this paper
• Reducing Class-Wise Performance Disparity via Margin Regularization

  2026influential citation
• Anatomy-Aware MR-Imaging-Only Radiotherapy

  2026cites this paper
• MoE-Net: A deep ensemble framework optimized by genetic algorithm for nasal bone fracture detection on lateral X-ray images

  2026cites this paper
• Resource-Efficient Model-Free Reinforcement Learning for Board Games

  2026influential citation
• Towards an Explainable Approach: Hybrid Residual Network and SVM for Automated Brain Tumor Detection and Classification

  2026cites this paper
• Lc-graspnet: a lightweight high-performance grasp detection network

  2026influential citation
• Toward a Neurosymbolic Understanding of Hidden Neuron Activations

  2026cites this paper
• C/R-PMGAN: A dual-path self-supervised and patch-masked generative adversarial network for transferable black-box attack.

  2026cites this paper
• A Unified Framework for Backdoor Trigger Segmentation

  2026cites this paper
• Wireless Interference Recognition Based on Multimodal Feature Fusion

  2026cites this paper
• Progressive Learning of Instance-Level Proxy Semantics for Few-Shot Action Recognition

  2026cites this paper
• Compression of fully-connected layers based on parameter sharing for large-scale category image classification

  2026cites this paper
• Refining decision boundaries via dynamic label adversarial training for robust traffic classification

  2026cites this paper
• Accounting for Optimal Control in the Sizing of Isolated Hybrid Renewable Energy Systems Using Imitation Learning

  2026cites this paper
• Flexible Acoustic Emission Sensor Signal Classification Using Convolutional Neural Networks for Pipeline Leak Detection

  2026cites this paper
• Residential-ECG: AI-driven one-step generation of low-energy residential floor plans

  2026cites this paper
• Transfer learning with Bayesian optimization for colorectal cancer histopathology classification

  2026cites this paper
• Deep-Learning and SIRV-Based Dual-Domain Speckle Suppression for PolSAR Imagery

  2026cites this paper
• Differentiable Logic Synthesis: Spectral Coefficient Selection via Sinkhorn-Constrained Composition

  2026cites this paper
• Neural Discrimination-Prompted Transformers for Efficient UHD Image Restoration and Enhancement

  2026cites this paper
• Balance divergence for knowledge distillation

  2026cites this paper
• Revisiting Diffusion Model Predictions Through Dimensionality

  2026cites this paper
• Monkeypox Detection and Classification Using Deep Learning With Ensemble Method and Attention Mechanisms

  2026cites this paper
• Hierarchical fusion of electrocardiogram and phonocardiogram data improves heart failure detection

  2026cites this paper
• SanD-Planner: Sample-Efficient Diffusion Planner in B-Spline Space for Robust Local Navigation

  2026cites this paper
• Research on cross-modal fusion building extraction technology based on knowledge distillation

  2026cites this paper
• Deep learning and firearm wound classification: a pilot study

  2026cites this paper
• The Evolutionary Structure of Acoustic Learnability: A Deep Learning Approach to Neotropical Birdsong

  2026influential citation
• BFA: A Lightweight Biologically-Inspired Feedback Attention Mechanism

  2026cites this paper
• ANCRe: Adaptive Neural Connection Reassignment for Efficient Depth Scaling

  2026influential citation
• Gradient Residual Connections

  2026cites this paper
• A vision transformer-based fine-tuned DINOv2 model for bangla sign language recognition

  2026cites this paper
• CODI: Contextual Object Detection via Image Inpainting

  2026cites this paper
• A data-driven decision model for selecting ML models in research software

  2026cites this paper
• SEDP-SegResnet for human eyeball and lens segmentation

  2026cites this paper
• Multimodal machine learning for Cr(Ⅵ) removal and floc settling using image-based floc features and operating parameters.

  2026cites this paper
• Transparent Deep Neural Networks for High-Fidelity Species Identification in Microalgae

  2026cites this paper
• FlashOptim: Optimizers for Memory Efficient Training

  2026influential citation
• Physics-informed hybrid U-Net architecture for phase-to-height reconstruction in dual divergent structured-light systems

  2026cites this paper
• Swish-T: Enhancing Swish Activation With Tanh-Based Bias for Improved Neural Network Performance

  2026cites this paper
• Towards Structural Transformation-based Attack for Boosting Transferability of Adversarial Examples

  2026cites this paper
• New idtracker.ai: rethinking multi-animal tracking as a representation learning problem to increase accuracy and reduce tracking times

  2026cites this paper
• FHD-HybridNet: A fuzzy hellinger distance-guided hybrid deep learning framework for robust multiclass brain tumor classification

  2026cites this paper
• A residual attention neural network for multi-circuit analog parameter prediction and analysis

  2026cites this paper
• Physics-informed neural network (PINN) for predicting the temperature field in high geothermal tunnels

  2026cites this paper
• Targeted Closed-Box Adversarial Example Generation Method Using Multilayer Heatmap Mapping for Industrial IoT

  2026cites this paper
• Universal Source Separation With Weakly Labelled Data for Computational Auditory Scene Analysis

  2026cites this paper
• DM-RRG: Radiology image report generation with disease-topic guidance and multi-grained Image-Text alignment

  2026cites this paper
• SynthraXCoreNet: An Interpretable, Well-Calibrated Six-CNN Ensemble for Dermoscopic Skin-Lesion Classification

  2026cites this paper
• Research on Tool Wear State Recognition Method Based on Multi-Scale Feature Extraction and Deep Residual Network Fusion

  2026cites this paper
• Multimodal deep learning approaches for improving polygenic risk scores with imaging data

  2026cites this paper
• Explainable Deep Learning Framework for Reliable Species-Level Classification Within the Genera Desmodesmus and Tetradesmus

  2026cites this paper
• mHC-lite: You Don't Need 20 Sinkhorn-Knopp Iterations

  2026cites this paper
• Proof of Reasoning for Privacy Enhanced Federated Blockchain Learning at the Edge

  2026cites this paper
• Explainable fusion-based transfer learning for lung diseases classification

  2026cites this paper
• Early prediction of diabetic retinopathy using a multimodal deep learning framework integrating fundus and OCT imaging

  2026cites this paper
• An inspection path planning approach based on hierarchical reinforcement learning

  2026cites this paper
• Chest X-ray deep learning comparative diagnosis using visual and semantic similarity with variational autoencoders

  2026cites this paper
• Trainable multi-channel front-ends for joint beamforming and speaker embedding extraction

  2026cites this paper
• CTHA: Constrained Temporal Hierarchical Architecture for Stable Multi-Agent LLM Systems

  2026cites this paper
• A Streamlined Attention-Based Network for Descriptor Extraction

  2026cites this paper
• Graph Recognition via Subgraph Prediction

  2026cites this paper
• Bridging packet and session: Cross-level dual-attention networks for encrypted traffic classification

  2026influential citation
• Explainable depth-wise and channel-wise fusion models for multi-class skin lesion classification

  2026cites this paper
• Collaborative deep learning framework based on adaptive feature fusion for malignancy prediction of lung nodules

  2026cites this paper
• Rethinking Intelligence: Brain-like Neuron Network

  2026cites this paper
• SENDAI: A Hierarchical Sparse-measurement, EfficieNt Data AssImilation Framework

  2026cites this paper
• Advancing cat facial emotion interpretation: evaluating supervised and semi-supervised models using facial landmark and image data

  2026cites this paper
• Compression and Ray-March Rendering Using Implicit Neural Representations of Data Values and Domain Extent

  2026cites this paper
• A lightweight pruning framework with minimal retraining using Taylor expansion and multi-knowledge preservation strategy

  2026cites this paper
• Warm-start or cold-start? A comparison of generalizability in gradient-based hyperparameter tuning.

  2026cites this paper
• UPGP:Backdoor Defense via Unlearning Perturbation and Orthogonality-Constraint Gradient Projection

  2026cites this paper
• Fully utilizing cross modal features to achieve precise segmentation of brain gliomas

  2026cites this paper
• Koopman Autoencoders with Continuous-Time Latent Dynamics for Fluid Dynamics Forecasting

  2026cites this paper
• Deep-Learning Denoising of Radio Observations for Ultra-High-Energy Cosmic-Ray Detection

  2026cites this paper
• SPICE: Structured Pruning for Inference on Constrained Edge Devices

  2026cites this paper
• Tuning Out-of-Distribution (OOD) Detectors Without Given OOD Data

  2026cites this paper
• Rebenchmarking Unsupervised Monocular 3D Occupancy Prediction

  2026cites this paper
• Preserving Information Flow Stability for Robust Network Pruning

  2026cites this paper