Direct Training for Spiking Neural Networks: Faster, Larger, Better

Spiking neural networks (SNNs) that enables energy efficient implementation on emerging neuromorphic hardware are gaining more attention. Yet now, SNNs have not shown competitive performance compared with artificial neural networks (ANNs), due to the lack of effective learning algorithms and efficient programming frameworks. We address this issue from two aspects: (1) We propose a neuron normalization technique to adjust the neural selectivity and develop a direct learning algorithm for deep SNNs. (2) Via narrowing the rate coding window and converting the leaky integrate-and-fire (LIF) model into an explicitly iterative version, we present a Pytorch-based implementation method towards the training of large-scale SNNs. In this way, we are able to train deep SNNs with tens of times speedup. As a result, we achieve significantly better accuracy than the reported works on neuromorphic datasets (N-MNIST and DVSCIFAR10), and comparable accuracy as existing ANNs and pre-trained SNNs on non-spiking datasets (CIFAR10). To our best knowledge, this is the first work that demonstrates direct training of deep SNNs with high performance on CIFAR10, and the efficient implementation provides a new way to explore the potential of SNNs.

• Publication year

  2018

• Venue

  AAAI Conference on Artificial Intelligence

• Publication date

  2018-09-16

• Fields of study

  Computer Science, Engineering

• Identifiers
  DOI 10.1609/aaai.v33i01.33011311arXiv 1809.05793
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Deep Learning in Spiking Neural Networks

  2018cited by this paper
• BindsNET: A Machine Learning-Oriented Spiking Neural Networks Library in Python

  2018influential reference
• $L1$ -Norm Batch Normalization for Efficient Training of Deep Neural Networks

  2018cited by this paper
• Loihi: A Neuromorphic Manycore Processor with On-Chip Learning

  2018cited by this paper
• Going Deeper in Spiking Neural Networks: VGG and Residual Architectures

  2018cited by this paper
• HATS: Histograms of Averaged Time Surfaces for Robust Event-Based Object Classification

  2018cited by this paper
• Group Normalization

  2018influential reference
• Hybrid Macro/Micro Level Backpropagation for Training Deep Spiking Neural Networks

  2018cited by this paper
• Is Neuromorphic MNIST Neuromorphic? Analyzing the Discriminative Power of Neuromorphic Datasets in the Time Domain

  2018cited by this paper
• Spiking Deep Residual Network

  2018cited by this paper
• Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks

  2017influential reference
• Automatic differentiation in PyTorch

  2017cited by this paper
• How to Train a Compact Binary Neural Network with High Accuracy?

  2017cited by this paper
• CIFAR10-DVS: An Event-Stream Dataset for Object Classification

  2017cited by this paper
• HOTS: A Hierarchy of Event-Based Time-Surfaces for Pattern Recognition

  2017cited by this paper
• Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification

  2017cited by this paper
• Unsupervised regenerative learning of hierarchical features in Spiking Deep Networks for object recognition

  2016cited by this paper
• Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences

  2016cited by this paper
• Training Deep Spiking Neural Networks Using Backpropagation

  2016cited by this paper
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

  2016cited by this paper
• Binarized Neural Networks

  2016cited by this paper
• Bio-Inspired Spiking Convolutional Neural Network using Layer-wise Sparse Coding and STDP Learning

  2016cited by this paper
• Effective sensor fusion with event-based sensors and deep network architectures

  2016cited by this paper
• Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades

  2015cited by this paper
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

  2015cited by this paper
• Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition

  2015cited by this paper
• Unsupervised learning of digit recognition using spike-timing-dependent plasticity

  2015influential reference
• Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing

  2015cited by this paper
• HFirst: A Temporal Approach to Object Recognition

  2015cited by this paper
• Spiking Deep Networks with LIF Neurons

  2015cited by this paper
• A million spiking-neuron integrated circuit with a scalable communication network and interface

  2014cited by this paper
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

  2013cited by this paper
• Brian simulator

  2013cited by this paper
• Normalization as a canonical neural computation

  2011cited by this paper
• The Book of Genesis

  2011cited by this paper
• The Book of Genesis

  2010cited by this paper
• Functional mechanisms shaping lateral geniculate responses to artificial and natural stimuli.

  2008cited by this paper
• SpiNNaker: Mapping neural networks onto a massively-parallel chip multiprocessor

  2008cited by this paper
• Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity

  2007cited by this paper
• Artificial Brains

  2007influential reference
• NEST (NEural Simulation Tool)

  2007cited by this paper
• Simulation of networks of spiking neurons: A review of tools and strategies

  2006cited by this paper
• Histograms of oriented gradients for human detection

  2005cited by this paper

• FM-DLM: A new method for image classification based on the fusion of multi-level deep learning models

  2026cites this paper
• TP-Spikformer: Token Pruned Spiking Transformer

  2026cites this paper
• SGSSA: Spatio-temporal granular-threshold spiking self-attention for spiking neural networks

  2026cites this paper
• Ternary Spiking Neural Networks Enhanced by Complemented Neurons and Membrane Potential Aggregation

  2026cites this paper
• TranSpikformer: Enhanced transformer-based spiking neural networks with uniform attention augmentation and depthwise convolutional positional encoding

  2026cites this paper
• An FPGA-Based High-Energy-Efficiency Hardware Accelerator for Spiking Transformer

  2026cites this paper
• Analysis of bio-plausible spiking neural networks for motor imagery recognition tasks

  2026cites this paper
• SPCNNet: spiking point cloud neural network for morphological neuron classification

  2026cites this paper
• Advanced fabrication strategies for Durable, High-Efficiency, and scalable solid oxide fuel cells

  2026cites this paper
• SQKformer: Spiking sparse QKformer with adaptive batch normalization for membrane potential

  2026cites this paper
• Robust Spiking Neural Networks Against Adversarial Attacks

  2026cites this paper
• Advanced Training Algorithms in Sigma-Delta Spiking YOLO for Energy-Efficient Object Detection on Neuromorphic Hardware

  2026cites this paper
• Event-based low-power spiking gaze estimation

  2026cites this paper
• Masked timesteps in spiking neural networks for efficient time series classification

  2026cites this paper
• Temporal local attention with adaptive decoding: Enhancing spiking neural networks for temporal computing applications

  2026cites this paper
• SEDformer: Event-Synchronous Spiking Transformers for Irregular Telemetry Time Series Forecasting

  2026cites this paper
• Orthogonal space constraint enhances learning scalability and convergence efficiency without gradient backpropagation

  2026cites this paper
• Adjoint propagation of error signal through modular recurrent neural networks for biologically plausible learning.

  2026cites this paper
• Phase-dependent plasticity: A brain-inspired unsupervised learning algorithm for deep neural networks

  2026cites this paper
• Evolutionary Multiobjective Spiking Neural Architecture Search for Image Classification

  2026cites this paper
• SConvNSys: Accelerating Spiking Convolutional Neural Networks With a Reconfigurable Neuromorphic Architecture for Diverse Applications

  2026cites this paper
• SNN-Drone: Low-Power Drone-View Object Detection With Integer-Valued Spiking Neural Networks

  2026cites this paper
• HASNN: Hierarchical attention spiking neural network for dynamic graph representation learning

  2026cites this paper
• A Brain-Inspired Hybrid Plasticity Feedforward-Inhibitory Spike Neural Network for Olfactory Perception

  2025cites this paper
• Temporal Regularization Training: Unleashing the Potential of Spiking Neural Networks

  2025cites this paper
• Brain-Inspired Spiking Neural Networks for Energy-Efficient Object Detection

  2025cites this paper
• SDSNN: A Single-Timestep Spiking Neural Network with Self-Dropping Neuron and Bayesian Optimization

  2025cites this paper
• ViStream: Improving Computation Efficiency of Visual Streaming Perception via Law-of-Charge-Conservation Inspired Spiking Neural Network

  2025cites this paper
• SpiNeRF: direct-trained spiking neural networks for efficient neural radiance field rendering

  2025cites this paper
• Enhancing robustness of spiking neural networks through retina-like coding and memory-based neurons

  2025cites this paper
• UniPRE: An SNN-ANN Accelerator With Unified Max-Pooling Prediction and Redundancy Elimination

  2025cites this paper
• Reset-dominant accurate synaptic weight mapping in passive memristor arrays for energy-efficient spiking neural networks

  2025cites this paper
• RSNN: Rate encoding mechanism-based spiking neural network for renewable energy forecasting

  2025cites this paper
• DCSNN: An Efficient and High-speed sEMG-based Transient-state Micro-gesture Recognition Method on Wearable Devices

  2025cites this paper
• Improving Performance of Spike-based Deep Q-Learning using Ternary Neurons

  2025cites this paper
• S3 KD: Semantic-aware Spiking Neural Networks Self-distillation for Human Action Recognition

  2025cites this paper
• ESTSformer: Efficient spatio-temporal spiking transformer

  2025cites this paper
• HetSyn: Versatile Timescale Integration in Spiking Neural Networks via Heterogeneous Synapses

  2025cites this paper
• Random heterogeneous spiking neural network for adversarial defense

  2025cites this paper
• SU-YOLO: Spiking Neural Network for Efficient Underwater Object Detection

  2025cites this paper
• EHGCN: Hierarchical Euclidean-Hyperbolic Fusion via Motion-Aware GCN for Hybrid Event Stream Perception

  2025cites this paper
• Adaptation and learning of spatio-temporal thresholds in spiking neural networks

  2025cites this paper
• Temporal Flexibility in Spiking Neural Networks: Towards Generalization Across Time Steps and Deployment Friendliness

  2025influential citation
• Research on target detection for autonomous driving based on ECS-spiking neural networks

  2025cites this paper
• TS-SNN: Temporal Shift Module for Spiking Neural Networks

  2025cites this paper
• ILIF: Temporal Inhibitory Leaky Integrate-and-Fire Neuron for Overactivation in Spiking Neural Networks

  2025cites this paper
• Adversarially Robust Spiking Neural Networks with Sparse Connectivity

  2025cites this paper
• ECC-SNN: Cost-Effective Edge-Cloud Collaboration for Spiking Neural Networks

  2025cites this paper
• Accuracy-Efficiency Trade-Offs in Spiking Neural Networks: A Lempel-Ziv Complexity Perspective on Learning Rules

  2025cites this paper
• ReverB-SNN: Reversing Bit of the Weight and Activation for Spiking Neural Networks

  2025influential citation
• A Hybrid Spiking Model for Anomaly Detection in Multivariate Time Series

  2025cites this paper
• The architecture design and training optimization of spiking neural network with low-latency and high-performance for classification and segmentation

  2025cites this paper
• A lightweight spiking neural network for EEG-based motor imagery classification

  2025cites this paper
• An empirical study on optimizing binary spiking neural networks for neuromorphic computing

  2025cites this paper
• Robust Spatiotemporal Prototype Learning for Spiking Neural Networks

  2025cites this paper
• Combining aggregated attention and transformer architecture for accurate and efficient performance of Spiking Neural Networks

  2025cites this paper
• CAT SNN: Conversion Aware Training for High Accuracy and Hardware Friendly Spiking Neural Networks

  2025cites this paper
• Towards Efficient and Accurate Spiking Neural Networks via Adaptive Bit Allocation

  2025cites this paper
• Edge Intelligence with Spiking Neural Networks

  2025cites this paper
• Knowledge distillation for spiking neural networks: aligning features and saliency

  2025cites this paper
• Graph Spiking Attention Network: Sparsity, Efficiency and Robustness

  2025cites this paper
• STF: Shallow-Level Temporal Feedback to Enhance Spiking Transformers

  2025cites this paper
• Two-Stream Spiking Neural Network for Event-based Action Recognition

  2025influential citation
• BISNN: bio-information-fused spiking neural networks for enhanced EEG-based emotion recognition

  2025cites this paper
• Towards Energy-efficient Audio-visual Classification via Multimodal Interactive Spiking Neural Network

  2025cites this paper
• SDTrack: A Baseline for Event-based Tracking via Spiking Neural Networks

  2025cites this paper
• Spiking Generative Models Based on Variational Autoencoder and Adversarial Training

  2025cites this paper
• Encrypted Spiking Neural Networks Based on Adaptive Differential Privacy Mechanism

  2025cites this paper
• Bridging Quantized Artificial Neural Networks and Neuromorphic Hardware

  2025cites this paper
• Memory-Free and Parallel Computation for Quantized Spiking Neural Networks

  2025cites this paper
• Enhanced Neuromorphic Semantic Segmentation Latency through Stream Event

  2025cites this paper
• Noise-Injected Spiking Graph Convolution for Energy-Efficient 3D Point Cloud Denoising

  2025cites this paper
• Spiking Transformer: Introducing Accurate Addition-Only Spiking Self-Attention for Transformer

  2025cites this paper
• Research on Anti-Interference Performance of Spiking Neural Network Under Network Connection Damage

  2025cites this paper
• Towards Accurate Binary Spiking Neural Networks: Learning with Adaptive Gradient Modulation Mechanism

  2025cites this paper
• Leakage Fault Diagnosis of Oil and Gas Pipelines Based on Improved Spiking Residual Network

  2025cites this paper
• Channel Characteristics-Based Domain Adaptation for Multiuser MISO Beamforming With Transformer and Spiking Neural Networks

  2025cites this paper
• Spatio-Temporal Channel Attention and Membrane Potential Modulation for Efficient Spiking neural network

  2025cites this paper
• Temporal Separation with Entropy Regularization for Knowledge Distillation in Spiking Neural Networks

  2025cites this paper
• Design of CMOS-memristor hybrid synapse and its application for noise-tolerant memristive spiking neural network

  2025cites this paper
• DTA: Dual Temporal-channel-wise Attention for Spiking Neural Networks

  2025cites this paper
• Dynamic SpikFormer: Low-Latency & Energy-Efficient Spiking Neural Networks with Dynamic Time Steps for Vision Transformers

  2025cites this paper
• Mixed Spiking NeRF: Towards a More Efficient Neural Radiance Fields

  2025cites this paper
• Schoenberg Kernel Loss for Spiking Neural Network Training

  2025cites this paper
• MONAS-ESNN: Multi-Objective Neural Architecture Search for Efficient Spiking Neural Networks

  2025cites this paper
• Temporal-Guided Spiking Neural Networks for Event-Based Human Action Recognition

  2025cites this paper
• Hardware Efficient Accelerator for Spiking Transformer With Reconfigurable Parallel Time Step Computing

  2025cites this paper
• Spatio-Temporal Enhancement-Based Spiking Neural Network for Morphological Neuron Classification

  2025cites this paper
• QUEST: A Quantized Energy-Aware SNN Training Framework for Multi-State Neuromorphic Devices

  2025cites this paper
• Evetac Meets Sparse Probabilistic Spiking Neural Network: Enhancing Snap-Fit Recognition Efficiency and Performance

  2025cites this paper
• Toward Energy-Efficient Spike-Based Deep Reinforcement Learning With Temporal Coding

  2025cites this paper
• Optimized negative spikes in Spiking Neural Networks for low-latency ECG classification

  2025cites this paper
• Threshold Modulation for Online Test-Time Adaptation of Spiking Neural Networks

  2025cites this paper
• SpikeRF-YOLO for drone radio frequency signal detection

  2025cites this paper
• SMVSNN: An Intelligent Framework for Anticancer Drug-Drug Interaction Prediction Utilizing Spiking Multi-view Siamese Neural Networks

  2025cites this paper
• TDFormer: A Top-Down Attention-Controlled Spiking Transformer

  2025cites this paper
• SVL: Spike-based Vision-language Pretraining for Efficient 3D Open-world Understanding

  2025cites this paper
• Time to Spike? Understanding the Representational Power of Spiking Neural Networks in Discrete Time

  2025cites this paper
• Integer Binary-Range Alignment Neuron for Spiking Neural Networks

  2025cites this paper
• qLIF: Mitigating the memory and computation overhead to implement spiking convolutional neural networks

  2025cites this paper