Single-shot Channel Pruning Based on Alternating Direction Method of Multipliers

Channel pruning has been identified as an effective approach to constructing efficient network structures. Its typical pipeline requires iterative pruning and fine-tuning. In this work, we propose a novel single-shot channel pruning approach based on alternating direction methods of multipliers (ADMM), which can eliminate the need for complex iterative pruning and fine-tuning procedure and achieve a target compression ratio with only one run of pruning and fine-tuning. To the best of our knowledge, this is the first study of single-shot channel pruning. The proposed method introduces filter-level sparsity during training and can achieve competitive performance with a simple heuristic pruning criterion (L1-norm). Extensive evaluations have been conducted with various widely-used benchmark architectures and image datasets for object classification purpose. The experimental results on classification accuracy show that the proposed method can outperform state-of-the-art network pruning works under various scenarios.

• Publication year

  2019

• Venue

  arXiv.org

• Publication date

  2019-02-18

• Fields of study

  Computer Science

• Identifiers
  arXiv 1902.06382
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Progressive Weight Pruning of Deep Neural Networks using ADMM

  2018cited by this paper
• Viterbi-based Pruning for Sparse Matrix with Fixed and High Index Compression Ratio

  2018cited by this paper
• Deep reinforcement learning of cell movement in the early stage of C.elegans embryogenesis

  2018cited by this paper
• A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers

  2018influential reference
• Efficient Sparse-Winograd Convolutional Neural Networks

  2018cited by this paper
• Fast-Converging Conditional Generative Adversarial Networks for Image Synthesis

  2018cited by this paper
• Large Scale GAN Training for High Fidelity Natural Image Synthesis

  2018cited by this paper
• SNIP: Single-shot Network Pruning based on Connection Sensitivity

  2018cited by this paper
• ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression

  2017cited by this paper
• Mask R-CNN

  2017cited by this paper
• Channel Pruning for Accelerating Very Deep Neural Networks

  2017cited by this paper
• Efficient Processing of Deep Neural Networks: A Tutorial and Survey

  2017cited by this paper
• Mastering the game of Go without human knowledge

  2017cited by this paper
• Pruning Filters for Efficient ConvNets

  2016cited by this paper
• Pruning Convolutional Neural Networks for Resource Efficient Inference

  2016influential reference
• Image-to-Image Translation with Conditional Adversarial Networks

  2016cited by this paper
• FaceNet: A unified embedding for face recognition and clustering

  2015cited by this paper
• Learning both Weights and Connections for Efficient Neural Network

  2015cited by this paper
• Channel-Level Acceleration of Deep Face Representations

  2015cited by this paper
• ImageNet classification with deep convolutional neural networks

  2012cited by this paper
• Learning Multiple Layers of Features from Tiny Images

  2009cited by this paper
• Gradient-based learning applied to document recognition

  1998cited by this paper

• Automated deep‐learning model optimization framework for microcontrollers

  2024cites this paper
• A Survey on Approximate Edge AI for Energy Efficient Autonomous Driving Services

  2023cites this paper
• Model Compression Methods for YOLOv5: A Review

  2023cites this paper
• Pruning via Iterative Ranking of Sensitivity Statistics

  2020cites this paper
• RANP: Resource Aware Neuron Pruning at Initialization for 3D CNNs

  2020cites this paper