Speeding up convolutional networks pruning with coarse ranking

Channel-based pruning has achieved significant successes in accelerating deep convolutional neural network, whose pipeline is an iterative three-step procedure: ranking, pruning and fine-tuning. However, this iterative procedure is computationally expensive. In this study, we present a novel computationally efficient channel pruning approach based on the coarse ranking that utilizes the intermediate results during fine-tuning to rank the importance of filters, built upon state-of-the-art works with data-driven ranking criteria. The goal of this work is not to propose a single improved approach built upon a specific channel pruning method, but to introduce a new general framework that works for a series of channel pruning methods. Various benchmark image datasets (CIFAR-10, ImageNet, Birds-200, and Flowers-102) and network architectures (AlexNet and VGG-16) are utilized to evaluate the proposed approach for object classification purpose. Experimental results show that the proposed method can achieve almost identical performance with the corresponding state-of-the-art works (baseline) while our ranking time is negligibly short. In specific, with the proposed method, 75% and 54% of the total computation time for the whole pruning procedure can be reduced for AlexNet on CIFAR-10, and for VGG-16 on ImageNet, respectively. Our approach would significantly facilitate pruning practice, especially on resource-constrained platforms.

• Publication year

  2019

• Venue

  arXiv.org

• Publication date

  2019-02-18

• Fields of study

  Computer Science

• Identifiers
  arXiv 1902.06385
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Cellular structure image classification with small targeted training samples

  2019cited by this paper
• Classes

  2019cited by this paper
• DAEN: Deep Autoencoder Networks for Hyperspectral Unmixing

  2019cited by this paper
• Deep reinforcement learning of cell movement in the early stage of C.elegans embryogenesis

  2018cited by this paper
• In Silico Labeling: Predicting Fluorescent Labels in Unlabeled Images.

  2018cited by this paper
• Fast-Converging Conditional Generative Adversarial Networks for Image Synthesis

  2018cited by this paper
• Learning Efficient Convolutional Networks through Network Slimming

  2017cited by this paper
• Channel Pruning for Accelerating Very Deep Neural Networks

  2017cited by this paper
• Automatic differentiation in PyTorch

  2017cited by this paper
• EIE: Efficient Inference Engine on Compressed Deep Neural Network

  2016cited by this paper
• Pruning Filters for Efficient ConvNets

  2016cited by this paper
• Pruning Convolutional Neural Networks for Resource Efficient Inference

  2016influential reference
• Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures

  2016cited by this paper
• Channel-Level Acceleration of Deep Face Representations

  2015cited by this paper
• Learning both Weights and Connections for Efficient Neural Network

  2015cited by this paper
• Very Deep Convolutional Networks for Large-Scale Image Recognition

  2014cited by this paper
• ImageNet classification with deep convolutional neural networks

  2012cited by this paper
• Caltech-UCSD Birds 200

  2010cited by this paper
• Learning Multiple Layers of Features from Tiny Images

  2009cited by this paper
• ImageNet: A large-scale hierarchical image database

  2009cited by this paper
• Automated Flower Classification over a Large Number of Classes

  2008cited by this paper
• Significance Testing of the Spearman Rank Correlation Coefficient

  1972cited by this paper

• No citing papers are available for this paper.