SCIENCE@home

Weightless: Lossy Weight Encoding For Deep Neural Network Compression

Brandon Reagen,Udit Gupta,Bob Adolf,M. Mitzenmacher,Alexander M. Rush,Gu-Yeon Wei,D. Brooks

Published 2017 in International Conference on Machine Learning

ABSTRACT

The large memory requirements of deep neural networks limit their deployment and adoption on many devices. Model compression methods effectively reduce the memory requirements of these models, usually through applying transformations such as weight pruning or quantization. In this paper, we present a novel scheme for lossy weight encoding co-designed with weight simplification techniques. The encoding is based on the Bloomier filter, a probabilistic data structure that can save space at the cost of introducing random errors. Leveraging the ability of neural networks to tolerate these imperfections and by re-training around the errors, the proposed technique, named Weightless, can compress weights by up to 496x without loss of model accuracy. This results in up to a 1.51x improvement over the state-of-the-art.

PUBLICATION RECORD

  • Publication year

    2017

  • Venue

    International Conference on Machine Learning

  • Publication date

    2017-11-13

  • Fields of study

    Mathematics, Computer Science

  • Identifiers
    arXiv 1711.04686
  • External record

    Open on Semantic Scholar

  • Source metadata

    Semantic Scholar

CITATION MAP

EXTRACTION MAP

CLAIMS

  • No claims are published for this paper.

CONCEPTS

  • No concepts are published for this paper.

REFERENCES

Showing 1-26 of 26 references · Page 1 of 1

CITED BY

Showing 1-41 of 41 citing papers · Page 1 of 1