Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations

We present a new approach to learn compressible representations in deep architectures with an end-to-end training strategy. Our method is based on a soft (continuous) relaxation of quantization and entropy, which we anneal to their discrete counterparts throughout training. We showcase this method for two challenging applications: Image compression and neural network compression. While these tasks have typically been approached with different methods, our soft-to-hard quantization approach gives results competitive with the state-of-the-art for both.

• Publication year

  2017

• Venue

  Neural Information Processing Systems

• Publication date

  2017-04-03

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv 1704.00648
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Lossy Image Compression with Compressive Autoencoders

  2017influential reference
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights

  2017cited by this paper
• Soft Weight-Sharing for Neural Network Compression

  2017cited by this paper
• Dermatologist-level classification of skin cancer with deep neural networks

  2017cited by this paper
• Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates for Recurrent Networks

  2017cited by this paper
• Learning Structured Sparsity in Deep Neural Networks

  2016cited by this paper
• Towards the Limit of Network Quantization

  2016influential reference
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks

  2016cited by this paper
• RECURRENT NEURAL NETWORKS

  2016cited by this paper
• Mastering the game of Go with deep neural networks and tree search

  2016cited by this paper
• Full Resolution Image Compression with Recurrent Neural Networks

  2016cited by this paper
• Is the deconvolution layer the same as a convolutional layer?

  2016cited by this paper
• End-to-end Optimized Image Compression

  2016influential reference
• Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network

  2016cited by this paper
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations

  2016cited by this paper
• End-to-end optimization of nonlinear transform codes for perceptual quality

  2016cited by this paper
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations

  2015cited by this paper
• Single image super-resolution from transformed self-exemplars

  2015cited by this paper
• Learning both Weights and Connections for Efficient Neural Network

  2015cited by this paper
• Deep Residual Learning for Image Recognition

  2015cited by this paper
• Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding

  2015influential reference
• A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution

  2014cited by this paper
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• ImageNet Large Scale Visual Recognition Challenge

  2014influential reference
• Auto-Encoding Variational Bayes

  2013cited by this paper
• Optimizing 1-Nearest Prototype Classifiers

  2013cited by this paper
• JPEG2000 - image compression fundamentals, standards and practice

  2013cited by this paper
• ImageNet classification with deep convolutional neural networks

  2012cited by this paper
• Using very deep autoencoders for content-based image retrieval

  2011cited by this paper
• Learning Multiple Layers of Features from Tiny Images

  2009cited by this paper
• ImageNet: A large-scale hierarchical image database

  2009influential reference
• Elements of Information Theory

  2005cited by this paper
• Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning

  2004cited by this paper
• Image quality assessment: from error visibility to structural similarity

  2004cited by this paper
• Multiscale structural similarity for image quality assessment

  2003cited by this paper
• Context-based adaptive binary arithmetic coding in the H.264/AVC video compression standard

  2003cited by this paper
• A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics

  2001cited by this paper
• Continuation-based learning algorithm for discrete-time cellular neural networks

  1994cited by this paper
• Vector quantization by deterministic annealing

  1992cited by this paper
• Competitive learning and soft competition for vector quantizer design

  1992cited by this paper
• The JPEG still picture compression standard

  1991influential reference
• Numerical continuation methods - an introduction

  1990cited by this paper
• Arithmetic coding for data compression

  1987cited by this paper

• The Equalizer: Introducing Shape-Gain Decomposition in Neural Audio Codecs

  2026cites this paper
• Generalized Radius and Integrated Codebook Transforms for Differentiable Vector Quantization

  2026cites this paper
• Enhanced image compression using wavelet-based encoder-decoder architectures

  2026cites this paper
• SCALED : Surrogate-gradient for Codec-Aware Learning of Downsampling in ABR Streaming

  2026cites this paper
• CADC: Content Adaptive Diffusion-Based Generative Image Compression

  2026cites this paper
• Hybrid Downlink Beamforming with Outage Constraints under Imperfect CSI using Model-Driven Deep Learning

  2026cites this paper
• Semantic-Oriented Modulation for Wireless Communication

  2025cites this paper
• BADiff: Bandwidth Adaptive Diffusion Model

  2025cites this paper
• Task-Specific Audio Coding for Machines: Machine-Learned Latent Features Are Codes for That Machine

  2025cites this paper
• MGVQ: Could VQ-VAE Beat VAE? A Generalizable Tokenizer with Multi-group Quantization

  2025cites this paper
• Switchable Token-Specific Codebook Quantization For Face Image Compression

  2025cites this paper
• LiLoCA: Lightweight Lossy Compression Approach for Edge-Enabled Multimedia IoT Applications

  2025cites this paper
• Learned Successive Convex Approximation for Sparse Signal Recovery

  2025cites this paper
• Lossy Neural Compression for Geospatial Analytics: A review

  2025cites this paper
• Improving 3D Gaussian Splatting Compression by Scene-Adaptive Lattice Vector Quantization

  2025cites this paper
• Exploiting Latent Properties to Optimize Neural Codecs

  2025cites this paper
• High Fault-Tolerant DNA Image Storage System Based on VAE

  2025cites this paper
• Digital Semantic Communication System With a Learnable Channel Estimator for OFDM Transmission

  2025cites this paper
• Comp-X: On Defining an Interactive Learned Image Compression Paradigm With Expert-driven LLM Agent

  2025cites this paper
• Generative Probabilistic Entropy Modeling With Conditional Diffusion for Learned Image Compression

  2025cites this paper
• Layered and scalable image coding with semantic features for human and machine

  2025cites this paper
• OASIS: Optimized Lightweight Autoencoder System for Distributed In-Sensor computing

  2025cites this paper
• DiVeQ: Differentiable Vector Quantization Using the Reparameterization Trick

  2025cites this paper
• NIC-RobustBench: A Comprehensive Open-Source Toolkit for Neural Image Compression and Robustness Analysis

  2025cites this paper
• HVQ-CGIC: Enabling Hyperprior Entropy Modeling for VQ-Based Controllable Generative Image Compression

  2025cites this paper
• MelTok: 2D Tokenization for Single-Codebook Audio Compression

  2025cites this paper
• End-to-End Optimized Image Compression With Deep Gaussian Process Regression

  2025cites this paper
• RDD: Pareto Analysis of the Rate-Distortion-Distinguishability Trade-off

  2025cites this paper
• Synonymity-Based Semantic Coding for Efficient Speech Compression

  2025cites this paper
• Multirate Neural Image Compression with Adaptive Lattice Vector Quantization

  2025influential citation
• CANeRV: Content Adaptive Neural Representation for Video Compression

  2025cites this paper
• Spherical Leech Quantization for Visual Tokenization and Generation

  2025cites this paper
• Towards practical vector quantization for image coding

  2025cites this paper
• Dataset Distillation as Data Compression: A Rate-Utility Perspective

  2025cites this paper
• Bitstream Collisions in Neural Image Compression via Adversarial Perturbations

  2025cites this paper
• Learnable Non-Uniform Quantization With Sampling-Based Optimization for Variable-Rate Learned Image Compression

  2025cites this paper
• Coded Deep Learning: Framework and Algorithm

  2025cites this paper
• Remote Inference Over Dynamic Links via Adaptive Rate Deep Task-Oriented Vector Quantization

  2025cites this paper
• 4DGC: Rate-Aware 4D Gaussian Compression for Efficient Streamable Free-Viewpoint Video

  2025cites this paper
• UVC: A Unified Deep Video Compression Framework

  2025cites this paper
• Interleaved Block-based Learned Image Compression with Feature Enhancement and Quantization Error Compensation

  2025cites this paper
• Efficient lightweight image compression approach for Multimedia Internet of Things

  2025cites this paper
• Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey

  2025cites this paper
• Reviving Legacy Seismic Data via Machine Learning Technique—Part 2: Estimating 3-D Seismic Volumes From 2-D Seismic Lines With VQ-VAE

  2025cites this paper
• UGINR: large-scale unstructured grid reduction via implicit neural representation

  2024cites this paper
• Neural-Based Video Compression on Solar Dynamics Observatory Images

  2024cites this paper
• A Versatile Diffusion Transformer with Mixture of Noise Levels for Audiovisual Generation

  2024cites this paper
• Image and Video Tokenization with Binary Spherical Quantization

  2024cites this paper
• Neural Speech Coding for Real-Time Communications Using Constant Bitrate Scalar Quantization

  2024cites this paper
• NLIC: Non-Uniform Quantization-Based Learned Image Compression

  2024cites this paper
• Survey on Visual Signal Coding and Processing With Generative Models: Technologies, Standards, and Optimization

  2024cites this paper
• Vesper: Learning to Manage Uncertainty in Video Streaming

  2024cites this paper
• Power-Aware Task-Based Learning of Neuromorphic ADCs

  2024cites this paper
• High Efficiency Deep-learning Based Video Compression

  2024cites this paper
• Neural Network Assisted Lifting Steps For Improved Fully Scalable Lossy Image Compression in JPEG 2000

  2024cites this paper
• Wavelet-Like Transform-Based Technology in Response to the Call for Proposals on Neural Network-Based Image Coding

  2024cites this paper
• Joint super-resolution-based fast face image coding for human and machine vision

  2024cites this paper
• Composite loss function for 3-D poststack seismic data compression

  2024cites this paper
• Learned Video Compression with Adaptive Temporal Prior and Decoded Motion-aided Quality Enhancement

  2024cites this paper
• Lightweight Context Model Equipped aiWave in Response to the AVS Call for Evidence on Volumetric Medical Image Coding

  2024cites this paper
• Exploration of Learned Lifting-Based Transform Structures for Fully Scalable and Accessible Wavelet-Like Image Compression

  2024cites this paper
• EUICN: An Efficient Underwater Image Compression Network

  2024cites this paper
• Robustness in Wireless Distributed Learning: An Information-Theoretic Analysis

  2024cites this paper
• Information Compression in the AI Era: Recent Advances and Future Challenges

  2024cites this paper
• PatchSVD: A Non-Uniform SVD-Based Image Compression Algorithm

  2024cites this paper
• Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depth

  2024cites this paper
• ESC: Efficient Speech Coding with Cross-Scale Residual Vector Quantized Transformers

  2024cites this paper
• Autoencoder-based joint image compression and encryption

  2024cites this paper
• FLLIC: Functionally Lossless Image Compression

  2024cites this paper
• Seismic data compression: an overview

  2024cites this paper
• Codebook-Based Downlink Beamforming with Imperfect CSI Using Model-Driven Deep Learning

  2024cites this paper
• Residual Quantization with Implicit Neural Codebooks

  2024cites this paper
• Lossy Coding for Spatially Adaptive Conditioning in Semantic Image Communication

  2024cites this paper
• Kaizen: Decomposing cellular images with VQ-VAE

  2024cites this paper
• Human-Machine Collaborative Image and Video Compression: A Survey

  2024cites this paper
• Adaptive Prediction Structure for Learned Video Compression

  2024cites this paper
• Deep Digital Joint Source-Channel Based Wireless Speech Transmission

  2024cites this paper
• BRIC: Bottom-Up Residual Vector Quantization for Learned Image Compression

  2024cites this paper
• Enhancing Learned Image Compression via Cross Window-based Attention

  2024cites this paper
• Exploring adversarial robustness of JPEG AI: methodology, comparison and new methods

  2024cites this paper
• Learned Image Compression Using A Long and Short Attention Module

  2024cites this paper
• EICNet: An End-to-End Efficient Learning-Based Image Compression Network

  2024cites this paper
• Learned Image Compression for Both Humans and Machines via Dynamic Adaptation

  2024cites this paper
• End-to-End Learning-Based Image Compression With a Decoupled Framework

  2024cites this paper
• Learning Task-Based Trainable Neuromorphic ADCs via Power-Aware Distillation

  2024cites this paper
• Prompt Sliders for Fine-Grained Control, Editing and Erasing of Concepts in Diffusion Models

  2024cites this paper
• STanH: Parametric Quantization for Variable Rate Learned Image Compression

  2024cites this paper
• Learning Optimal Lattice Vector Quantizers for End-to-end Neural Image Compression

  2024cites this paper
• Neural Speech and Audio Coding: Modern AI technology meets traditional codecs [Special Issue On Model-Based and Data-Driven Audio Signal Processing]

  2024cites this paper
• Bi-Level Spatial and Channel-aware Transformer for Learned Image Compression

  2024cites this paper
• Sketch-Based Replay Projection for Continual Learning

  2024cites this paper
• Finite-State Autoregressive Entropy Coding for Efficient Learned Lossless Compression

  2024cites this paper
• 466.4-Gbit/s PCS-64QAM wireless transmission enhanced by dual-polarized SISO link and vector-quantized variational autoencoders.

  2024cites this paper
• Rapid and Power-Aware Learned Optimization for Modular Receive Beamforming

  2024cites this paper
• Contemporary Advances in Neural Network Quantization: A Survey

  2024cites this paper
• Learned HDR Image Compression for Perceptually Optimal Storage and Display

  2024cites this paper
• HPC: Hierarchical Progressive Coding Framework for Volumetric Video

  2024cites this paper
• Robustly overfitting latents for flexible neural image compression

  2024cites this paper
• Inducing Systematicity in Transformers by Attending to Structurally Quantized Embeddings

  2024cites this paper
• Hyperspectral image compression based on multiple priors

  2024cites this paper