CS-MUVI: Video compressive sensing for spatial-multiplexing cameras

Compressive sensing (CS)-based spatial-multiplexing cameras (SMCs) sample a scene through a series of coded projections using a spatial light modulator and a few optical sensor elements. SMC architectures are particularly useful when imaging at wavelengths for which full-frame sensors are too cumbersome or expensive. While existing recovery algorithms for SMCs perform well for static images, they typically fail for time-varying scenes (videos). In this paper, we propose a novel CS multi-scale video (CS-MUVI) sensing and recovery framework for SMCs. Our framework features a co-designed video CS sensing matrix and recovery algorithm that provide an efficiently computable low-resolution video preview. We estimate the scene's optical flow from the video preview and feed it into a convex-optimization algorithm to recover the high-resolution video. We demonstrate the performance and capabilities of the CS-MUVI framework for different scenes.

• Publication year

  2012

• Venue

  International Conference on Computational Photography

• Publication date

  2012-04-28

• Fields of study

  Computer Science, Engineering

• Identifiers
  DOI 10.1109/ICCPhot.2012.6215212
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Compressive Acquisition of Linear Dynamical Systems

  2012cited by this paper
• Compressed Sensing

  2012cited by this paper
• Video from a single coded exposure photograph using a learned over-complete dictionary

  2011cited by this paper
• P2C2: Programmable pixel compressive camera for high speed imaging

  2011influential reference
• Residual Reconstruction for Block-Based Compressed Sensing of Video

  2011cited by this paper
• Flexible Voxels for Motion-Aware Videography

  2010cited by this paper
• Coded Strobing Photography : Compressive Sensing of High-speed Periodic Events

  2010cited by this paper
• Moving gradients: a path-based method for plausible image interpolation

  2009cited by this paper
• Beyond pixels: exploring new representations and applications for motion analysis

  2009cited by this paper
• SubEdit: a representation for editing measured heterogeneous subsurface scattering

  2009cited by this paper
• A multiscale framework for Compressive Sensing of video

  2009cited by this paper
• Kalman filtered Compressed Sensing

  2008cited by this paper
• Single-pixel imaging via compressive sampling

  2008influential reference
• Probing the Pareto Frontier for Basis Pursuit Solutions

  2008cited by this paper
• Multiplexing for Optimal Lighting

  2007cited by this paper
• Compressive imaging for video representation and coding

  2006cited by this paper
• Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information

  2004cited by this paper
• Determining Optical Flow

  1981cited by this paper
• Hadamard transform optics

  1979cited by this paper

• Multivideo Stitching Method Based on Temporal Compressive Sensing

  2025cites this paper
• Structured residual sparsity for video compressive sensing reconstruction

  2024cites this paper
• AdaCS: Adaptive Compressive Sensing With Restricted Isometry Property-Based Error-Clamping

  2024cites this paper
• A Wavelet-Domain Consistency-Constrained Compressive Sensing Framework Based on Memory-Boosted Guidance Filtering

  2024cites this paper
• Adaptive Compressive Tactile Subsampling: Enabling High Spatiotemporal Resolution in Scalable Robotic Skin

  2024cites this paper
• Iterative shrinkage thresholding-based anti-multi-noise compression perceptual image reconstruction network

  2024cites this paper
• LUCMT: Learnable under-sampling and reconstructed network with cross multi-head attention transformer for accelerating MR image reconstruction

  2024cites this paper
• Gates-Controlled Deep Unfolding Network for Image Compressed Sensing

  2024cites this paper
• Optimization-Inspired Cross-Attention Transformer for Compressive Sensing

  2023cites this paper
• Dynamic Path-Controllable Deep Unfolding Network for Compressive Sensing

  2023cites this paper
• UCSNet: Priors Guided Adaptive Compressive Sensing Framework for Underwater Images

  2023cites this paper
• Deep Memory-Augmented Proximal Unrolling Network for Compressive Sensing

  2023cites this paper
• Recognition-Oriented Image Compressive Sensing With Deep Learning

  2023cites this paper
• InSPIRED: single pixel imaging regularized by EBS data

  2023cites this paper
• Soft threshold iteration-based anti-noise compressed sensing image reconstruction network

  2023cites this paper
• SODAS-Net: Side-Information-Aided Deep Adaptive Shrinkage Network for Compressive Sensing

  2023cites this paper
• Novel Copper Oxide Phyto-Nanocatalyst Utilized for the Synthesis of Sustainable Biodiesel from Citrullus colocynthis Seed Oil

  2023cites this paper
• Simultaneous compressed sensing and single-image super resolution for SAR image reconstruction

  2023cites this paper
• CS-Audio: A 16 pJ/b 0.1–15 Mbps Compressive Sensing IC With DWT Sparsifier for Audio-AR

  2022cites this paper
• Foveated Thermal Computational Imaging in the Wild Using All-Silicon Meta-Optics

  2022cites this paper
• Tree-Structured Dilated Convolutional Networks for Image Compressed Sensing

  2022cites this paper
• Iterative Progressive-hypothesis Prediction for Forward Interframe Reconstruction of Video Compressive Sensing

  2022cites this paper
• Adaptive and Cascaded Compressive Sensing

  2022cites this paper
• Multiscale deep network for compressive sensing image reconstruction

  2022cites this paper
• Dynamic proximal unrolling network for compressive imaging

  2021cites this paper
• Memory-Augmented Deep Unfolding Network for Compressive Sensing

  2021cites this paper
• Dynamic Proximal Unrolling Network for Compressive Sensing Imaging

  2021cites this paper
• Neural Sensors: Learning Pixel Exposures for HDR Imaging and Video Compressive Sensing With Programmable Sensors

  2020cites this paper
• Video compressive sensing reconstruction via long-short-term double-pattern prediction

  2020cites this paper
• LSHR-Net: a hardware-friendly solution for high-resolution computational imaging using a mixed-weights neural network

  2020cites this paper
• Perceptual spatial-temporal video compressive sensing network

  2020cites this paper
• Optimization-Inspired Compact Deep Compressive Sensing

  2020cites this paper
• Video compressed sensing reconstruction based on structural group sparsity and successive approximation estimation model

  2020cites this paper
• Single-Pixel MEMS Imaging Systems

  2020cites this paper
• Measurement Matrix Optimization via Mutual Coherence Minimization for Compressively Sensed Signals Reconstruction

  2020cites this paper
• Feature fusion via dual-resolution compressive measurement matrix analysis for spectral image classification

  2020cites this paper
• Ultrafast Imaging With Optical Encoding and Compressive Sensing

  2019cites this paper
• Sinusoidal Sampling Enhanced Compressive Camera for High Speed Imaging

  2019cites this paper
• Real-time Motion Detection in Extremely Subsampled Compressive Sensing Video

  2019cites this paper
• Single-Pixel Imaging and Its Application in Three-Dimensional Reconstruction: A Brief Review

  2019cites this paper
• Turn a Silicon Camera Into an InGaAs Camera

  2019cites this paper
• Improving the Signal-to-Noise Ratio of Superresolution Imaging Based on Single-Pixel Camera

  2019cites this paper
• Real-time Compressive Video Reconstruction for Spatial Multiplexing Cameras

  2019cites this paper
• Reconstruction-Free Compressive Vision for Surveillance Applications

  2019cites this paper
• Single-Pixel Camera Sensing Matrix Design for Hierarchical Compressed Spectral Clustering

  2019cites this paper
• High Resolution 2 D Imaging and 3 D Scanning with Line Sensors

  2018cites this paper
• Principles and prospects for single-pixel imaging

  2018cites this paper
• Collective noise model for focal plane modulated single-pixel imaging

  2018cites this paper
• A Hardware Realization of Superresolution Combining Random Coding and Blurring

  2018cites this paper
• Video compressive sensing using Russian dolls ordering of Hadamard basis for multi-scale sampling of a scene in motion using a single pixel camera

  2018cites this paper
• Single-pixel imaging using balanced detection and a digital micromirror device

  2018cites this paper
• High spatial resolution time-of-flight imaging

  2018cites this paper
• Single-Pixel Imaging Using the Hadamard Transform

  2018cites this paper
• Solving Inverse Computational Imaging Problems Using Deep Pixel-Level Prior

  2018cites this paper
• Tempo-Spatial Compressed Sensing of Organ-on-a-Chip for Pervasive Health

  2018cites this paper
• Principles and prospects for single-pixel imaging

  2018cites this paper
• A High-Speed Imaging Method Based on Compressive Sensing for Sound Extraction Using a Low-Speed Camera

  2018cites this paper
• Ultra-fast imaging with optical encoding and compressive sensing

  2018cites this paper
• Phase modulation in super-resolution microscopy

  2018cites this paper
• Two-layer compressive sensing based video encoding and decoding framework for WMSN

  2018cites this paper
• Snapshot Compressed Sensing: Performance Bounds and Algorithms

  2018cites this paper
• Space-Time-Brightness Sampling Using an Adaptive Pixel-Wise Coded Exposure

  2018cites this paper
• Single-pixel compressive imaging based on motion compensation

  2018cites this paper
• Video Compressive Sensing Reconstruction via Reweighted Residual Sparsity

  2017cites this paper
• ISTA-Net: Iterative Shrinkage-Thresholding Algorithm Inspired Deep Network for Image Compressive Sensing

  2017cites this paper
• Infrared single‐pixel video imaging for sea surface targets surveillance

  2017cites this paper
• ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing

  2017cites this paper
• Compressive video sensing with side information.

  2017cites this paper
• A tempo-spatial compressed sensing architecture for efficient high-throughput information acquisition in organs-on-a-chip

  2017cites this paper
• Recovery of Structured Signals With Prior Information via Maximizing Correlation

  2017cites this paper
• Spatio-spectro-temporal coded aperture design for multiresolution compressive spectral video sensing

  2017cites this paper
• Real-time computational photon-counting LiDAR

  2017cites this paper
• Computer Vision from Spatial-Multiplexing Cameras at Low Measurement Rates

  2017cites this paper
• Compressed Sensing With Prior Information: Strategies, Geometry, and Bounds

  2017cites this paper
• A Russian Dolls ordering of the Hadamard basis for compressive single-pixel imaging

  2017cites this paper
• High-speed imaging using CMOS image sensor with quasi pixel-wise exposure

  2016cites this paper
• Tensor low-rank and sparse light field photography

  2016cites this paper
• Adaptive-Rate Reconstruction of Time-Varying Signals With Application in Compressive Foreground Extraction

  2016cites this paper
• ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Measurements

  2016cites this paper
• Hierarchical frame based spatial-temporal recovery for video compressive sensing coding

  2016cites this paper
• Compressive image recovery using recurrent generative model

  2016cites this paper
• CSVideoNet: A Real-Time End-to-End Learning Framework for High-Frame-Rate Video Compressive Sensing

  2016cites this paper
• Fast Integral Image Estimation at 1% measurement rate

  2016cites this paper
• ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Random Measurements

  2016cites this paper
• Compressed sensing super resolution of color images

  2016cites this paper
• DeepBinaryMask: Learning a Binary Mask for Video Compressive Sensing

  2016cites this paper
• Operator Splitting Methods in Compressive Sensing and Sparse Approximation

  2016cites this paper
• CSVideoNet: A Recurrent Convolutional Neural Network for Compressive Sensing Video Reconstruction

  2016cites this paper
• Compressive sensing super resolution from multiple observations with application to passive millimeter wave images

  2016cites this paper
• Compressive Sensing Based Soft Video Broadcast Using Spatial and Temporal Sparsity

  2016cites this paper
• High-Speed Imaging Using CMOS Image Sensor With Quasi Pixel-Wise Exposure

  2016cites this paper
• Enhanced super-resolution microscopy by phase modulation by Wenxiao Wang

  2016cites this paper
• Adaptive video compressed sampling in the wavelet domain

  2016cites this paper
• Temporal Coding of Volumetric Imagery

  2016cites this paper
• Image super resolution using compressed sensing observations

  2016cites this paper
• Direct classification from compressively sensed images via deep Boltzmann machine

  2016cites this paper
• Video compressive sensing with on-chip programmable subsampling

  2015cites this paper
• Simultaneous real-time visible and infrared video with single-pixel detectors

  2015cites this paper
• Cross-Scale Predictive Dictionaries

  2015cites this paper
• High spatio-temporal resolution video with compressed sensing.

  2015cites this paper