Hierarchical Spatial-Frequency Aggregation for Spectral Deconvolution Imaging

Computational spectral imaging (CSI) achieves real-time hyperspectral imaging through co-designed optics and algorithms, but typical CSI methods suffer from a bulky footprint and limited fidelity. Therefore, Spectral Deconvolution imaging (SDI) methods based on PSF engineering have been proposed to achieve high-fidelity compact CSI design recently. However, the composite convolution-integration operations of SDI render the normal-equation coefficient matrix scene-dependent, which hampers the efficient exploitation of imaging priors and poses challenges for accurate reconstruction. To tackle the inherent data-dependent operators in SDI, we introduce a Hierarchical Spatial-Spectral Aggregation Unfolding Framework (HSFAUF). By decomposing subproblems and projecting them into the frequency domain, HSFAUF transforms nonlinear processes into linear mappings, thereby enabling efficient solutions. Furthermore, to integrate spatial-spectral priors during iterative refinement, we propose a Spatial-Frequency Aggregation Transformer (SFAT), which explicitly aggregates information across spatial and frequency domains. By integrating SFAT into HSFAUF, we develop a Transformer-based deep unfolding method, \textbf{H}ierarchical \textbf{S}patial-\textbf{F}requency \textbf{A}ggregation \textbf{U}nfolding \textbf{T}ransformer (HSFAUT), to solve the inverse problem of SDI. Systematic simulated and real experiments show that HSFAUT surpasses SOTA methods with cheaper memory and computational costs, while exhibiting optimal performance on different SDI systems.

• Publication year

  2025

• Venue

  arXiv.org

• Publication date

  2025-11-10

• Fields of study

  Physics, Computer Science, Engineering, Environmental Science

• Identifiers
  DOI 10.48550/arXiv.2511.06751arXiv 2511.06751
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Improving Spectral Snapshot Reconstruction with Spectral-Spatial Rectification

  2024cited by this paper
• Efficient Snapshot Spectral Imaging: Calibration-Free Parallel Structure with Aperture Diffraction Fusion

  2024cited by this paper
• Split-Aperture 2-in-1 Computational Cameras

  2024cited by this paper
• Non-Serial Quantization-Aware Deep Optics for Snapshot Hyperspectral Imaging

  2024influential reference
• Avian eye–inspired perovskite artificial vision system for foveated and multispectral imaging

  2024cited by this paper
• Heat-assisted detection and ranging

  2023cited by this paper
• Video-rate hyperspectral camera based on a CMOS-compatible random array of Fabry–Pérot filters

  2023influential reference
• A broadband hyperspectral image sensor with high spatio-temporal resolution

  2023influential reference
• Latent Diffusion Prior Enhanced Deep Unfolding for Snapshot Spectral Compressive Imaging

  2023cited by this paper
• Aperture Diffraction for Compact Snapshot Spectral Imaging

  2023influential reference
• Pixel Adaptive Deep Unfolding Transformer for Hyperspectral Image Reconstruction

  2023cited by this paper
• Handheld snapshot multi-spectral camera at tens-of-megapixel resolution

  2023cited by this paper
• Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction

  2022cited by this paper
• Residual Degradation Learning Unfolding Framework with Mixing Priors Across Spectral and Spatial for Compressive Spectral Imaging

  2022cited by this paper
• NTIRE 2022 Spectral Demosaicing Challenge and Data Set

  2022cited by this paper
• Ultraspectral Imaging Based on Metasurfaces with Freeform Shaped Meta‐Atoms

  2022cited by this paper
• Hyperspectral Image Reconstruction Based on the Fusion of Diffracted Rotation Blurred and Clear Images

  2022cited by this paper
• Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging

  2022influential reference
• MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction

  2022cited by this paper
• Recurrent Neural Networks for Snapshot Compressive Imaging

  2022cited by this paper
• HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging

  2022cited by this paper
• Deep Gaussian Scale Mixture Prior for Spectral Compressive Imaging

  2021cited by this paper
• Multi-Stage Progressive Image Restoration

  2021cited by this paper
• A review of deep learning used in the hyperspectral image analysis for agriculture

  2021cited by this paper
• Computational Spectral Imaging With Diffractive Lenses And Spectral Filter Arrays

  2021cited by this paper
• Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction

  2021influential reference
• Restormer: Efficient Transformer for High-Resolution Image Restoration

  2021cited by this paper
• Shift-variant color-coded diffractive spectral imaging system: supplement

  2021cited by this paper
• Hyperspectral/Multispectral Imaging

  2020cited by this paper
• Learning Enriched Features for Real Image Restoration and Enhancement

  2020cited by this paper
• High-Resolution Multi-Spectral Imaging With Diffractive Lenses and Learned Reconstruction

  2020cited by this paper
• End-to-End Low Cost Compressive Spectral Imaging with Spatial-Spectral Self-Attention

  2020cited by this paper
• Spectral DiffuserCam: lensless snapshot hyperspectral imaging with a spectral filter array

  2020influential reference
• Compact snapshot hyperspectral imaging with diffracted rotation

  2019influential reference
• Hyperspectral Imaging With Random Printed Mask

  2019cited by this paper
• Compressive spectral imaging with diffractive lenses.

  2019cited by this paper
• Single-shot on-chip spectral sensors based on photonic crystal slabs

  2019cited by this paper
• Ultrasensitive hyperspectral imaging and biodetection enabled by dielectric metasurfaces

  2019cited by this paper
• Imaging-based molecular barcoding with pixelated dielectric metasurfaces

  2018cited by this paper
• High-quality hyperspectral reconstruction using a spectral prior

  2017cited by this paper
• Hyperspectral Image Superresolution by Transfer Learning

  2017cited by this paper
• SGDR: Stochastic Gradient Descent with Warm Restarts

  2016influential reference
• Computational Snapshot Multispectral Cameras: Toward dynamic capture of the spectral world

  2016cited by this paper
• U-Net: Convolutional Networks for Biomedical Image Segmentation

  2015cited by this paper
• Snapshot colored compressive spectral imager.

  2015cited by this paper
• Multi-spectral compressive snapshot imaging using RGB image sensors.

  2015cited by this paper
• Compressive spectral imaging with colored-patterned detectors

  2014cited by this paper
• Dual-coded compressive hyperspectral imaging.

  2014cited by this paper
• Colored Coded Aperture Design by Concentration of Measure in Compressive Spectral Imaging

  2014cited by this paper
• Review of spectral imaging technology in biomedical engineering: achievements and challenges

  2013cited by this paper
• High resolution multispectral video capture with a hybrid camera system

  2011cited by this paper
• Single disperser design for coded aperture snapshot spectral imaging.

  2008influential reference
• Concept of a high-resolution miniature spectrometer using an integrated filter array.

  2007cited by this paper
• Multispectral Imaging Using Multiplexed Illumination

  2007cited by this paper
• Single-shot compressive spectral imaging with a dual-disperser architecture.

  2007influential reference
• Computed-tomography imaging spectrometer: experimental calibration and reconstruction results.

  1995influential reference
• Imaging Spectrometry for Earth Remote Sensing

  1985cited by this paper
• Limitations of Data-Driven Spectral Reconstruction – Optics-Aware Analysis and Mitigation

  year unknowncited by this paper
• Submitted to Ieee Transactions on Pattern Analysis and Machine Intelligence 1 a Prism-mask System for Multispectral Video Acquisition

  year unknowninfluential reference

• No citing papers are available for this paper.