Lightweight Transformer for Image Interpolation Via Unrolling of Multiple Learned Graph Laplacian Regularizers

We build an interpretable and lightweight transformer-like neural net by unrolling an iterative algorithm that minimizes multiple realizations of the quadratic graph Laplacian regularizer (GLR), subject to an interpolation constraint. The pivotal insight is that a normalized signal-dependent graph learning module amounts to a variation of the self-attention mechanism in conventional transformers. Unlike “blackbox” transformers that require learning of large key, query and value matrices to compute transformed dot products as affinities and output embeddings, we employ shallow CNNs to learn low-dimensional features per pixel to establish pairwise Mahalanobis distances and construct sparse similarity graphs. At each layer, given a learned graph, the target interpolated signal is simply a low-pass filtered output derived from the minimization of GLRs, resulting in a steep reduction in parameter count. Image interpolation experiments demonstrate competitive restoration performance and notable parameter reduction compared to mainstream transformers.

• Publication year

  2025

• Venue

  2025 IEEE International Conference on Image Processing Workshops (ICIPW)

• Publication date

  2025-09-14

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1109/ICIPW68931.2025.11386014
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Interpretable Lightweight Transformer via Unrolling of Learned Graph Smoothness Priors

  2024cited by this paper
• White-Box Transformers via Sparse Rate Reduction

  2023cited by this paper
• Scaling Vision Transformers to 22 Billion Parameters

  2023cited by this paper
• Manifold Graph Signal Restoration Using Gradient Graph Laplacian Regularizer

  2022cited by this paper
• Residual Swin Transformer Channel Attention Network for Image Demosaicing

  2022cited by this paper
• Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

  2021cited by this paper
• SwinIR: Image Restoration Using Swin Transformer

  2021cited by this paper
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

  2020cited by this paper
• Image Interpolation Using Multi-Scale Attention-Aware Inception Network

  2020cited by this paper
• Signed Graph Metric Learning via Gershgorin Disc Perfect Alignment

  2020cited by this paper
• Feature Graph Learning for 3D Point Cloud Denoising

  2019cited by this paper
• Graph Spectral Image Processing

  2018cited by this paper
• Graph Signal Processing: Overview, Challenges, and Applications

  2017influential reference
• Attention is All you Need

  2017cited by this paper
• Graph Laplacian Regularization for Inverse Imaging: Analysis in the Continuous Domain

  2016influential reference
• Images

  2014cited by this paper
• Neural Machine Translation by Jointly Learning to Align and Translate

  2014cited by this paper
• The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains

  2012cited by this paper
• Convex optimization

  2010cited by this paper
• Demosaicing With Directional Filtering and a posteriori Decision

  2007cited by this paper
• Gaussian Markov Random Fields: Theory and Applications

  2005cited by this paper
• High-quality linear interpolation for demosaicing of Bayer-patterned color images

  2004cited by this paper
• Image quality assessment: from error visibility to structural similarity

  2004cited by this paper
• An Introduction to the Conjugate Gradient Method Without the Agonizing Pain

  1994influential reference
• Pattern Recognition and Machine Learning

  1963cited by this paper

• No citing papers are available for this paper.