UST-Net: A U-Shaped Transformer Network Using Shifted Windows for Hyperspectral Unmixing

Autoencoders (AEs) are commonly utilized for acquiring low-dimensional data representations and performing data reconstruction, which makes them suitable for hyperspectral unmixing (HU). However, AE networks trained pixel by pixel and those employing localized convolutional filters disregard the global material distribution and distant interdependencies, resulting in the loss of necessary spatial feature information essential for the unmixing process. To overcome this limitation, we propose an innovative deep neural network model named U-shaped transformer network using shifted windows (UST-Net). UST-Net prioritizes spatial information in the scene that is more discriminative and significant by using multihead self-attention blocks based on shifted windows. Unlike patch-based unmixing networks, UST-Net operates on the complete image, eliminating inconsistencies associated with patches. Moreover, the downsampling and upsampling stages are used to extract hyperspectral image (HSI) feature maps at different scales. This process generates a context-rich and spatially accurate abundance map without losing local details. The experimental results of one synthetic dataset and three real datasets demonstrate that UST-Net significantly outperforms both traditional and several other advanced neural network methods. Our code is publicly available at https://github.com/UPCGIT/UST-Net.

• Publication year

  2023

• Venue

  IEEE Transactions on Geoscience and Remote Sensing

• Publication date

  Unknown publication date

• Fields of study

  Computer Science, Environmental Science

• Identifiers
  DOI 10.1109/TGRS.2023.3321839
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Manifold Regularized Sparse Archetype Analysis Considering Endmember Variability

  2023cited by this paper
• Multi-stage convolutional autoencoder network for hyperspectral unmixing

  2022cited by this paper
• Spectral–Spatial–Temporal Transformers for Hyperspectral Image Change Detection

  2022cited by this paper
• AutoNAS: Automatic Neural Architecture Search for Hyperspectral Unmixing

  2022cited by this paper
• Integration of Physics-Based and Data-Driven Models for Hyperspectral Image Unmixing: A summary of current methods

  2022cited by this paper
• Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review

  2022cited by this paper
• L₁ Sparsity-Constrained Archetypal Analysis Algorithm for Hyperspectral Unmixing

  2022cited by this paper
• Hyperspectral Unmixing Using Transformer Network

  2022influential reference
• A 3D-CNN Framework for Hyperspectral Unmixing with Spectral Variability

  2022cited by this paper
• Multimodal Hyperspectral Unmixing: Insights From Attention Networks

  2022cited by this paper
• Probabilistic Generative Model for Hyperspectral Unmixing Accounting for Endmember Variability

  2022cited by this paper
• Blind Hyperspectral Unmixing Using Autoencoders: A Critical Comparison

  2022cited by this paper
• TANet: An Unsupervised Two-Stream Autoencoder Network for Hyperspectral Unmixing

  2022cited by this paper
• CyCU-Net: Cycle-Consistency Unmixing Network by Learning Cascaded Autoencoders

  2022cited by this paper
• JMnet: Joint Metric Neural Network for Hyperspectral Unmixing

  2022cited by this paper
• Deep Autoencoders With Multitask Learning for Bilinear Hyperspectral Unmixing

  2021cited by this paper
• Convolutional Autoencoder for Spectral-Spatial Hyperspectral Unmixing

  2021cited by this paper
• LSTM-DNN Based Autoencoder Network for Nonlinear Hyperspectral Image Unmixing

  2021cited by this paper
• Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

  2021cited by this paper
• NMF-SAE: An Interpretable Sparse Autoencoder for Hyperspectral Unmixing

  2021cited by this paper
• Adversarial Autoencoder Network for Hyperspectral Unmixing

  2021cited by this paper
• More Diverse Means Better: Multimodal Deep Learning Meets Remote-Sensing Imagery Classification

  2020cited by this paper
• Multiple Clustering Guided Nonnegative Matrix Factorization for Hyperspectral Unmixing

  2020cited by this paper
• Near infrared hyperspectral imaging and spectral unmixing methods for evaluation of fiber distribution in enriched pasta.

  2020cited by this paper
• Improved Collaborative Non-Negative Matrix Factorization and Total Variation for Hyperspectral Unmixing

  2020cited by this paper
• Multiple-Priors Ensemble Constrained Nonnegative Matrix Factorization for Spectral Unmixing

  2020cited by this paper
• Convolutional Autoencoder for Blind Hyperspectral Image Unmixing

  2020cited by this paper
• Graph Convolutional Networks for Hyperspectral Image Classification

  2020cited by this paper
• Deep Generative Endmember Modeling: An Application to Unsupervised Spectral Unmixing

  2019cited by this paper
• DAEN: Deep Autoencoder Networks for Hyperspectral Unmixing

  2019cited by this paper
• uDAS: An Untied Denoising Autoencoder With Sparsity for Spectral Unmixing

  2019influential reference
• Hyperspectral Unmixing for Additive Nonlinear Models With a 3-D-CNN Autoencoder Network

  2019cited by this paper
• Modern Trends in Hyperspectral Image Analysis: A Review

  2018cited by this paper
• Spatial Group Sparsity Regularized Nonnegative Matrix Factorization for Hyperspectral Unmixing

  2017cited by this paper
• Attention is All you Need

  2017cited by this paper
• Geometric N-finder algorithm for finding endmembers in hyperspectral imagery

  2017cited by this paper
• EndNet: Sparse AutoEncoder Network for Endmember Extraction and Hyperspectral Unmixing

  2017cited by this paper
• Recursive Orthogonal Projection-Based Simplex Growing Algorithm

  2016cited by this paper
• Minimum Volume Simplex Analysis: A Fast Algorithm for Linear Hyperspectral Unmixing

  2015cited by this paper
• Robust Minimum Volume Simplex Analysis for Hyperspectral Unmixing

  2014cited by this paper
• Crop Stage Classification of Hyperspectral Data Using Unsupervised Techniques

  2013cited by this paper
• Application of Hyperspectral Imaging in Food Safety Inspection and Control: A Review

  2012cited by this paper
• Chance-Constrained Robust Minimum-Volume Enclosing Simplex Algorithm for Hyperspectral Unmixing

  2011cited by this paper
• Convex analysis based minimum-volume enclosing simplex algorithm for hyperspectral unmixing

  2009cited by this paper
• Pharmaceutical applications of vibrational chemical imaging and chemometrics: a review.

  2008cited by this paper
• Hyperspectral imaging – an emerging process analytical tool for food quality and safety control

  2007cited by this paper
• Airborne hyperspectral imagery and linear spectral unmixing for mapping variation in crop yield

  2007cited by this paper
• Vertex component analysis: a fast algorithm to unmix hyperspectral data

  2005cited by this paper
• Blind separation of sources, Part I" An adaptive algorithm based on neuromimetic architecture

  2002cited by this paper
• Spectral unmixing of vegetation, soil and dry carbon cover in arid regions: Comparing multispectral and hyperspectral observations

  2002cited by this paper
• Spectral unmixing

  2002cited by this paper
• Fully constrained least squares linear spectral mixture analysis method for material quantification in hyperspectral imagery

  2001cited by this paper
• HYDICE: an airborne system for hyperspectral imaging

  1993cited by this paper

• MS2AE-Net: A Multiscale Spectral–Spatial Autoencoder Network for Hyperspectral Unmixing

  2026cites this paper
• GGCT-Net: A Dual-Branch Gated Graph Convolution and Grouped Cross-Attention Transformer Network for Hyperspectral Unmixing

  2026cites this paper
• Hyperspectral unmixing algorithm based on channel multi-scale dual-stream autoencode

  2025cites this paper
• Multiscale Spectral–Spatial Unmixing Network With Boltzmann-Inspired Adaptive Temperature

  2025cites this paper
• Nesterov-accelerated non-negative matrix factorization unrolling network for hyperspectral unmixing

  2025cites this paper
• Preprocessing Algorithm Leveraging Geometric Modeling for Scale Correction in Hyperspectral Images for Improved Unmixing Performance

  2025influential citation
• Mamba-Enhanced Spatial–Spectral Feature Learning for Hyperspectral Unmixing

  2025cites this paper
• Conventional to Deep Learning Methods for Hyperspectral Unmixing: A Review

  2025cites this paper
• Dual-stream feature fusion network with transformer for hyperspectral unmixing

  2025cites this paper
• Latent Dirichlet Transformer VAE for Hyperspectral Unmixing with Bundled Endmembers

  2025cites this paper
• Differential Spectral Guided Deep Hyperspectral Unmixing Network

  2025cites this paper
• Unsupervised Nonlinear Hyperspectral Unmixing using Kernel-Based Autoencoder Network

  2025cites this paper
• MVCUN: A New CNN-Based Autoencoder With Minimum Volume Constraint for Hyperspectral Unmixing

  2025cites this paper
• URDM: Hyperspectral Unmixing Regularized by Diffusion Models

  2025cites this paper
• Hyperspectral Unmixing with 3D Convolutional Sparse Coding and Projected Simplex Volume Maximization

  2025cites this paper
• Dual-Stream Hyperspectral Unmixing Network Combining Transformer and CNN

  2025cites this paper
• SSLT-Net: A Spatial–Spectral Linear Transformer Unmixing Network for Hyperspectral Image

  2025cites this paper
• Dual Embedding Transformer Network for Hyperspectral Unmixing

  2025influential citation
• An Endmember-Oriented Transformer Network for Bundle-Based Hyperspectral Unmixing

  2025cites this paper
• A Joint Multiscale Graph Attention and Classify-Driven Autoencoder Framework for Hyperspectral Unmixing

  2025cites this paper
• Hyperspectral unmixing with spatial context and endmember ensemble learning with attention mechanism

  2025cites this paper
• Pixel-Level and Global Similarity-Based Adversarial Autoencoder Network for Hyperspectral Unmixing

  2025influential citation
• DTU-Net: A Multi-Scale Dilated Transformer Network for Nonlinear Hyperspectral Unmixing

  2025influential citation
• SRViT: Self-Supervised Relation-Aware Vision Transformer for Hyperspectral Unmixing

  2025cites this paper
• A Biobjective Model-Driven Autocoder for Blind Hyperspectral Unmixing

  2025cites this paper
• A Spectral-Spatial Attention Network for Hyperspectral Unmixing

  2025cites this paper
• Mutual Incoherence and Relative Total Variation Regularizations for Blind Hyperspectral Unmixing

  2024cites this paper
• Traditional to Transformers: A Survey on Current Trends and Future Prospects for Hyperspectral Image Classification

  2024cites this paper
• Low-Rank Prompt-Guided Transformer for Hyperspectral Image Denoising

  2024cites this paper
• Transformer for Multitemporal Hyperspectral Image Unmixing

  2024cites this paper
• A Global Spatial-Spectral Feature Fused Autoencoder for Nonlinear Hyperspectral Unmixing

  2024cites this paper
• Feedback Information-Guided Spectral Variability Attention Network for Hyperspectral Unmixing

  2024cites this paper
• Dual-Branch Autoencoder with Clustering Information for Hyperspectral Blind Unmixing

  2024cites this paper
• MSCC-ViT:A Multiscale Visual-Transformer Network Using Convolution Crossing Attention for Hyperspectral Unmixing

  2024cites this paper
• MAT-Net: Multiscale Aggregation Transformer Network for Hyperspectral Unmixing

  2024cites this paper
• A comprehensive survey for Hyperspectral Image Classification: The evolution from conventional to transformers and Mamba models

  2024cites this paper
• TCCU-Net: Transformer and CNN Collaborative Unmixing Network for Hyperspectral Image

  2024cites this paper
• A Supervised Approach for Estimating Fractional Abundances of Binary Intimate Mixtures

  2024cites this paper
• Transformer based Endmember Fusion with Spatial Context for Hyperspectral Unmixing

  2024cites this paper