4D U-Nets for Multi-Temporal Remote Sensing Data Classification

Multispectral sensors constitute a core earth observation imaging technology generating massive high-dimensional observations acquired across multiple time instances. The collected multi-temporal remote sensed data contain rich information for Earth monitoring applications, from flood detection to crop classification. To easily classify such naturally multidimensional data, conventional low-order deep learning models unavoidably toss away valuable information residing across the available dimensions. In this work, we extend state-of-the-art convolutional network models based on the U-Net architecture to their high-dimensional analogs, which can naturally capture multi-dimensional dependencies and correlations. We introduce several model architectures, both of low as well as of high order, and we quantify the achieved classification performance vis-à-vis the latest state-of-the-art methods. The experimental analysis on observations from Landsat-8 reveals that approaches based on low-order U-Net models exhibit poor classification performance and are outperformed by our proposed high-dimensional U-Net scheme.

• Publication year

  2022

• Venue

  Remote Sensing

• Publication date

  2022-01-28

• Fields of study

  Computer Science, Environmental Science

• Identifiers
  DOI 10.3390/rs14030634
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Land cover classification from remote sensing images based on multi-scale fully convolutional network

  2020cited by this paper
• Metrics for Multi-Class Classification: an Overview

  2020cited by this paper
• A Discriminative Long Short Term Memory Network with Metric Learning Applied to Multispectral Time Series Classification

  2020cited by this paper
• Deep Learning for Land Cover Change Detection

  2020influential reference
• V4D: 4D Convolutional Neural Networks for Video-level Representation Learning

  2020cited by this paper
• U-Net Convolutional Networks for Mining Land Cover Classification Based on High-Resolution UAV Imagery

  2020cited by this paper
• Classification of Compressed Remote Sensing Multispectral Images via Convolutional Neural Networks

  2020cited by this paper
• Time-Space Tradeoff in Deep Learning Models for Crop Classification on Satellite Multi-Spectral Image Time Series

  2019cited by this paper
• Land Cover Maps Production with High Resolution Satellite Image Time Series and Convolutional Neural Networks: Adaptations and Limits for Operational Systems

  2019cited by this paper
• 4D CNN for semantic segmentation of cardiac volumetric sequences

  2019cited by this paper
• Deep learning based multi-temporal crop classification

  2019cited by this paper
• Survey of Deep-Learning Approaches for Remote Sensing Observation Enhancement

  2019cited by this paper
• 3D Convolutional Neural Networks for Crop Classification with Multi-Temporal Remote Sensing Images

  2018cited by this paper
• Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders

  2018cited by this paper
• Sentinel-2 cropland mapping using pixel-based and object-based time-weighted dynamic time warping analysis

  2018cited by this paper
• Land Cover Classification from Satellite Imagery with U-Net and Lovász-Softmax Loss

  2018cited by this paper
• Temporal Convolutional Neural Network for the Classification of Satellite Image Time Series

  2018cited by this paper
• Convolutional LSTMs for Cloud-Robust Segmentation of Remote Sensing Imagery

  2018cited by this paper
• Urban Land Use and Land Cover Classification Using Novel Deep Learning Models Based on High Spatial Resolution Satellite Imagery

  2018cited by this paper
• Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data

  2017cited by this paper
• End-to-end Learning of Deep Spatio-temporal Representations for Satellite Image Time Series Classification

  2017cited by this paper
• Operational High Resolution Land Cover Map Production at the Country Scale Using Satellite Image Time Series

  2017cited by this paper
• Deep Learning with Python

  2017cited by this paper
• Automated cropland mapping of continental Africa using Google Earth Engine cloud computing

  2017cited by this paper
• TensorFlow: A system for large-scale machine learning

  2016cited by this paper
• Optical remotely sensed time series data for land cover classification: A review

  2016cited by this paper
• 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation

  2016cited by this paper
• Deep Learning

  2016cited by this paper
• A guide to convolution arithmetic for deep learning

  2016cited by this paper
• Land Use Classification in Remote Sensing Images by Convolutional Neural Networks

  2015cited by this paper
• Crop classification of upland fields using Random forest of time-series Landsat 7 ETM+ data

  2015cited by this paper
• U-Net: Convolutional Networks for Biomedical Image Segmentation

  2015influential reference
• Combined use of multi-seasonal high and medium resolution satellite imagery for parcel-related mapping of cropland and grassland

  2014cited by this paper
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• MAD-MEX: Automatic Wall-to-Wall Land Cover Monitoring for the Mexican REDD-MRV Program Using All Landsat Data

  2014cited by this paper
• Very Deep Convolutional Networks for Large-Scale Image Recognition

  2014cited by this paper
• Deep Learning-Based Classification of Hyperspectral Data

  2014cited by this paper
• Accurate mapping of forest types using dense seasonal Landsat time-series

  2014cited by this paper
• Fully convolutional networks for semantic segmentation

  2014cited by this paper
• Impact of feature selection on the accuracy and spatial uncertainty of per-field crop classification using Support Vector Machines

  2013cited by this paper
• Long-term land cover dynamics by multi-temporal classification across the Landsat-5 record

  2013cited by this paper
• Hyperspectral Remote Sensing Data Analysis and Future Challenges

  2013cited by this paper
• ImageNet classification with deep convolutional neural networks

  2012cited by this paper
• Land change for all municipalities in Latin America and the Caribbean assessed from 250-m MODIS imagery (2001–2010)

  2012cited by this paper
• Temporal segmentation of MODIS time series for improving crop classification in Central Asian irrigation systems

  2011cited by this paper
• Understanding the difficulty of training deep feedforward neural networks

  2010cited by this paper
• Nearest Neighbor Classification of Remote Sensing Images With the Maximal Margin Principle

  2008cited by this paper
• Crop classification by support vector machine with intelligently selected training data for an operational application

  2008cited by this paper
• The use of high‐resolution image time series for crop classification and evapotranspiration estimate over an irrigated area in central Morocco

  2008cited by this paper
• Analysis of time-series MODIS 250 m vegetation index data for crop classification in the U.S. Central Great Plains

  2007cited by this paper
• Mapping paddy rice agriculture in southern China using multi-temporal MODIS images

  2005cited by this paper
• Land cover classification in the Argentine Pampas using multi-temporal Landsat TM data

  2003cited by this paper
• Classification of wheat crop with multi-temporal images: performance of maximum likelihood and artificial neural networks

  2003cited by this paper

• Long-term monitoring and forecasting of glacial lake dynamics using Landsat time series data, Google Earth Engine, machine learning, and geospatial analysis

  2025cites this paper
• Enhancing a Building Change Detection Model in Remote Sensing Imagery for Encroachments and Construction on Government Lands in Egypt as a Case Study

  2025cites this paper
• Exploring Applications of Convolutional Neural Networks in Analyzing Multispectral Satellite Imagery: A Systematic Review

  2025cites this paper
• Aerial Imagery Segmentation of Landcover by Using Deep Learning Algorithm

  2025cites this paper
• Machine learning in multi-spectral thermal imaging for enhanced detection of neurological disorders through thermoplasmonics.

  2025cites this paper
• ViT-ChangeFormer: A Deep Learning Approach for Cropland Abandonment Detection in Lahore, Pakistan Using Landsat-8 and Sentinel-2 Data

  2025cites this paper
• Constructing a knowledge base from remote sensing indicators for deforestation assessment

  2025cites this paper
• Accurate and efficient feature classification of urban public open spaces: A deep learning-based multivariate time-series approach

  2024cites this paper
• Land Cover Segmentation Using 3-D FCN-Based Architecture With Coordinate Attention

  2024cites this paper
• Higher-Order Convolutional Neural Networks for Essential Climate Variables Forecasting

  2024cites this paper
• Unfolding Time: Generative Modeling for Turbulent Flows in 4D

  2024cites this paper
• A fine crop classification model based on multitemporal Sentinel-2 images

  2024cites this paper
• SCECA U-Net crop classification for UAV remote sensing image

  2024cites this paper
• Application of Deep Learning in Multitemporal Remote Sensing Image Classification

  2023cites this paper
• Deep Learning Application for Crop Classification via Multi-Temporal Remote Sensing Images

  2023cites this paper
• Deforestation detection using a spatio-temporal deep learning approach with synthetic aperture radar and multispectral images

  2023cites this paper
• Digital infrared thermal imaging system based breast cancer diagnosis using 4D U-Net segmentation

  2023cites this paper
• A Series-Based Deep Learning Approach to Lung Nodule Image Classification

  2023cites this paper
• A Transferable Learning Classification Model and Carbon Sequestration Estimation of Crops in Farmland Ecosystem

  2022cites this paper