Differential Angular Imaging for Material Recognition

Material recognition for real-world outdoor surfaces has become increasingly important for computer vision to support its operation in the wild. Computational surface modeling that underlies material recognition has transitioned from reflectance modeling using in-lab controlled radiometric measurements to image-based representations based on internet-mined images of materials captured in the scene. We propose to take a middle-ground approach for material recognition that takes advantage of both rich radiometric cues and flexible image capture. We realize this by developing a framework for differential angular imaging, where small angular variations in image capture provide an enhanced appearance representation and significant recognition improvement. We build a large-scale material database, Ground Terrain in Outdoor Scenes (GTOS) database, geared towards real use for autonomous agents. The database consists of over 30,000 images covering 40 classes of outdoor ground terrain under varying weather and lighting conditions. We develop a novel approach for material recognition called a Differential Angular Imaging Network (DAIN) to fully leverage this large dataset. With this novel network architecture, we extract characteristics of materials encoded in the angular and spatial gradients of their appearance. Our results show that DAIN achieves recognition performance that surpasses single view or coarsely quantized multiview images. These results demonstrate the effectiveness of differential angular imaging as a means for flexible, in-place material recognition.

• Publication year

  2016

• Venue

  Computer Vision and Pattern Recognition

• Publication date

  2016-12-07

• Fields of study

  Materials Science, Computer Science, Engineering

• Identifiers
  DOI 10.1109/CVPR.2017.734arXiv 1612.02372
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• A 4D Light-Field Dataset and CNN Architectures for Material Recognition

  2016influential reference
• The Information Available to a Moving Observer on Shape with Unknown, Isotropic BRDFs

  2016cited by this paper
• Deep TEN: Texture Encoding Network

  2016cited by this paper
• SVBRDF-Invariant Shape and Reflectance Estimation from Light-Field Cameras

  2016cited by this paper
• Friction from Reflectance: Deep Reflectance Codes for Predicting Physical Surface Properties from One-Shot In-Field Reflectance

  2016cited by this paper
• Geometry-Informed Material Recognition

  2016influential reference
• Fine-Grained Material Classification Using Micro-geometry and Reflectance

  2016cited by this paper
• Capturing Computational Appearance: More than meets the eye

  2016cited by this paper
• Convolutional Two-Stream Network Fusion for Video Action Recognition

  2016cited by this paper
• Simultaneous Estimation of Near IR BRDF and Fine-Scale Surface Geometry

  2016cited by this paper
• Automatically discovering local visual material attributes

  2015cited by this paper
• Deep Residual Learning for Image Recognition

  2015cited by this paper
• Deep filter banks for texture recognition and segmentation

  2015cited by this paper
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

  2015cited by this paper
• Multi-view Convolutional Neural Networks for 3D Shape Recognition

  2015influential reference
• Reflectance hashing for material recognition

  2015cited by this paper
• Bilinear CNN Models for Fine-Grained Visual Recognition

  2015cited by this paper
• Return of the Devil in the Details: Delving Deep into Convolutional Nets

  2014cited by this paper
• Learning Spatiotemporal Features with 3D Convolutional Networks

  2014cited by this paper
• Material recognition in the wild with the Materials in Context Database

  2014cited by this paper
• Template‐Based Sampling of Anisotropic BRDFs

  2014cited by this paper
• Material Classification Based on Training Data Synthesized Using a BTF Database

  2014cited by this paper
• Two-Stream Convolutional Networks for Action Recognition in Videos

  2014cited by this paper
• Deep visual-semantic alignments for generating image descriptions

  2014cited by this paper
• On Differential Photometric Reconstruction for Unknown, Isotropic BRDFs

  2013cited by this paper
• Describing Textures in the Wild

  2013cited by this paper
• Visual Material Traits: Recognizing Per-Pixel Material Context

  2013cited by this paper
• Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

  2013cited by this paper
• ImageNet classification with deep convolutional neural networks

  2012cited by this paper
• The Scale of Geometric Texture

  2012cited by this paper
• Efficient Additive Kernels via Explicit Feature Maps

  2012cited by this paper
• UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild

  2012cited by this paper
• Discriminative illumination: Per-pixel classification of raw materials based on optimal projections of spectral BRDF

  2012cited by this paper
• Toward Robust Material Recognition for Everyday Objects

  2011cited by this paper
• Efficient additive kernels via explicit feature maps

  2010cited by this paper
• Material perception: What can you see in a brief glance?

  2010cited by this paper
• Exploring features in a Bayesian framework for material recognition

  2010cited by this paper
• ImageNet: A large-scale hierarchical image database

  2009cited by this paper
• Material classification using BRDF slices

  2009cited by this paper
• Material Classification Using Color and NIR Images

  2009cited by this paper
• On the Significance of Real-World Conditions for Material Classification

  2004cited by this paper
• Device for convenient measurement of spatially varying bidirectional reflectance.

  2004cited by this paper
• Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons

  2001cited by this paper
• Recognition methods for 3D textured surfaces

  2001cited by this paper
• Acquiring the Reflectance Field of a Human Face

  2000cited by this paper
• Reflectance and texture of real-world surfaces

  1999cited by this paper
• Light field rendering

  1996cited by this paper
• Measuring and modeling anisotropic reflection

  1992cited by this paper
• Geometrical considerations and nomenclature for reflectance

  1977influential reference

• RF-MatID: Dataset and Benchmark for Radio Frequency Material Identification

  2026cites this paper
• IFAP: Input-Frequency Adaptive Adversarial Perturbation via Full-Spectrum Envelope Constraint for Spectral Fidelity

  2025cites this paper
• Surface multi-spectral reflectivity and texture material recognition in non-constrained environments

  2025cites this paper
• Chebyshev Attention Depth Permutation Texture Network with Latent Texture Attribute Loss

  2025cites this paper
• Hierarchical Material Recognition from Local Appearance

  2025cites this paper
• Light Field Depth Estimation for Non-Lambertian Objects via Adaptive Cross Operator

  2024cites this paper
• RGB Road Scene Material Segmentation

  2024cites this paper
• Generate Any Scene: Evaluating and Improving Text-to-Vision Generation with Scene Graph Programming

  2024cites this paper
• Beyond Appearances: Material Segmentation with Embedded Spectral Information from RGB-D imagery

  2024cites this paper
• Optimization of lighting design on façades of residential buildings of heritage interest

  2024cites this paper
• Generate Any Scene: Scene Graph Driven Data Synthesis for Visual Generation Training

  2024cites this paper
• Material semantic segmentation algorithm based on polarization information

  2024cites this paper
• Material Attribute Estimation as Part of Telecommunication Augmented Reality, Virtual Reality, and Mixed Reality System: Systematic Review

  2024cites this paper
• Automated Aircraft Recognition via Vision Transformers

  2023cites this paper
• Socially Cognizant Robotics for a Technology Enhanced Society

  2023cites this paper
• On Exploring Multiplicity of Primitives and Attributes for Texture Recognition in the Wild

  2023influential citation
• Enhancing Outdoor Mobility and Environment Perception for Visually Impaired Individuals Through a Customized CNN-based System

  2023cites this paper
• Enhancing texture representation with deep tracing pattern encoding

  2023cites this paper
• TNES: terrain traversability mapping, navigation and excavation system for autonomous excavators on worksite

  2023cites this paper
• Material recognition using time of flight Lidar surface analysis

  2023cites this paper
• HD Ground - A Database for Ground Texture Based Localization

  2022cites this paper
• A Dense Material Segmentation Dataset for Indoor and Outdoor Scene Parsing

  2022cites this paper
• Monocular vision-based multiinformation map construction framework for field terrain

  2022cites this paper
• COFENet: Co-Feature Neural Network Model for Fine-Grained Image Classification

  2022cites this paper
• Self-Supervised Learning for Texture Classification Using Limited Labeled Data

  2022cites this paper
• VINet: Visual and Inertial-based Terrain Classification and Adaptive Navigation over Unknown Terrain

  2022cites this paper
• Frequency-Tuned Universal Adversarial Attacks on Texture Recognition

  2022influential citation
• Recognizing object surface materials to adapt robotic disinfection in infrastructure facilities

  2022cites this paper
• Egocentric vision-based detection of surfaces: towards context-aware free-living digital biomarkers for gait and fall risk assessment

  2022cites this paper
• Tactile Classification of Object Materials for Virtual Reality based Robot Teleoperation

  2022cites this paper
• Multimodal Material Segmentation

  2022cites this paper
• Material Type Recognition of Indoor Scenes via Surface Reflectance Estimation

  2021cites this paper
• T1K+: A Database for Benchmarking Color Texture Classification and Retrieval Methods

  2021cites this paper
• GANav: Group-wise Attention for Classifying Navigable Regions in Unstructured Outdoor Environments

  2021cites this paper
• Multimodal Surface Material Classification Based on Ensemble Learning with Optimized Features

  2021cites this paper
• Surface IR Reflectance Estimation and Material Recognition using ToF Camera

  2021cites this paper
• A Study on Ensemble feature learning for material recognition

  2021cites this paper
• Material Classification in Construction Sites

  2021cites this paper
• GLAVNet: Global-Local Audio-Visual Cues for Fine-Grained Material Recognition

  2021cites this paper
• Autonomous nondestructive evaluation of resistance spot welded joints

  2021cites this paper
• TTM: Terrain Traversability Mapping for Autonomous Excavator Navigation in Unstructured Environments

  2021cites this paper
• Deep Texture Recognition via Exploiting Cross-Layer Statistical Self-Similarity‡

  2021cites this paper
• Self-Supervised Material and Texture Representation Learning for Remote Sensing Tasks

  2021cites this paper
• A Photorealistic Terrain Simulation Pipeline for Unstructured Outdoor Environments

  2021cites this paper
• Stochastic Partial Swap: Enhanced Model Generalization and Interpretability for Fine-grained Recognition

  2021cites this paper
• TNS: Terrain Traversability Mapping and Navigation System for Autonomous Excavators

  2021cites this paper
• Deep Texture Exemplar Extraction Based on Trimmed T-CNN

  2020cites this paper
• Multimodal medical volumes translation and segmentation with generative adversarial network

  2020cites this paper
• One-Shot Generative Adversarial Learning for MRI Segmentation of Craniomaxillofacial Bony Structures

  2020cites this paper
• Deep Material Recognition in Light-Fields via Disentanglement of Spatial and Angular Information

  2020influential citation
• Towards Imperceptible Universal Attacks on Texture Recognition

  2020influential citation
• Probabilistic Surface Friction Estimation Based on Visual and Haptic Measurements

  2020cites this paper
• MATERIAL CLASSIFICATION SYSTEM: LITERATURE SURVEY

  2020cites this paper
• MATERIAL CLASSIFICATION SYSTEM – A STUDY

  2020cites this paper
• A Study for Universal Adversarial Attacks on Texture Recognition

  2020influential citation
• Differential Viewpoints for Ground Terrain Material Recognition

  2020cites this paper
• Gathering Effective Information for Real-Time Material Recognition

  2020cites this paper
• Self-Supervised Learning of a Biologically-Inspired Visual Texture Model

  2020cites this paper
• Illuminance Compensation and Texture Enhancement via the Hodge Decomposition

  2020cites this paper
• Teaching Cameras to Feel: Estimating Tactile Physical Properties of Surfaces From Images

  2020influential citation
• 4D Light Field Segmentation From Light Field Super-Pixel Hypergraph Representation

  2020cites this paper
• Fabric surface characterization: assessment of deep learning-based texture representations using a challenging dataset

  2020influential citation
• Generalized Operational Classifiers for Material Identification

  2020cites this paper
• A Data-centric Approach to Unsupervised Texture Segmentation Using Principle Representative Patterns

  2019cites this paper
• Fine-grained Recognition: Accounting for Subtle Differences between Similar Classes

  2019influential citation
• Fusion of Deep Feature and Hand-Crafted Features for Terrain Recognition

  2019cites this paper
• Deep Multiple-Attribute-Perceived Network for Real-World Texture Recognition

  2019cites this paper
• DICENet: Fine-Grained Recognition via Dilated Iterative Contextual Encoding

  2019cites this paper
• Feature-Level and Model-Level Audiovisual Fusion for Emotion Recognition in the Wild

  2019cites this paper
• Material Segmentation of Multi-View Satellite Imagery

  2019cites this paper
• Multispectral Imaging for Fine-Grained Recognition of Powders on Complex Backgrounds

  2019cites this paper
• Surface Material Retrieval Using Weakly Paired Cross-Modal Learning

  2019cites this paper
• Textile Retrieval Based on Image Content from CDC and Webcam Cameras in Indoor Environments

  2018cites this paper
• Ensemble feature learning for material recognition with convolutional neural networks

  2018cites this paper
• Texture description using multi-scale morphological GLCM

  2018cites this paper
• Computational Texture and Patterns: From Textons to Deep Learning

  2018cites this paper
• Density-Aware Single Image De-raining Using a Multi-stream Dense Network

  2018cites this paper
• Towards Universal Representation for Unseen Action Recognition

  2018cites this paper
• Deep Texture Manifold for Ground Terrain Recognition

  2018influential citation
• DEFO-NET: Learning Body Deformation Using Generative Adversarial Networks

  2018influential citation
• 3D-PhysNet: Learning the Intuitive Physics of Non-Rigid Object Deformations

  2018cites this paper
• ActivityNet Kernelised Representation Generalised Multiple Instance Learning Deep Features NMF with JSD Transfer Joint Matching Actions in Unknown Datasets Word 2 Vec New Concepts Deep Network Matching 1 2 3

  2018cites this paper
• PhotoShape

  2018cites this paper
• Combining Deep Feature and Handcrafted Features for Material Classification

  2018influential citation
• From BoW to CNN: Two Decades of Texture Representation for Texture Classification

  2018cites this paper
• Group-Level Emotion Recognition using Deep Models with A Four-stream Hybrid Network

  2018cites this paper
• Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Network

  2018influential citation
• Island Loss for Learning Discriminative Features in Facial Expression Recognition

  2017cites this paper
• Light Field Segmentation From Super-pixel Graph Representation

  2017cites this paper
• Reflectance and texture encoding for material recognition and synthesis

  2017cites this paper
• Recognizing Textures with Mobile Cameras for Pedestrian Safety Applications

  2017cites this paper
• Deep TEN: Texture Encoding Network

  2016cites this paper
• Invited Plenary Talks

  year unknowncites this paper
• Hierarchical Material Recognition from Local Appearance Supplementary Material

  year unknowncites this paper