Norm-guided Adaptive Visual Embedding for Zero-Shot Sketch-Based Image Retrieval

Zero-shot sketch-based image retrieval (ZS-SBIR), which aims to retrieve photos with sketches under the zero-shot scenario, has shown extraordinary talents in real-world applications. Most existing methods leverage language models to generate class-prototypes and use them to arrange the locations of all categories in the common space for photos and sketches. Although great progress has been made, few of them consider whether such pre-defined prototypes are necessary for ZS-SBIR, where locations of unseen class samples in the embedding space are actually determined by visual appearance and a visual embedding actually performs better. To this end, we propose a novel Norm-guided Adaptive Visual Embedding (NAVE) model, for adaptively building the common space based on visual similarity instead of language-based pre-defined prototypes. To further enhance the representation quality of unseen classes for both photo and sketch modality, modality norm discrepancy and noisy label regularizer are jointly employed to measure and repair the modality bias of the learned common embedding. Experiments on two challenging datasets demonstrate the superiority of our NAVE over state-of-the-art competitors.

• Publication year

  2021

• Venue

  International Joint Conference on Artificial Intelligence

• Publication date

  2021-08-01

• Fields of study

  Computer Science

• Identifiers
  DOI 10.24963/ijcai.2021/153
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Zero-Shot Sketch-Based Image Retrieval via Graph Convolution Network

  2020cited by this paper
• Progressive Cross-Modal Semantic Network for Zero-Shot Sketch-Based Image Retrieval

  2020cited by this paper
• Adaptive Margin Diversity Regularizer for Handling Data Imbalance in Zero-Shot SBIR

  2020cited by this paper
• Semantic-Aware Knowledge Preservation for Zero-Shot Sketch-Based Image Retrieval

  2019influential reference
• When Does Label Smoothing Help?

  2019cited by this paper
• Learning to Discover Novel Visual Categories via Deep Transfer Clustering

  2019cited by this paper
• Style-Guided Zero-Shot Sketch-based Image Retrieval

  2019cited by this paper
• Zero-Shot Sketch-Image Hashing

  2018influential reference
• Larger Norm More Transferable: An Adaptive Feature Norm Approach for Unsupervised Domain Adaptation

  2018cited by this paper
• Batch-Instance Normalization for Adaptively Style-Invariant Neural Networks

  2018cited by this paper
• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness

  2018cited by this paper
• Recent Advances in Zero-Shot Recognition: Toward Data-Efficient Understanding of Visual Content

  2017cited by this paper
• Zero-Shot Visual Recognition Using Semantics-Preserving Adversarial Embedding Networks

  2017cited by this paper
• Deep Sketch Hashing: Fast Free-Hand Sketch-Based Image Retrieval

  2017influential reference
• Learning to cluster in order to Transfer across domains and tasks

  2017cited by this paper
• DisturbLabel: Regularizing CNN on the Loss Layer

  2016cited by this paper
• Sketch-based image retrieval via Siamese convolutional neural network

  2016cited by this paper
• SketchNet: Sketch Classification with Web Images

  2016cited by this paper
• GloVe: Global Vectors for Word Representation

  2014cited by this paper
• DeViSE: A Deep Visual-Semantic Embedding Model

  2013cited by this paper
• Zero-Shot Learning Through Cross-Modal Transfer

  2013cited by this paper
• Visualizing and Understanding Convolutional Networks

  2013cited by this paper
• Distributed Representations of Words and Phrases and their Compositionality

  2013cited by this paper
• IEEE Transactions on Image Processing

  2004influential reference
• Combining local context and wordnet similarity for word sense identification

  1998cited by this paper
• Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy

  1997cited by this paper
• Verb Semantics and Lexical Selection

  1994cited by this paper

• AdaAlign: A unified solution for traditional and modern zero-shot sketch-based image retrieval.

  2026cites this paper
• A causality-invariant relation learning framework for geometry-driven zero-shot cross-modal retrieval of dermoscopic vascular patterns

  2026cites this paper
• Zero-Shot Sketch-Based Image Retrieval with teacher-guided and student-centered cross-modal bidirectional knowledge distillation

  2025cites this paper
• DCDL: Dual Causal Disentangled Learning for Zero-Shot Sketch-Based Image Retrieval

  2025cites this paper
• Domain disentanglement and fusion based on hyperbolic neural networks for zero-shot sketch-based image retrieval

  2025cites this paper
• Adapter With Textual Knowledge Graph for Zero-Shot Sketch-Based Image Retrieval

  2025cites this paper
• Similar norm more transferable: Rethinking feature norms discrepancy in adversarial domain adaptation

  2024cites this paper
• Zero-Shot Sketch Based Image Retrieval via Modality Capacity Guidance

  2024cites this paper
• AI for Supporting the Freedom of Drawing

  2024cites this paper
• Relation-Aware Meta-Learning for Zero-Shot Sketch-Based Image Retrieval

  2024cites this paper
• Causality-Invariant Interactive Mining for Cross-Modal Similarity Learning

  2024cites this paper
• Zero-shot sketch-based image retrieval via adaptive relation-aware metric learning

  2024cites this paper
• Zero-Shot Sketch-Based Image Retrieval Using StyleGen and Stacked Siamese Neural Networks

  2024cites this paper
• Task-like training paradigm in CLIP for zero-shot sketch-based image retrieval

  2023influential citation
• CSN: Component-Supervised Network for Few-Shot Classification

  2023cites this paper
• Attention-Based Multi-View Feature Collaboration for Decoupled Few-Shot Learning

  2023cites this paper
• Contrastive Learning of Semantic Concepts for Open-set Cross-domain Retrieval

  2023cites this paper
• Feature Fusion and Metric Learning Network for Zero-Shot Sketch-Based Image Retrieval

  2023cites this paper
• If At First You Don't Succeed: Test Time Re-ranking for Zero-shot, Cross-domain Retrieval

  2023cites this paper
• Adapt and Align to Improve Zero-Shot Sketch-Based Image Retrieval

  2023cites this paper
• A review of disentangled representation learning for visual data processing and analysis

  2023cites this paper
• Attention map feature fusion network for Zero-Shot Sketch-based Image Retrieval

  2023cites this paper
• SketchTrans: Disentangled Prototype Learning With Transformer for Sketch-Photo Recognition

  2023cites this paper
• Structure-Aware Semantic-Aligned Network for Universal Cross-Domain Retrieval

  2022influential citation
• Energy-Guided Feature Fusion for Zero-Shot Sketch-Based Image Retrieval

  2022cites this paper
• Deep cross-modal discriminant adversarial learning for zero-shot sketch-based image retrieval

  2022cites this paper
• Towards Unsupervised Sketch-based Image Retrieval

  2021cites this paper
• ACNet: Approaching-and-Centralizing Network for Zero-Shot Sketch-Based Image Retrieval

  2021cites this paper