Zero-Shot Learning via Joint Latent Similarity Embedding

Zero-shot recognition (ZSR) deals with the problem of predicting class labels for target domain instances based on source domain side information (e.g. attributes) of unseen classes. We formulate ZSR as a binary prediction problem. Our resulting classifier is class-independent. It takes an arbitrary pair of source and target domain instances as input and predicts whether or not they come from the same class, i.e. whether there is a match. We model the posterior probability of a match since it is a sufficient statistic and propose a latent probabilistic model in this context. We develop a joint discriminative learning framework based on dictionary learning to jointly learn the parameters of our model for both domains, which ultimately leads to our class-independent classifier. Many of the existing embedding methods can be viewed as special cases of our probabilistic model. On ZSR our method shows 4.90% improvement over the state-of-the-art in accuracy averaged across four benchmark datasets. We also adapt ZSR method for zero-shot retrieval and show 22.45% improvement accordingly in mean average precision (mAP).

• Publication year

  2015

• Venue

  Computer Vision and Pattern Recognition

• Publication date

  2015-11-14

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1109/CVPR.2016.649arXiv 1511.04512
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Latent Embeddings for Zero-Shot Classification

  2016cited by this paper
• Zero-shot object recognition by semantic manifold distance

  2015cited by this paper
• Semi-Supervised Zero-Shot Classification with Label Representation Learning

  2015cited by this paper
• Zero-Shot Learning via Semantic Similarity Embedding

  2015influential reference
• Group Membership Prediction

  2015cited by this paper
• Predicting Deep Zero-Shot Convolutional Neural Networks Using Textual Descriptions

  2015cited by this paper
• Max-Margin Zero-Shot Learning for Multi-class Classification

  2015cited by this paper
• Efficient Activity Retrieval through Semantic Graph Queries

  2015cited by this paper
• Unsupervised Domain Adaptation for Zero-Shot Learning

  2015cited by this paper
• An embarrassingly simple approach to zero-shot learning

  2015cited by this paper
• Zero-shot recognition with unreliable attributes

  2014cited by this paper
• Transductive Multi-view Embedding for Zero-Shot Recognition and Annotation

  2014cited by this paper
• PRISM: Person Reidentification via Structured Matching

  2014cited by this paper
• A Novel Visual Word Co-occurrence Model for Person Re-identification

  2014cited by this paper
• COSTA: Co-Occurrence Statistics for Zero-Shot Classification

  2014cited by this paper
• Evaluation of output embeddings for fine-grained image classification

  2014influential reference
• Very Deep Convolutional Networks for Large-Scale Image Recognition

  2014cited by this paper
• The SUN Attribute Database: Beyond Categories for Deeper Scene Understanding

  2014influential reference
• Zero-Shot Event Detection Using Multi-modal Fusion of Weakly Supervised Concepts

  2014cited by this paper
• Zero-Shot Learning Through Cross-Modal Transfer

  2013cited by this paper
• DeViSE: A Deep Visual-Semantic Embedding Model

  2013cited by this paper
• Label-Embedding for Attribute-Based Classification

  2013influential reference
• A Unified Probabilistic Approach Modeling Relationships between Attributes and Objects

  2013cited by this paper
• Designing Category-Level Attributes for Discriminative Visual Recognition

  2013cited by this paper
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition

  2013cited by this paper
• Proximal Algorithms

  2013cited by this paper
• Zero-Shot Learning by Convex Combination of Semantic Embeddings

  2013cited by this paper
• Transfer Learning in a Transductive Setting

  2013cited by this paper
• Efficient max-margin multi-label classification with applications to zero-shot learning

  2012cited by this paper
• Metric Learning for Large Scale Image Classification: Generalizing to New Classes at Near-Zero Cost

  2012cited by this paper
• ImageNet classification with deep convolutional neural networks

  2012cited by this paper
• A joint learning framework for attribute models and object descriptions

  2011cited by this paper
• Interactively building a discriminative vocabulary of nameable attributes

  2011cited by this paper
• The Caltech-UCSD Birds-200-2011 Dataset

  2011influential reference
• Evaluating knowledge transfer and zero-shot learning in a large-scale setting

  2011cited by this paper
• Attribute-Based Transfer Learning for Object Categorization with Zero/One Training Example

  2010cited by this paper
• Automatic Attribute Discovery and Characterization from Noisy Web Data

  2010cited by this paper
• Learning image similarities via Probabilistic Feature Matching

  2010cited by this paper
• Zero-shot Learning with Semantic Output Codes

  2009cited by this paper
• Describing objects by their attributes

  2009cited by this paper
• Learning Multiple Layers of Features from Tiny Images

  2009cited by this paper
• Efficient projections onto the l1-ball for learning in high dimensions

  2008cited by this paper
• Visualizing Data using t-SNE

  2008cited by this paper
• Wiley Series in Telecommunications and Signal Processing

  2006cited by this paper
• Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing)

  2006cited by this paper
• Elements of Information Theory

  2005cited by this paper
• Pattern classification and scene analysis

  1974cited by this paper
• PRISM: Person Re-Identification via Structured Matching

  year unknowncited by this paper
• Attribute-Based Classification for Zero-Shot Visual Object Categorization

  year unknowncited by this paper

• Feature Space Sharing via Visual Projector for Open-Vocabulary Multi-Label Classification

  2026cites this paper
• Interpretable Novel Target Discovery through Open-Set Domain Adaptation

  2025cites this paper
• DPSeg: Dual-Prompt Cost Volume Learning for Open-Vocabulary Semantic Segmentation

  2025cites this paper
• Deep ensemble learning method for zero-shot object detection

  2025cites this paper
• HZSCM: Hyperspectral Image Zero-Shot Classification via Vision-Language Models

  2025cites this paper
• ZS-CEBE: leveraging zero-shot cross and bi-encoder architecture for cold-start news recommendation

  2024cites this paper
• Low-rank Multi-modal Features Fusion Semantic Autoencoder For Zero-shot Learning

  2024cites this paper
• Zero-shot learning via categorization-relevant disentanglement and discriminative samples synthesis

  2024cites this paper
• Global-Mapping-Consistency-Constrained Visual-Semantic Embedding for Interpreting Autonomous Perception Models

  2024influential citation
• Prototype-Augmented Self-Supervised Generative Network for Generalized Zero-Shot Learning

  2024cites this paper
• Improving Generalized Zero-Shot Learning by Exploring the Diverse Semantics from External Class Names

  2024cites this paper
• DAAL: Density-Aware Adaptive Line Margin Loss for Multi-Modal Deep Metric Learning

  2024cites this paper
• Application of CLIP for efficient zero-shot learning

  2024cites this paper
• RetVec: Resilient and Efficient Text Vectorizer

  2023cites this paper
• Compound Projection Learning for Bridging Seen and Unseen Objects

  2023cites this paper
• Direct side information learning for zero-shot regression

  2023cites this paper
• Zero-Shot Object Detection by Semantics-Aware DETR with Adaptive Contrastive Loss

  2023cites this paper
• Frustratingly Simple but Effective Zero-shot Detection and Segmentation: Analysis and a Strong Baseline

  2023cites this paper
• Semantics Guided Contrastive Learning of Transformers for Zero-shot Temporal Activity Detection

  2023cites this paper
• Primitive Generation and Semantic-Related Alignment for Universal Zero-Shot Segmentation

  2023cites this paper
• (ML)2P-Encoder: On Exploration of Channel-Class Correlation for Multi-Label Zero-Shot Learning

  2023cites this paper
• Prototype Adaption and Projection for Few- and Zero-Shot 3D Point Cloud Semantic Segmentation

  2023cites this paper
• Semi-transductive Learning for Generalized Zero-Shot Sketch-Based Image Retrieval

  2023cites this paper
• Zero-Shot Learning with Noisy Labels

  2023cites this paper
• Generalized Zero-Shot Activity Recognition with Embedding-Based Method

  2023cites this paper
• GBE-MLZSL: A Group Bi-Enhancement Framework for Multi-Label Zero-Shot Learning

  2023cites this paper
• Instance Adaptive Prototypical Contrastive Embedding for Generalized Zero Shot Learning

  2023cites this paper
• Towards Zero-Shot Learning: A Brief Review and an Attention-Based Embedding Network

  2023cites this paper
• Graph Knows Unknowns: Reformulate Zero-Shot Learning as Sample-Level Graph Recognition

  2023cites this paper
• WAD-CMSN: Wasserstein Distance based Cross-Modal Semantic Network for Zero-Shot Sketch-Based Image Retrieval

  2022cites this paper
• Zero-Shot Sketch Based Image Retrieval Using Graph Transformer

  2022cites this paper
• A Gating Model for Bias Calibration in Generalized Zero-Shot Learning

  2022cites this paper
• Robust image features for classification and zero-shot tasks by merging visual and semantic attributes

  2022influential citation
• Visual feature synthesis with semantic reconstructor for traditional and generalized zero‐shot object classification

  2022cites this paper
• Towards Unbiased Multi-Label Zero-Shot Learning With Pyramid and Semantic Attention

  2022cites this paper
• Generative Mixup Networks for Zero-Shot Learning

  2022cites this paper
• A Survey on Visual Transfer Learning using Knowledge Graphs

  2022cites this paper
• Meta-DZSL: a meta-dictionary learning based approach to zero-shot recognition

  2022cites this paper
• Entity and relation collaborative extraction approach based on multi-head attention and gated mechanism

  2022cites this paper
• Attribute self-representation steered by exclusive lasso for zero-shot learning

  2022cites this paper
• Generative Adversarial Zero-Shot Learning for Cold-Start News Recommendation

  2022cites this paper
• CoHOZ: Contrastive Multimodal Prompt Tuning for Hierarchical Open-set Zero-shot Recognition

  2022cites this paper
• GNDAN: Graph Navigated Dual Attention Network for Zero-Shot Learning

  2022cites this paper
• Learning Multipart Attention Neural Network for Zero-Shot Classification

  2022cites this paper
• Denoising Multi-Similarity Formulation: A Self-paced Curriculum-Driven Approach for Robust Metric Learning

  2022cites this paper
• Semantic guided knowledge graph for large-scale zero-shot learning

  2022cites this paper
• Hyperspectral Imaging Zero-Shot Learning for Remote Marine Litter Detection and Classification

  2022cites this paper
• Learning exclusive discriminative semantic information for zero-shot learning

  2022cites this paper
• Zero-Shot Attribute Attacks on Fine-Grained Recognition Models

  2022cites this paper
• Attention-Based Dynamic Subspace Learners for Medical Image Analysis

  2022cites this paper
• Target inductive methods for zero-shot regression

  2022cites this paper
• Crossmodal Representation Learning for Zero-shot Action Recognition

  2022cites this paper
• Dual Projective Zero-Shot Learning Using Text Descriptions

  2022cites this paper
• MNER-QG: An End-to-End MRC framework for Multimodal Named Entity Recognition with Query Grounding

  2022cites this paper
• Decoupling Zero-Shot Semantic Segmentation

  2021influential citation
• A zero-shot learning framework via cluster-prototype matching

  2021cites this paper
• Search and Explore Strategies for Interactive Analysis of Real-Life Image Collections with Unknown and Unique Categories

  2021cites this paper
• Visual Language Based Succinct Zero-Shot Object Detection

  2021cites this paper
• Relationship-Preserving Knowledge Distillation for Zero-Shot Sketch Based Image Retrieval

  2021cites this paper
• Zero-Shot Object Detection With Transformers

  2021cites this paper
• ZSpeedL - Evaluating the Performance of Zero-Shot Learning Methods using Low-Power Devices

  2021cites this paper
• Interaction Compass: Multi-Label Zero-Shot Learning of Human-Object Interactions via Spatial Relations

  2021influential citation
• Double Discriminative Graph Regularized Semantic Auto-Encoder for Zero-shot Learning

  2021cites this paper
• Scaling Human-Object Interaction Recognition in the Video through Zero-Shot Learning

  2021cites this paper
• Universal Cross-Domain Retrieval: Generalizing Across Classes and Domains

  2021cites this paper
• Semantic-Guided High-Order Region Attention Embedding for Zero-Shot Learning

  2021cites this paper
• Visual Structure Constraint for Transductive Zero-Shot Learning in the Wild

  2021influential citation
• Disentangling Semantic-to-Visual Confusion for Zero-Shot Learning

  2021cites this paper
• Zero-Shot Day-Night Domain Adaptation with a Physics Prior

  2021cites this paper
• An Improved Embedding Model for Zero-Shot Classification based on Attention Mechanism

  2021cites this paper
• Rethinking Deep Contrastive Learning with Embedding Memory

  2021cites this paper
• VMAN: A Virtual Mainstay Alignment Network for Transductive Zero-Shot Learning

  2021cites this paper
• SDM-Net: A simple and effective model for generalized zero-shot learning

  2021cites this paper
• Fairer Evaluation of Zero Shot Action Recognition in Videos

  2021cites this paper
• An Integrated Attribute Guided Dense Attention Model for Fine-Grained Generalized Zero-Shot Learning

  2021cites this paper
• Zero-Shot Instance Segmentation

  2021cites this paper
• A label-oriented loss function for learning sentence representations

  2021cites this paper
• StyleGuide: Zero-Shot Sketch-Based Image Retrieval Using Style-Guided Image Generation

  2021cites this paper
• Zero-Shot Recognition via Optimal Transport

  2021cites this paper
• HSVA: Hierarchical Semantic-Visual Adaptation for Zero-Shot Learning

  2021influential citation
• Generative Replay-based Continual Zero-Shot Learning

  2021cites this paper
• Semi-Supervised Low-Rank Semantics Grouping for Zero-Shot Learning

  2021cites this paper
• DFS: A Diverse Feature Synthesis Model for Generalized Zero-Shot Learning

  2021cites this paper
• Incrementally Zero-Shot Detection by an Extreme Value Analyzer

  2021cites this paper
• Proxy Synthesis: Learning with Synthetic Classes for Deep Metric Learning

  2021cites this paper
• Embedding Transfer with Label Relaxation for Improved Metric Learning

  2021cites this paper
• Learning Deep Cross-Modal Embedding Networks for Zero-Shot Remote Sensing Image Scene Classification

  2021cites this paper
• Holistically Associated Transductive Zero-Shot Learning

  2021cites this paper
• MultiMedia Modeling: 27th International Conference, MMM 2021, Prague, Czech Republic, June 22–24, 2021, Proceedings, Part II

  2021cites this paper
• Compositional Fine-Grained Low-Shot Learning

  2021influential citation
• Efficient Data Transmission for Community Detection Algorithm Based on Node Similarity in Opportunistic Social Networks

  2021cites this paper
• Fold2Seq: A Joint Sequence(1D)-Fold(3D) Embedding-based Generative Model for Protein Design

  2021cites this paper
• COCOA: Context-Conditional Adaptation for Recognizing Unseen Classes in Unseen Domains

  2021cites this paper
• FREE: Feature Refinement for Generalized Zero-Shot Learning

  2021cites this paper
• Adding Attention to Subspace Metric Learning

  2020cites this paper
• Fine-Grained Generalized Zero-Shot Learning via Dense Attribute-Based Attention

  2020influential citation
• A Joint Label Space for Generalized Zero-Shot Classification

  2020cites this paper
• Zero-shot learning for action recognition using synthesized features

  2020cites this paper
• Model-Agnostic Metric for Zero-Shot Learning

  2020cites this paper
• Joint discriminative attributes and similarity embeddings modeling for zero-shot recognition

  2020cites this paper