SCIENCE@home

Active Learning for Vision-Language Models

Bardia Safaei,Vishal M. Patel

Published 2024 in IEEE Workshop/Winter Conference on Applications of Computer Vision

ABSTRACT

Pre-trained vision-language models (VLMs) like CLIP have demonstrated impressive zero-shot performance on a wide range of downstream computer vision tasks. However, there still exists a considerable performance gap between these models and a supervised deep model trained on a downstream dataset. To bridge this gap, we propose a novel active learning (AL) framework that enhances the zero-shot classification performance of VLMs by selecting only a few informative samples from the unlabeled data for annotation during training. To achieve this, our approach first cali-brates the predicted entropy of VLMs and then utilizes a combination of self-uncertainty and neighbor-aware uncer-tainty to calculate a reliable uncertainty measure for active sample selection. Our extensive experiments show that the proposed approach outperforms existing AL approaches on several image classification datasets, and significantly en-hances the zero-shot performance of VLMs.

PUBLICATION RECORD

CITATION MAP

EXTRACTION MAP

CLAIMS

  • No claims are published for this paper.

CONCEPTS

  • No concepts are published for this paper.

REFERENCES

Showing 1-63 of 63 references · Page 1 of 1

CITED BY

Showing 1-14 of 14 citing papers · Page 1 of 1