The work “Concepts Worth Having: Refining VLM-Guided Concept Bottleneck Models with Minimal Annotations” with Nicola Debole, Andrea Passerini, Stefano Teso and Emanuele Marconato has been accepted at NeurIPS 2026!
This paper studies how to improve the quality of concepts in concept bottleneck models guided by vision-language models, using only a small amount of human supervision. We introduce VH-CBM, a hybrid approach that uses a Gaussian process in the vision-language model’s embedding space to propagate expert annotations to other data points. We finally show that annotating as little as 1% of the data can yield more accurate concepts than VLM-guided alternatives, while improving concept calibration and supporting active learning.
A final thanks to all the amazing co-authors involved in this project.
See you in Paris 🗼🥐🇫🇷!