GroVE

Code for "Probabilistic Embeddings for Frozen Vision-Language Models: Uncertainty Quantification with Gaussian Process Latent Variable Models", UAI 2025.

Dataset

The code supports the MS-COCO, Flickr30k, CUB-200-2011 and Oxford Flowers 102.

COCO: Download the 2014 data containing images and captions and setup the directory in the following way

coco
|-images/
|--train2014 
|--val2014 
|-captions_train2014.json 
|-captions_val2014.json

Flickr30k: Download the images and captions. CUB-200-2011: Download the CUB-200-2011 images and the captions Oxford Flowers 102: Download the images and captions

Training and Evaluation

Update the config.py script with the dataset path and the hyperparameter values. Run the training using train.py. For evaluation, use evaluation.py.

If you use this code, please cite:

@article{venkataramanan2025probabilistic,
  title={Probabilistic Embeddings for Frozen Vision-Language Models: Uncertainty Quantification with Gaussian Process Latent Variable Models},
  author={Venkataramanan, Aishwarya and Bodesheim, Paul and Denzler, Joachim},
  journal={arXiv preprint arXiv:2505.05163},
  year={2025}
}