Companion repository for the MICCAI 2026 paper "Lost in the Folds: When Cross-Validation Is Not a Deep Ensemble for Uncertainty Estimation".
The code supports public multi-rater medical image segmentation experiments comparing cross-validation (CV) ensembles with deep ensembles (DE). It includes dataset preparation helpers and ensemble uncertainty metrics used for calibration, ambiguity modeling, failure detection, and segmentation quality analysis.
Use Python 3.12 or a compatible recent Python 3 release.
python -m venv .venv
. .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txtRun the test suite with:
pytest -q- Datasets — Multi-rater datasets (GleasonXAI, RIGA, CURVAS, etc.), folder layout, and preparation instructions.
- Ensemble metrics — Task definitions, metrics (ACE, BA-ECE, SPACE, NCC, GED, AURC), and usage of the metrics module.
| Topic | Location |
|---|---|
| Datasets layout (imagesTr, labelsTr, …) | src/data/README.md |
| Datasets preparation | src/data/README.md |
| Metrics module (CLI, API, structure) | src/ensemble_metrics/README.md |
| Metrics & uncertainty tasks | src/ensemble_metrics/README.md |
Datasets must be downloaded from their original providers and prepared locally. This repository does not include medical image data, model checkpoints, predictions, logs, or generated nnU-Net folders. Common local artifact directories such as data/, nnUNet_raw/, nnUNet_preprocessed/, nnUNet_results/, results/, and checkpoints/ are ignored by Git.
If you use this work, please cite:
@inproceedings{kirscher2026lost,
title = {Lost in the Folds: When Cross-Validation Is Not a Deep Ensemble for Uncertainty Estimation},
author = {Kirscher, Tristan and Bujotzek, Markus and Kirchhoff, Yannick and Rokuss, Maximilian and Isensee, Fabian and Kahl, Kim-Celine and Kovacs, Balint and Maier-Hein, Klaus},
booktitle = {29th International Conference on Medical Image Computing and Computer Assisted Intervention},
address = {Strasbourg, France},
year = {2026},
url = {https://hal.science/hal-05616884}
}