LostInFolds

Companion repository for the MICCAI 2026 paper "Lost in the Folds: When Cross-Validation Is Not a Deep Ensemble for Uncertainty Estimation".

The code supports public multi-rater medical image segmentation experiments comparing cross-validation (CV) ensembles with deep ensembles (DE). It includes dataset preparation helpers and ensemble uncertainty metrics used for calibration, ambiguity modeling, failure detection, and segmentation quality analysis.

Installation

Use Python 3.12 or a compatible recent Python 3 release.

python -m venv .venv
. .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

Run the test suite with:

pytest -q

Documentation

• Datasets — Multi-rater datasets (GleasonXAI, RIGA, CURVAS, etc.), folder layout, and preparation instructions.
• Ensemble metrics — Task definitions, metrics (ACE, BA-ECE, SPACE, NCC, GED, AURC), and usage of the metrics module.

Quick links

Topic	Location
Datasets layout (imagesTr, labelsTr, …)	src/data/README.md
Datasets preparation	src/data/README.md
Metrics module (CLI, API, structure)	src/ensemble_metrics/README.md
Metrics & uncertainty tasks	src/ensemble_metrics/README.md

Data and Model Artifacts

Datasets must be downloaded from their original providers and prepared locally. This repository does not include medical image data, model checkpoints, predictions, logs, or generated nnU-Net folders. Common local artifact directories such as data/, nnUNet_raw/, nnUNet_preprocessed/, nnUNet_results/, results/, and checkpoints/ are ignored by Git.

Citation

If you use this work, please cite:

@inproceedings{kirscher2026lost,
  title = {Lost in the Folds: When Cross-Validation Is Not a Deep Ensemble for Uncertainty Estimation},
  author = {Kirscher, Tristan and Bujotzek, Markus and Kirchhoff, Yannick and Rokuss, Maximilian and Isensee, Fabian and Kahl, Kim-Celine and Kovacs, Balint and Maier-Hein, Klaus},
  booktitle = {29th International Conference on Medical Image Computing and Computer Assisted Intervention},
  address = {Strasbourg, France},
  year = {2026},
  url = {https://hal.science/hal-05616884}
}