The repository contains the evaluation workflows for the BraTS 2023 Challenge and beyond, including:
- BraTS 2023
- BraTS-GoAT 2024
- FeTS 2024
- BraTS 2024
- BraTS-Lighthouse 2025 🆕
Workflows are organized by task type, e.g. the wf-augmentation folder contains
the CWL workflows for the Augmentation task. CWL scripts used by 1+ workflow
are located in the shared folder.
Source code of the metrics computations mentioned in the README are available
in the evaluation folder of this repo, organized into sub-folders by task.
BraTS 2023
Metrics returned and used for ranking will depend on the task:
| Task | Metrics | Ranking |
|---|---|---|
| Segmentations | Lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity | Lesion-wise dice, lesion-wise HD95 |
| Inpainting | Structural similarity index measure (SSIM), peak-signal-to-noise-ratio (PSNR), mean-square-error (MSE) | SSIM, PSNR, MSE |
| Augmentations | Full dice, full HD95, sensitivity, specificity | Dice mean, dice variance, HD95 mean, HD95 variance |
BraTS-GoAT 2024
Metrics returned and used for ranking are:
| Metrics | Ranking |
|---|---|
| Lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity | Lesion-wise dice, lesion-wise HD95 |
FeTS 2024
Metrics returned are: lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity
Note: Code submission evaluations and ranking were handled by the FeTS-AI Task 1 infrastructure.
BraTS 2024
Metrics returned and used for ranking will depend on the task:
| Task | Metrics | Ranking |
|---|---|---|
| Segmentations | Lesion-wise dice, lesions-wise Hausdorff 95% distance (HD95), full dice, full HD95, sensitivity, specificity | Lesion-wise dice, lesion-wise HD95 |
| Inpainting | Structural similarity index measure (SSIM), peak-signal-to-noise-ratio (PSNR), mean-square-error (MSE) | All 3 metrics |
| Augmentations | Full dice, full HD95, sensitivity, specificity | Dice mean, Dice GINI index, HD95 mean, HD95 GINI index |
| Pathology | Matthews correlation coefficient (MCC), F1, sensitivity, specificity | All 4 metrics |
BraTS-Lighthouse 2025
Metrics returned and used for ranking will depend on the task:
| Task | Metrics | Ranking |
|---|---|---|
| Segmentations | Under discussion | Under discussion |
| Inpainting | Structural similarity index measure (SSIM), peak-signal-to-noise-ratio (PSNR), mean-square-error (MSE) | All 3 metrics |
| Pathology | Matthews correlation coefficient (MCC), F1, sensitivity, specificity | All 4 metrics |
BraTS 2023+ evaluation would not be possible without the work of:
- @FelixSteinbauer - inpainting metrics
- @rachitsaluja - lesionwise segmentation metrics
- @sarthakpati - pathology metrics
In addition to:
