Evaluate a brain tumor segmentation model with more than one score: report per-region Dice for voxel overlap, HD95 (or another precisely defined surface-distance measure) for boundary error, and sensitivity/specificity or lesion-wise detection measures to expose misses and false positives. Calculate results for each held-out case and each target region before summarizing the cohort. State the label protocol, image spacing, distance units, metric implementation, empty-mask handling, and aggregation rule. These measures support research evaluation; they do not by themselves establish clinical acceptability or suitability for diagnosis.
What each metric tells you
Segmentation scores answer different questions. Dice describes how much predicted and reference volume overlaps; Hausdorff-derived measures describe boundary separation; sensitivity and specificity help reveal missed and excess voxels. No one measure captures all three.
As an Amazon Associate I earn from qualifying purchases.
Dice similarity coefficient: overlap
For binary masks, Dice is twice the volume of the intersection divided by the sum of the predicted and reference volumes. It ranges from no overlap to perfect overlap. Dice is intuitive, but it does not say how far a displaced contour lies from the reference. Different spatial arrangements can produce the same overlap, and a modest voxel difference can noticeably affect the score for a small target. Report the region scored, averaging method, and treatment of empty prediction or reference masks. The 2022 review of brain-tumor imaging and segmentation reporting discusses overlap and surface measures as complementary views.
Hausdorff distance and HD95: boundary separation
The Hausdorff distance is a worst-case surface measure: it considers nearest-point distances in both directions between two boundaries and takes the maximum. A remote false-positive island or one extreme mismatch can therefore dominate it. HD95 uses the 95th percentile of surface distances to reduce the influence of the most extreme tail, but it is not outlier-proof. Implementations may differ in surface extraction, directionality, percentile convention, and use of image spacing.
#1 Best Overall
Report the exact implementation and convention, and use physical units such as millimetres when image geometry permits. Do not compare voxel-distance values with millimetres without accounting for spacing. The historical BRATS benchmark documented how a severe miss could be hidden by a favorable average Dice: one method missed every active-tumor voxel in three volumes, producing Hausdorff values above 50 mm. This is an example from an older benchmark, not a current model comparison. Its authors also reported a 74%–85% Dice inter-rater range for human raters on that benchmark’s tumor-subregion task; that range is specific to that annotation setting, not a universal human-agreement range or a model target. See Menze et al.’s 2015 BRATS benchmark article.
Complementary surface and detection measures
- Average symmetric surface distance (ASSD): summarizes typical bidirectional contour separation and complements HD95’s focus on the high-distance tail.
- Surface Dice or normalized surface distance: reports how much of a surface falls within a specified tolerance. State the tolerance and why it is relevant to the task.
- Boundary F1: summarizes boundary precision and recall under a stated tolerance.
- Sensitivity (recall) and specificity: sensitivity is the fraction of reference-positive voxels recovered; specificity is the fraction of reference-negative voxels correctly rejected. Alongside Dice, they can help characterize under- and over-segmentation.
- Lesion-wise detection: for multifocal tumors or tasks where every lesion matters, report lesion-level counts or precision and recall. Whole-volume overlap can obscure an entirely missed small lesion.
These metrics are not interchangeable. Choose them to answer distinct questions about overlap, contour error, and detection; brain-tumor segmentation literature distinguishes voxel-wise, patient-wise, and instance-wise reporting views (2022 review; 2024 NRG Oncology review).
Rank #2
Define the regions before scoring
Metric values only make sense when the target labels are explicit. For the BraTS 2020 adult glioma task, the benchmark defines enhancing tumor (ET), tumor core (TC), and whole tumor (WT): TC includes ET plus necrotic and non-enhancing core components, while WT includes TC plus peritumoral edema. These are benchmark-specific conventions, not definitions to assume for other tumor types, datasets, treatment stages, or annotation protocols. The BraTS 2020 task definitions describe these regions.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Clear out junk files and repair common Windows errors3Fix the driver behind crashes, sound loss and screen glitchesReport each relevant region separately. A macro-average or challenge-style aggregate may be useful, but define exactly how it is calculated and keep per-region results visible. Otherwise, strong performance on a large region can conceal failure on a smaller subregion. Include case-level distributions—such as median and spread—as well as means when suitable, and show failures or outliers rather than relying on a single cohort average. The sources do not establish one aggregation rule that fits every study.
A reproducible evaluation workflow
- Specify the task and target. Name the tumor population, imaging setting, label definitions and subregions, reference annotation process, and whether the task is semantic whole-volume segmentation or lesion-wise detection.
- Freeze the evaluation protocol. Use held-out cases that were not used to tune thresholds or choose the model. Document preprocessing and postprocessing, image geometry and voxel spacing, label mapping, and how empty masks and missing labels are handled.
- Calculate complementary measures. At minimum, calculate Dice and HD95 per region. Add sensitivity and specificity or lesion-wise measures where missed lesions and false positives matter. Add ASSD or a tolerance-based surface measure if typical contour agreement is important, stating the units and tolerance.
- Summarize cases transparently. State the sample count, case-level distributions, and aggregation method. Show failures and outliers instead of relying only on one cohort mean; the historical BRATS benchmark illustrates that metric choice can affect rankings and that average Dice can obscure failures.
- Compare models on the same cases. Use an identical test cohort and describe uncertainty and statistical comparison. Do not present a tiny score difference as meaningful without an analysis appropriate to the study design and outcome distribution. The cited sources do not prescribe one universal statistical test.
- Add expert review when perceived quality matters. Describe who reviewed the masks, the rubric, whether reviewers were blinded, and how disagreement was handled. In a 2023 RSNA study, five of 180 surveyed deep-learning brain-tumor segmentation articles (2.8%) included clinical-expert evaluation of segmentation quality. In that study’s experiment, interrater agreement was Krippendorff α = 0.34 and the correlation between Dice and mean expert quality rating was Kendall τ = 0.23. These are findings from that study, not estimates for all medical AI research. The authors reported that expert quality ratings varied with ambiguous tumor boundaries and perceptual differences, and that common metrics did not capture clinical perception of quality. See the 2023 RSNA study.
- Record the implementation. Name the software and version, configuration, and relevant settings. The BraTS Evaluation repository describes an official Python package that accepts reference and prediction NIfTI files, provides task configurations, and can produce JSON summaries and CSV reports. Its documentation also describes instance-wise HD95 and normalized surface-distance capabilities. Verify that the exact version and configuration match the task, then state them in the report.
How to compare two or more models
| Question | Useful measures or checks | What to inspect |
|---|---|---|
| Is overlap better? | Per-region Dice | Check whether gains hold across ET, TC, and WT, or are driven by one large region. |
| Are severe contour errors present? | HD95 | Inspect high-distance cases and isolated failures. |
| What is typical contour deviation? | ASSD or a justified tolerance-based surface measure | State the distance units or tolerance and why it fits the task. |
| Are voxels or lesions being missed or added? | Sensitivity, specificity, precision/recall, false-positive burden, and lesion-wise detection where relevant | Look for under-segmentation, over-segmentation, and missed individual lesions. |
| Does performance hold across cases and settings? | Case-level variation, supported subgroup or site analyses, and qualified expert review | Do not infer robustness or clinical quality from a cohort mean alone. |
| Is the comparison reproducible? | Protocol and implementation details | Confirm identical test cases, label definitions, geometry, metric implementation, empty-mask behavior, and aggregation. |
What scores cannot establish
There is no universal clinically acceptable Dice or HD95 threshold established by these sources. Acceptability depends on the target, intended use, reference labels, image resolution, annotation uncertainty, and consequences of error. Overlap and boundary distances are measurements, not clinical judgments; expert review can add context, but reviewers also vary and their rubric and agreement should be reported.
Challenge rules are task-specific and change over time. The BraTS 2019 evaluation page documented Dice and 95% Hausdorff distance for its segmentation challenge, consistent with prior challenge configurations at that time. Treat it as an example of a defined protocol, not evidence that current challenge rules are identical; check the active protocol for the task being submitted.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
