What the benchmark is for
The benchmark lets models be compared fairly and reproducibly. It is an engineering comparison, not a clinical evaluation. There are no results yet.
Tasks: instrument and anatomy segmentation, surgical phase recognition, 3D segmentation and smoke removal. Each task has its own metrics, and we do not collapse them into a single “overall score”.
The same conditions for everyone
- Patient-level splits. Frames from one operation can never be in both training and evaluation.
- Equal budget. Epochs, GPU hours and batch size are comparable; unavoidable differences are stated.
- At least three runs per configuration: individual values, the mean and a confidence interval.
- Ablations for any architectural change, on the same split and budget.
Validation and the held-out test
Development uses the validation set only. The held-out test is used at most once per release, and every use shows on a counter. Before that, another person must reproduce the result. Experiment automation has no access to the held-out test.
How to submit a result
Submissions will open later. A submission will need:
- the task, code version, configuration and data version;
- complete run records and compute conditions;
- ablations and known limitations;
- licenses and the origin of materials.
Results will appear on the leaderboard and in the experiment log.