M1: synthetic instruments against real labels – hypothesis rejected
Can an instrument segmenter trained only on rendered graspers and L-hooks match the same model trained on real labels? It cannot. On 7 held-out CholecSeg8k videos, the synthetic-only model reaches an IoU of 0.116 and the real-label model 0.772. The 95 % interval of the gap, −75.9 to −53.5 points, lies far beyond the 2-point margin fixed in advance.
- Synthetic, then 10 % real: 0.775, as good as all real labels. Without a control arm, this does not show what the synthetic step adds.
- Test discipline: 3 seeds per arm; each chosen model was run on the test once.
Status: hypothesis rejected. Details, open questions and the next experiments are in Can synthetic instruments replace real labels?
M5: removing surgical smoke – unfinished series
The first trained models restore a clean laparoscopic frame from a smoky one. They were trained on synthetic smoke over CholecT50 frames, with the split by video, and scored against the smoky frame left as it is.
- Residual U-Net: 33.08 dB PSNR against 28.99 for the input (+4.09), and better than the input at every smoke density.
- Pix2Pix: 27.85 dB (−1.14). It damages frames with light smoke.
- SRGAN: did not converge in three attempts.
Status: unfinished. This is one seed on the validation split; the test split has not been touched and no model is released. The negative results – training on crops and the three SRGAN attempts – are described in Removing surgical smoke: the first baselines.
How the log works
The first series, the smoke-removal baselines above, is the reference that later experiments will be compared with. Each experiment will compare an architectural change with a baseline under the same conditions:
- a fixed patient-level data split;
- the same compute budget;
- at least three complete runs with different seeds.
An experiment card links the hypothesis, the change, the data version, every run’s results, the uncertainty and the conclusion.
Negative results count equally
Failed and inconclusive experiments are published next to successful ones. An unfinished series is labelled as such: it cannot be presented as proof that there is no effect.
Demonstrations
A short clip for an experiment appears only if the source material may be shown, the clip carries the run ID, and a person has approved publication. Illustrations on this site are not experiment results.