What this track is
This track is for anyone who wants to train a surgical-scene segmentation model without a GPU of their own and without access to medical data. Everything runs in the browser. Google Colab provides a free T4 GPU. The dataset and notebooks sit in a public Google Drive folder, and checkpoints are saved to your own Drive.
Every frame is synthetic. OSV AI Engine renders it from anatomy built from open CT studies. The dataset has no frames from real operations and no patient data. That is also why scores on it say nothing about performance on real laparoscopic video. The project measures that separately, on real frames, outside Colab.
RESEARCH USE ONLY — NOT FOR CLINICAL USE. Models trained in these notebooks are not for clinical use.
What you need
- A Google account and a browser.
- A Colab runtime with a GPU: Runtime → Change runtime type → T4 GPU. On a CPU, training is tens of times slower.
- Space on Google Drive for checkpoints. A ResNet-34 run keeps
last.ptandbest.pt, about 0.4 GB. A notebook usually has 3–4 runs.
Start in five minutes
- Open the osv_open folder on Google Drive.
- Click Add shortcut to Drive and choose My Drive, keeping the shortcut’s name. The notebooks load the
osv_learn.pymodule and the dataset archive fromMyDrive/osv_open/. With the shortcut, the archive is not downloaded again in each new session. - Open notebook 01 from the table below and save a copy: File → Save a copy in Drive. Your edits then stay yours.
- Switch to a T4 GPU and run the cells in order. The first cell mounts your Drive (Colab asks for permission) and checks the archive’s SHA-256. It then checks every frame and mask and, once per session, builds a fast local cache.
If the session drops
Free Colab disconnects idle sessions and limits run time. osv_learn is built for that:
- After every epoch, the
last.ptcheckpoint is written to your Drive underMyDrive/osv_experiments/<notebook>/<run>/. The write is atomic, with a SHA-256 file next to it. - Running the same cell again resumes from the last saved epoch. Weights, optimiser and random-generator state are restored.
- A run that already has
results.jsonis not trained again. To retrain, delete the run’s folder or give it a new name. - A checkpoint whose SHA-256 does not match is not loaded, and training starts over.
The osv_synth_v1 dataset
| Field | Value |
|---|---|
| File | osv_synth_v1.tar, 3.8 GB (3,781,273,600 bytes) |
| SHA-256 | a4814fe99eaad9945290a9dd53a58d047b1c6cd16b4ff7bc44f8461973b34223 |
| Frame | 854×480 PNG; each mask pixel holds a class number |
| Classes | the 13 CholecSeg8k classes |
| Licence | CC BY-NC-SA 4.0, non-commercial use only |
Splits are made by CT subject. No subject appears in two splits, and osv_learn checks this on every run.
| Split | Frames | CT subjects | Use |
|---|---|---|---|
train |
7,520 | 20 | training |
val |
942 | 2 (s0193, s0733) |
choosing the epoch and comparing settings |
test |
707 | 2 (s0495, s1131) |
one final score for the chosen model |
Classes: 0 background, 1 abdominal wall, 2 liver, 3 GI tract, 4 fat, 5 grasper, 6 connective tissue, 7 blood, 8 cystic duct, 9 L-hook, 10 gallbladder, 11 hepatic vein, 12 liver ligament. Hepatic vein does not occur in this version of the dataset; classes with no pixels are left out of tissue mIoU. The rarest classes present are blood and the cystic duct: together under 1% of pixels.
Compare variants on val. If you pick the best variant by its test score, the test split stops being an independent check. Look at test once, at the end: that is notebook 10.
Ten notebooks
Work through them in order; each builds on the one before. These are the English versions, in the en folder of osv_open; the Russian ones sit in the folder itself.
| # | Notebook | What it teaches |
|---|---|---|
| 01 | Quick start: U-Net | Semantic segmentation, the 13 classes, a baseline U-Net on ResNet-18, viewing predictions |
| 02 | Loss functions | Cross-entropy, Dice and their sum; why plain CE loses small structures |
| 03 | Augmentations | The none, basic, color and strong presets: smoke, glare, scope rotation; overfitting |
| 04 | Architecture showdown | U-Net, DeepLabV3+, FPN and U-Net++ on the same ResNet-34 encoder; speed and quality |
| 05 | Choosing an encoder | ResNet-18, ResNet-34, EfficientNet-B0; ImageNet pretraining against training from scratch |
| 06 | Learning rates and schedules | Constant LR, StepLR, cosine, OneCycle |
| 07 | Class imbalance | Weighted CE and focal loss for rare structures: the cystic duct and blood |
| 08 | Transfer learning | Freezing the encoder, and separate learning rates for encoder and decoder |
| 09 | Hyperparameter search (Optuna) | Bayesian search over LR, weight decay, augmentation and loss, scored on val |
| 10 | Ensembles and TTA | Test-time augmentation, averaging models and the final score on test |
Share your result
Run ol.submit_summary("run_name"). It prints a short summary: model and settings, seed, GPU, training time, and tissue mIoU on val. If you reached notebook 10, add its single test score by hand. Post it in the repository Discussions with the notebook number and what you changed.
A negative result is still a result: “focal loss did not help” is more useful than no record at all. Attach nothing but synthetic frames and your own metrics.
All links
- osv_open folder: dataset, module and the Russian notebooks; the English notebooks are in its
enfolder. - osv_synth_v1.tar archive: 3.8 GB.
- osv_learn.py module: the notebooks’ shared code for download, data checks, training and plots.
- How contributing works: DCO, community norms, your first pull request.