In short
The project builds open computer vision models for surgery: they recognise organs, instruments and the phases of an operation in video. The code is open, the data comes from published surgical datasets, and surgeons and medical students work alongside the engineers.
For you, it is a chance to solve a real problem on real data, to write code that experienced engineers review, and to build an open portfolio you can show any employer or supervisor.
What you get
A real problem instead of an exercise
Not another Titanic or MNIST, but recordings of real operations: smoke, glare, blood, organs hidden behind instruments, long videos and few labelled frames. Here you see why a good validation score guarantees nothing, and how research differs from a tutorial.
Computer vision at full scale
Segmentation of organs and instruments, surgical phase recognition from video, instrument detection and tracking, pre-annotation with Segment Anything 2. Ablations, repeated runs, confidence intervals and fair comparisons, following the rules of scientific publications.
Industry-grade engineering
Tests and CI on every pull request, code review, data versions in DVC, configuration in Hydra, experiments in MLflow, FastAPI services in Docker. The project’s infrastructure is defined as code: Kubernetes, Terraform, Helm. It is not a classroom sandbox; annotators use it.
An open portfolio
Your commits and pull requests live in a public repository under the Apache-2.0 licence. You can link them from your CV: an employer sees real code, discussion and review, not a line saying “knows PyTorch”. A profile on the people page is published with your consent.
Review from experienced engineers
Another contributor reviews every change. Feedback is specific and respectful: why this test is better, where data leaks, how to make an experiment reproducible. We try to answer a newcomer’s first pull request within 48 hours.
A medical domain
You will learn how medical software differs from ordinary software: de-identification, patient-level splits instead of frame-level splits, dataset licences, responsibility for a model’s mistakes. You will discuss problems with surgeons and medical students, a rare kind of cross-disciplinary experience that medical AI companies and research groups value.
Theses, term papers and placements
We have collected more than 50 research topics, more than twenty of them technical: from comparing segmentation architectures to video de-identification and infrastructure for medical AI. Only your university can count work in the project as a placement; on request, the project confirms the amount and content of your work.
Publications and conferences
You can present results at student conferences, young-researcher sessions and workshops on medical computer vision. Authorship of project publications follows the contribution agreement.
What you can work on
- Data. Loaders and converters for open datasets, leak-free patient-level splits, data cards.
- Models. Baseline models for segmentation and phase recognition, ablations, speeding them up to real time.
- Evaluation. Metrics, confidence intervals, an open benchmark and leaderboard.
- Annotation. Integrating pre-annotation models into CVAT, active learning, automatic quality checks on annotation.
- De-identification. Reading text on medical equipment screens, detecting faces and out-of-body frames.
- Services and MLOps. FastAPI, Docker, MLflow, CI.
- Web. The project website in Astro and TypeScript, training tools for medical students.
- Testing and quality. Edge cases, data checks, tests on synthetic data.
- Infrastructure. Kubernetes, Terraform, Helm. This is a production system, so infrastructure work is for contributors who have settled into the project; access is granted separately.
Technologies
Python 3.10+, PyTorch, torchvision, OpenCV, Albumentations, Hydra, MLflow, DVC, FastAPI, Docker, pytest, GitHub Actions. Annotation: CVAT and Segment Anything 2. Website: Astro and TypeScript. Infrastructure: Kubernetes, Terraform, Helm.
You do not need to know all of this in advance; you will pick it up along the way.
How it works
- Pick a task. Start with a good first issue or suggest your own.
- A first contribution in about 30 minutes. Step-by-step instructions are on the Contribute page.
- Review and discussion. An experienced contributor reviews your change and you refine it.
- Bigger tasks. A module, an experiment, a service, together with the person responsible for that area.
- Your own topic. Optionally, a term paper, a thesis or an article from the list of topics or your own idea.
Time: about four hours a week, remotely, whenever suits you. Most first tasks need no GPU: tests, converters and metrics run on an ordinary laptop. How to get compute for training models is agreed in the issue.
Who we are looking for
- Students of software engineering, applied mathematics and computer science, data science and related programmes: undergraduates from the second year, master’s students and PhD students.
- People interested in computer vision, machine learning, MLOps, back end, front end, DevOps or testing.
You need basic Python and git. Machine learning experience helps for model work but is not required for anything else.
What to know first
- No patient data. We work only with published datasets that each person downloads under their licence terms. Never commit medical data to the repository, not even de-identified data.
- Research use only. The project’s models are not intended for diagnosis or treatment.
- DCO sign-off. Every commit is signed off (
git commit -s) to confirm you have the right to contribute the code. - A negative result is a result. An experiment that honestly shows an idea does not work is valuable to the project.
For educators at technical universities
The project suits term papers and theses, semester-long team projects, and study or industrial placements. Take a ready topic from the list or propose your own, including joint topics with medical students. Write to support@opensurgicalvision.org with the subject “Department: technical university” and we will discuss the format.
How to join
Start with a good first issue and introduce yourself in the project discussions. If you prefer email, write to support@opensurgicalvision.org with the subject “Student: engineering”.
Studying medicine? You have your own page.