Introduction
What MedOtter is
MedOtter is a medical-image segmentation benchmark and leaderboard platform. The public site reports approved evaluations across radiology and imaging tasks, while a Python interface for catalog, inference, training, and local evaluation is being prepared for release.
The planned Python SDK is the programmatic entry point to that platform: browse the dataset catalog, load data as uniform samples, run inference with SAM2 or a trained checkpoint, and score predictions with the same metrics the leaderboard uses.
The two packages
The development source is organized as two planned packages for different audiences:
| Package | Import | Audience | Public status |
|---|---|---|---|
medotter | import medotter as mo | Researchers: dataset catalog, data loading, inference, training, evaluation | Release pending |
medotter_bench | import medotter_bench | Leaderboard submitters and the GPU evaluation worker: torch-free scoring, benchmark bundles, prompt-free inference contracts | Release pending |
The similarly named PyPI distributions are placeholders/name reservations, not supported MedOtter releases. See Installation status for the current public availability.
After public release, researchers will use the planned medotter facade for
catalog, inference, training, and evaluation workflows. The narrower
medotter_bench interface is intended for model packaging and scoring
contracts. Until supported artifacts exist, the examples in this guide are
interface previews; see Installation status.
The dependency-light design
import medotter costs only numpy. The catalog (list_datasets,
describe), the Sample and DatasetInfo data classes, and label semantics
(LabelSchema) are pure Python and numpy — no torch, no scipy, no
network access.
Everything heavier is pulled in lazily, on first use:
mo.load_dataset(...)importstorchand the dataloader package the first time you call it.mo.load_model(...)andmo.train(...)importtorch(and, for SAM2, the model backbone) only when called.- The first call to
mo.score(...)ormo.evaluate(...)— with anymetrics=, not justhd95— imports the scoring stack, which pulls inscipy. Accessingmo.evaldirectly does the same. A catalog-only script that never scores anything never pays for it.
In the development source, this keeps a catalog-only import medotter
lightweight without pulling in the full ML stack. The contract still needs to
be validated against the first supported package artifact before release.
Where to go next
- Reviewing the planned SDK? Start with Installation status, then read the Quick Start as an interface preview.
- Want to go deeper on a specific area — the dataset catalog, data loading, inference, evaluation, training, or 3D volume benchmarks? See the Guides section in the sidebar.
- Submitting a model to the public leaderboard? See the Leaderboard section in the sidebar for the submission contract and packaging steps.