Design
The Introduction covers what MedOtter’s planned SDK is and how its pieces fit together. This page covers why three of its more surprising properties are the way they are: the dependency-light import contract, the two-package split, and the deliberate absence of a submission call.
The dependency-light import contract
import medotter is a contract, not an accident: it must cost only numpy.
From the package’s own docstring:
Importing this package is torch-free AND scoring-stack-free: the catalog
(list_datasets, describe_dataset), the Sample dataclass, and the label
semantics are pure-Python/numpy. Heavy machinery is pulled only on use —
load_dataset pulls torch + the dataloader package, and `medotter.metrics`
lazily imports the scoring stack (medotter_core -> medotter_core.metrics.hd95
-> scipy) on first access, so catalog-only users never pay for scipy.
The reason is the audience for the cheapest operations in the SDK — listing
and filtering the 169-dataset catalog,
reading a DatasetInfo — is not the
same audience that needs a GPU-capable torch install or scipy. A script
that only calls mo.list_datasets(modality="CT") to build a report, or a CI
job that validates catalog metadata, shouldn’t need to resolve, download, and
import the ML stack just to do that. Three lazy boundaries enforce this:
-
mo.load_dataset(...)importstorchand thedataloaderpackage the first time it’s called, not atimport medottertime. -
mo.load_model(...)andmo.train(...)importtorch(and, for SAM2, the model backbone) only when called. -
mo.metrics— the seven pure metrics, re-exported frommedotter_core— is exposed through a module-level__getattr__rather than a normal import, alongside its three lazy siblings (bridges,prompts,selfreport):_LAZY_MODULES = ("metrics", "bridges", "prompts", "selfreport") def __getattr__(name): if name in _LAZY_MODULES: import importlib mod = importlib.import_module(f"medotter.{name}") globals()[name] = mod return mod raise AttributeError(f"module {__name__!r} has no attribute {name!r}")All four are also deliberately left out of
medotter.__all__, even though__dir__reports them. If they were in__all__,from medotter import *and any__all__-walking tool (Sphinx autosummary and similar) would resolve them eagerly on import — silently reintroducing thescipycost the lazy lane exists to avoid. (Onlymetricsandbridgescarry that cost;promptsis numpy-only andselfreportis stdlib-only — they ride the same lane so the surface stays uniform.) Keeping them out of__all__but reachable by attribute access gets both: discoverable, but never resolved unless something actually touches one of them directly. The firstmo.score/mo.evaluatecall pulls in the samescipydependency too — regardless of whichmetrics=you ask for, not onlyhd95— but through a separate, torch-free import of the scoring stack, not through this module.
One nuance worth being precise about: the scipy-free half of the promise is
scoped to import medotter itself, not to medotter_core in isolation.
medotter_core’s own docstring calls it “torch-free,” not “scipy-free” —
its __init__.py imports medotter_core.metrics.hd95 (which needs scipy.ndimage) at
module load time. Once the package is released, a submitter who imports
medotter_core directly will pay the scipy cost immediately; a
researcher who only ever does import medotter and calls the catalog
functions pays for neither scipy nor torch, because that import path
never reaches medotter_core at all.
The two-package split
The development source is split into medotter (the researcher-facing facade:
catalog, data, inference, training, evaluation) and medotter_core (the
narrower benchmark, prompt-free inference, and scoring surface). See
Introduction — The two packages
for the audience-facing summary; the design reason for keeping them separate
packages, rather than one package with two audiences, is what medotter_core
has to run as.
medotter_core is what backs the leaderboard’s GPU evaluation worker on
every submission — including model code the worker has never seen before.
That makes it infrastructure with a narrow, auditable job: load a frozen
bundle, run a prompt-free predictor, score the result. Keeping it torch-free
and free of medotter’s much larger research surface (the dataset registry,
training loops, SAM2 backbones) keeps that infrastructure’s dependency
footprint small and its behavior easy to reason about — a submitter or
reviewer auditing what code runs against untrusted model output only has to
read medotter_core, not all of medotter.
The split also isn’t a fork: medotter.metrics re-exports medotter_core’s
seven pure metrics byte-identically (mo.metrics.hd95 is medotter_core.hd95
is True) rather than wrapping or duplicating them, and the bridge
predictors in mo.bridges re-export medotter_core.bridges the same way.
Those two are the only pieces of medotter_core the facade re-exports — the
engine orchestration names (Bundle, run_submission, evaluate_bundle,
and the rest) are importable only from medotter_core itself. Local evaluation in medotter
and the worker’s evaluation both run the same scoring code — one source of
truth, imported lazily by the facade and directly by the worker.
Why there is no mo.submit
The planned SDK boundary ends at local mo.evaluate(...): scoring a model against a
dataset, on your own machine, with no network calls and nothing left behind.
Submitting a model to the public leaderboard is a different kind of
operation — it validates a Hugging Face repo, POSTs to a Supabase Edge
Function, schedules work on a GPU worker, and persists run/case scores — and
that is stateful, server-owned behavior outside the package surface. See
Submitting — There is no mo.submit
for the current maintenance state and planned public submission paths.
Keeping submission out of medotter keeps the boundary from the two sections
above intact: medotter’s heaviest verb (evaluate) is still a pure local
call, and the code that talks to the leaderboard’s infrastructure is the same
medotter_core-adjacent tooling (the CLI’s local predictor.py preflight,
the GPU worker itself) already described above — not a new surface bolted
onto the facade.
Next: Versioning covers what stability guarantee each of these surfaces actually carries.