Packaging your model

The leaderboard only ever downloads one file from your Hugging Face repo: predictor.py, fetched unauthenticated from the repo root on its default (main) branch:

https://huggingface.co/<owner>/<model>/resolve/main/predictor.py

That single constraint shapes the whole packaging story.

Repo layout

owner/model-name/            (Hugging Face model repo, public)
├── predictor.py             ← required, at the repo root
├── model.pt                 ← your weights, any format/name you like
└── (anything else your load() needs, e.g. a config file)

• predictor.py must be at the repo root. It is fetched by a fixed .../resolve/main/predictor.py URL — a nested path (src/predictor.py, models/predictor.py) will not be found.
• Everything else is up to you. The platform never downloads your weights directly. load() is responsible for getting them — e.g. by reading a sibling file if load() is running with your repo checked out, or by calling huggingface_hub.hf_hub_download(repo_id=..., filename=...) to pull a weight file from your own repo at runtime. (This is the common pattern, not a documented requirement — the only hard requirement is that load() ends up with a working predictor, however it gets there.)
• The repo must be public. The pre-submission check fetches predictor.py with no auth and fails with “looks private/gated — make the repo public” if it can’t. The eval worker’s own download (evaluation/eval_worker.py’s _download_predictor, in the MedOtter-Web repo) uses huggingface_hub.hf_hub_download(repo_id=hf_model_id, filename="predictor.py") — the standard unauthenticated path for a public repo, run in the worker’s trusted parent process before the file is ever executed.

predictor.py is executed, not just parsed

There are two different checks on predictor.py, at two different times, and it matters which one you’re thinking about:

1. Local pre-flight (CLI submit only). Before submitting, scripts/submit_model.py downloads your predictor.py and runs it through Python’s ast module — parsing it into a syntax tree to confirm it defines a top-level load (a def load, async def load, a module-level assignment load = ..., or a re-export like from .pkg import load). This is static analysis only: your file is never imported or executed during this check.
2. Real execution (the eval worker). When your submission actually runs, the worker’s loader (medotter_bench.adapters.base.load_predictor) does importlib.util.spec_from_file_location(...) and spec.loader.exec_module(mod) on your downloaded predictor.py, then calls mod.load(). This does execute every line of module-level code in your file, before load() is ever called.

The practical consequence: keep predictor.py importable and side-effect-free at module scope.

• Importable — the file must be valid Python with no syntax errors (the static check will already tell you this) and must not raise on import (the static check will not tell you this — a NameError or a bad top-level import only surfaces when the worker actually executes it).
• Side-effect-free at module scope — don’t load weights, open network connections, or do heavy work at the top level of the file. Put that inside load() (or inside your predictor’s __init__, called from load()). Module-scope code runs once, immediately, as soon as the worker imports the file — before it has any chance to catch or report an error cleanly, and before your load() function even gets a chance to run.

# Bad — model construction at import time; a failure here has nowhere to be
# reported cleanly, and it runs even if something downstream never calls load().
import torch
MODEL = torch.load("model.pt")

def load():
    return MyPredictor(MODEL)


# Good — nothing but definitions at module scope; all the work happens
# inside load(), where a failure is attributable to "this submission's load()".
def load():
    import torch
    model = torch.load("model.pt")
    return MyPredictor(model)

Optional platform hooks

Two helpers in medotter_bench.inference let your predictor cooperate with the worker’s batching and memory limits, if you want that — using them is optional; a predictor that ignores both still works correctly, it just can’t be batch-tuned or self-recover from an out-of-memory error:

• batch_size_from_env(default) — returns the worker’s BATCH_SIZE env var (set per-GPU, and halved on an OOM retry) if present and positive, else your default. Route your model’s batch size through this so the platform can tune it per card.
• oom_safe_map(items, batch_size, fn, *, min_batch=1, on_oom=None) — runs fn over items in chunks of batch_size, halving the batch size and clearing the CUDA cache in-process on a caught CUDA out-of-memory error, down to min_batch, before re-raising (letting the worker’s own process-level retry be the final backstop).

BatchedSlicePredictor (see Contract) already wires both of these in for 2D-slice models — you only need them directly if your model has a different structure (whole-volume, 3D patches) and you’re writing predict() yourself.

Next: Submitting covers how to point the leaderboard at your packaged repo.