Leaderboard

Packaging your model

Public package status: the Python distributions and desktop installers are not released yet. These pages document an evolving interface, not currently usable install artifacts. View release status.

The leaderboard only ever downloads one file from your Hugging Face repo: predictor.py, fetched unauthenticated from the repo root on its default (main) branch:

https://huggingface.co/<owner>/<model>/resolve/main/predictor.py

That single constraint shapes the whole packaging story.

Repo layout

owner/model-name/            (Hugging Face model repo, public)
├── predictor.py             ← required, at the repo root
├── model.pt                 ← your weights, any format/name you like
└── (anything else your load() needs, e.g. a config file)

predictor.py is executed, not just parsed

Two different things happen to predictor.py, at two different times, and it matters which one you’re thinking about:

  1. Submit time: fetched and pinned, never parsed. Nothing inspects your code before it runs. The submit CLI (medotter/cli.py, run as python -m medotter submit) does no static analysis of predictor.py at all — the old AST pre-flight (a scripts/submit_model.py that parsed the file to confirm a top-level load) no longer exists. At submit time the file is only fetched: the server’s submit function downloads it unauthenticated from .../resolve/<head>/predictor.py, rejects the submission if it’s missing (which is also what a private or gated repo looks like) or over 1 MiB, and pins its sha256. The CLI itself touches the file just once, computing that same sha256 (medotter/cli.py’s resolve_predictor_sha) when it attaches a self-report. Whether the file defines load — or even parses — is not checked until the worker executes it.
  2. Real execution (the eval worker). When your submission actually runs, the worker’s loader (medotter_core.adapters.base.load_predictor) does importlib.util.spec_from_file_location(...) and spec.loader.exec_module(mod) on your downloaded predictor.py, then calls mod.load(). This does execute every line of module-level code in your file, before load() is ever called.

The practical consequence: keep predictor.py importable and side-effect-free at module scope.

# Bad — model construction at import time; a failure here has nowhere to be
# reported cleanly, and it runs even if something downstream never calls load().
import torch
MODEL = torch.load("model.pt")

def load():
    return MyPredictor(MODEL)


# Good — nothing but definitions at module scope; all the work happens
# inside load(), where a failure is attributable to "this submission's load()".
def load():
    import torch
    model = torch.load("model.pt")
    return MyPredictor(model)

Optional platform hooks

Two helpers in medotter_core.inference let your predictor cooperate with the worker’s batching and memory limits, if you want that — using them is optional; a predictor that ignores both still works correctly, it just can’t be batch-tuned or self-recover from an out-of-memory error:

BatchedSlicePredictor (see Contract) already wires both of these in for 2D-slice models — you only need them directly if your model has a different structure (whole-volume, 3D patches) and you’re writing predict() yourself.

Next: Submitting covers how to point the leaderboard at your packaged repo.