The submission contract

To be scored by the leaderboard, a Hugging Face model repo exposes one file at its root, predictor.py, defining a top-level load() function:

def load():
    """Called once by the eval worker. Return an object with .predict(image)."""
    ...
    return predictor

load() must return an object with a .predict(image) method. That’s the entire contract — the eval worker’s inference step is, in full:

predictor = load()
prediction = predictor.predict(image)

It’s duck-typed

The object load() returns does not need to subclass anything. Nothing in the worker’s call path does an isinstance check — it calls .predict(image) on whatever load() gives it. A plain class with a predict method is a complete, valid submission:

class MyPredictor:
    def predict(self, image):
        ...
        return labels

medotter_bench.inference.Predictor (shown below) is a convenience base class, not a requirement. Subclassing it buys you nothing the worker checks for — it exists so you have somewhere to hang a docstring and a NotImplementedError if you want one, and so the optional BatchedSlicePredictor helper (batching + OOM backoff, see below) has something to build on.

A note on other docs you may run into: the MedOtter-Web repo’s README and CLAUDE.md describe the contract as load() -> Predictor, i.e. as if a Predictor subclass were required. The actual contract enforced by the worker (medotter_bench.adapters.base.load_predictor, which just calls mod.load() and uses whatever comes back) is duck-typed. Treat “an object with .predict” as the source of truth.

The predict signature

def predict(self, image: np.ndarray) -> np.ndarray: ...

• image — a numpy.ndarray shaped (C, *spatial): one channel axis followed by the task’s spatial axes, in whatever order the task’s loader reads them (for the volumetric glioma task this is (C, i, j, k) in nibabel’s native voxel order — the loader does not reorient to canonical (Z, Y, X); your prediction must stay on that same grid, not transpose it).
• Return value — a label array matching image’s spatial shape (no channel axis), with integer class ids in the task’s canonical label space. For the glioma task that’s {0, 1, 2, 3} (background / necrotic core / edema / enhancing); other tasks define their own label_space in eval_spec.yaml — see Bundle schema.

There is no spacing argument here (unlike medotter’s mo.evaluate Predictor protocol, which passes spacing=None as a keyword). The leaderboard’s inference contract is exactly predict(self, image).

Importing the reference base class

If you do want to subclass Predictor (or use the optional BatchedSlicePredictor helper for 2D-slice models), import from the submodule:

from medotter_bench.inference import Predictor, BatchedSlicePredictor

Not from medotter_bench import Predictor — that fails. Predictor and BatchedSlicePredictor live in medotter_bench.inference but are not re-exported in medotter_bench’s top-level __all__ (which only exposes Bundle, the scoring/harness functions, the metrics, and evaluate / resolve_adapter / resolve_task_type). This is deliberate, not an oversight — the submission-side classes are documentation/convenience for people writing a predictor.py, not part of the package’s own evaluation API surface.

# Fails: ImportError — Predictor is not in medotter_bench.__all__
from medotter_bench import Predictor

# Works:
from medotter_bench.inference import Predictor

BatchedSlicePredictor is a subclass of Predictor for models that consume one 2D slice at a time: implement units(image), infer_batch(units), and combine(image, outputs), and it implements predict() for you — reading the worker’s BATCH_SIZE env var and halving the batch in-process on a CUDA OOM before falling back to the worker’s own process-level retry.

A minimal predictor.py

import numpy as np


class MyPredictor:
    def __init__(self, weights_path):
        import torch
        self.model = torch.load(weights_path, map_location="cpu")
        self.model.eval()

    def predict(self, image: np.ndarray) -> np.ndarray:
        import torch
        with torch.no_grad():
            x = torch.from_numpy(image).unsqueeze(0).float()
            logits = self.model(x)
            labels = logits.argmax(dim=1).squeeze(0).numpy()
        return labels.astype(np.uint8)


def load():
    return MyPredictor(weights_path="model.pt")

Next: Packaging covers how to lay this out in your Hugging Face repo, and Submitting covers how to point the leaderboard at it.