Backends

A backend is the thing that actually answers. SystemOne owns one and exposes it as agent.backend; which one you get is the config you pass, or SYSTEM_ONE_BACKEND when you pass none.

Backend	Extra	What it does
http (default)	system-one[http]	Calls a hosted provider over HTTP.
onnx	system-one[onnx]	Runs the model in-process. No network, no API key.
stub	none	Answers at random. For wiring tests and demos only.

Vendor imports live inside the backend factory, so the extras stay genuinely optional: installing the core package pulls neither httpx2 nor onnxruntime. Selecting a backend whose extra is missing raises ImportError with the pip command that fixes it.

The protocol

Every backend satisfies the same one-method protocol — Backend, or AsyncBackend for the awaited version:

def ask(self, request: SystemOneInput) -> SystemOneOutput: ...

That is the whole contract. Anything implementing it can be handed to an agent.

HTTP

One class per vendor, each an HTTPConfig with its endpoint filled in:

SystemOne(TypesafeConfig())  # https://api.typesafe.ai/v1/systemone
SystemOne(OpenRouterConfig())  # https://openrouter.ai/api/alpha/decisions
SystemOne(HTTPConfig(base_url="https://my-host", model="mine"))

The request goes to base_url + path. path defaults to /v1/systemone, and you can point it anywhere with path= or SYSTEM_ONE_PATH:

SystemOne(HTTPConfig(base_url="https://my-host", path="/decide", model="mine"))

The presets require SYSTEM_ONE_API_KEY (or api_key=); a plain HTTPConfig sends no Authorization header without one. Every field also reads its SYSTEM_ONE_* variable, so SystemOne() alone needs SYSTEM_ONE_BASE_URL and SYSTEM_ONE_MODEL.

Retries are built in: 408, 429 and 5xx are retried up to SYSTEM_ONE_MAX_RETRIES times with exponential backoff, honouring the Retry-After header when the server sends one. 401 and 403 are never retried — see Errors.

Backend-specific calls stay reachable through agent.backend.

Bringing your own client

The HTTP backends take an optional client= — an httpx2.Client, or an httpx2.AsyncClient for AsyncHTTPBackend. Without one they build their own with timeout=SYSTEM_ONE_TIMEOUT. Pass one to share a connection pool or to control its lifetime; closing it is up to you:

with httpx2.Client(timeout=30) as client:
    agent = SystemOne(using=HTTPBackend(TypesafeConfig(), client=client))
    agent.ask(...)

Fastino GLiDE

GLiDE is Fastino's hosted decision model. It answers the same POST /v1/systemone form, so a plain HTTPConfig is all it takes. Create a key at agent.fastino.ai (Settings → API keys):

import os

from pydantic import SecretStr

agent = SystemOne(
    HTTPConfig(
        base_url="https://api.fastino.ai",
        model="fastino/GLiDE",
        api_key=SecretStr(os.environ["FASTINO_API_KEY"]),
        timeout=300,
    )
)
response = agent.ask(
    "Refund request: 10 days old, 30-day window applies.",
    {"refund_allowed": {"type": "noul", "instructions": "Does this qualify for refund?"}},
)
print(response.nouls["refund_allowed"].noul)

The same thing from the environment, with no config object:

SYSTEM_ONE_BASE_URL=https://api.fastino.ai
SYSTEM_ONE_MODEL=fastino/GLiDE
SYSTEM_ONE_API_KEY=<your Fastino key>
SYSTEM_ONE_TIMEOUT=300

Things to know:

• Auth. The key goes out as Authorization: Bearer …; Fastino accepts that as well as its native X-API-Key header.
• Cold starts. An idle or newly deployed model can take a while to answer. Fastino recommends a read timeout of at least 300 seconds, hence timeout=300. A 425 (deployment warming) is not in the SDK's retry set — catch APIError, wait about 60 seconds and ask again. 429 and 5xx are retried as usual.
• Limits. Each rendered prompt (state plus one question) must fit in 40,000 tokens, and a choice or score takes at most 255 options or levels. No streaming, and one state per request.
• Thresholds. Probabilities are calibrated per model, not per business — tune the cut-off on representative data instead of assuming 0.5.
• Not covered. POST /v1/chat/completions (GLiNER extraction, decoder LLMs) is a different contract and is not part of this SDK.

A local server that speaks the form

HTTPConfig is not tied to a hosted vendor: any server that answers a POST in the same request and answer shape works, including one on the machine itself. It can live on any route; set path to match. coreai-kit runs a catalog model on a Mac (Apple's Core AI runtime, macOS 27) behind the default /v1/systemone, so it needs no path:

cd Examples/Decide && swift run -c release decide-cli serve   # http://127.0.0.1:8090/v1/systemone

SystemOne(HTTPConfig(base_url="http://127.0.0.1:8090", model="minicpm5-2b"))

No key, nothing leaves the machine. One difference from the hosted API to know about: a choice there lists at most 16 options, and a longer list comes back as a 422 with the reason.

kev

kev is a family of small decision models you train and serve yourself, a drop-in for hosted jev. Its server answers on the default /v1/systemone, so it needs no path:

git clone https://github.com/jaredpalmer/kev.git && cd kev
uv sync --extra serve
uv run --extra serve python -m kev.serve --run jaredpalmer/kev-4b --port 8009

SystemOne(HTTPConfig(base_url="http://127.0.0.1:8009", model="jaredpalmer/kev-4b"))

If the server was started with KEV_API_KEY, pass the same value as api_key (or SYSTEM_ONE_API_KEY); it is sent as a bearer token. For a deployed instance (e.g. Modal), point base_url at its URL.

ONNX

SystemOne(ONNXConfig(onnx_dir="onnx", model="laya"))  # both are the defaults

or SYSTEM_ONE_BACKEND=onnx with SYSTEM_ONE_ONNX_DIR / SYSTEM_ONE_MODEL.

The directory must hold three things for the configured model:

onnx/
├── laya.onnx            # the exported graph
├── laya.json            # model config: prompt layout, temperature buckets
└── tokenizer/
    └── tokenizer.json

Sessions are cached per (directory, model) pair, so a second agent — or a second model on the same agent — does not reload gigabytes of weights. A missing graph raises SystemOneError naming the path it looked for.

Those artifacts are produced once from a laya checkpoint — see Running a local model.

The async ONNX backend runs the same synchronous session off the event loop, so await agent.ask(...) does not block it.

Stub

SystemOne(StubConfig(seed=1))  # `seed` is optional; set it for reproducible output

or SYSTEM_ONE_BACKEND=stub. Every question gets a uniformly random answer of the right shape — a noul probability, a choice drawn from its labels with a distribution, an expected score with its legend — and zero usage. Nothing about the state is read, so never use it for decisions.

Injecting your own

Pass any object satisfying the protocol as using=. Tests use this to avoid the network entirely:

class FakeBackend:
    def ask(self, request):
        return SystemOneOutput(
            model=request.model,
            usage=Usage(input_tokens=0, output_tokens=0),
            answers={"urgent": NoulAnswer(noul=0.9)},
        )


agent = SystemOne(using=FakeBackend())

An injected backend owns its model: ask falls back to backend.model, not SYSTEM_ONE_MODEL.