Backends
LlmBackend
Abstract execution contract for an LLM backend.
Usage
LlmBackend()Concrete backends map abstract model sizes to provider-specific model names and encapsulate how to execute a RunSpec and parse the raw response. The portable decisions — argv, output resolution, schema strictification — run in the shared wasm core; a backend supplies only its provider and the I/O.
Attributes
models: dict[TModel, str]-
Mapping from abstract model size to the provider’s model name.
provider: ProviderName-
Provider identifier keying a RunSpec’s
provider_configs. schema_dialect: str | None-
Strict-schema dialect the core applies to a
response_model("anthropic","openai", orNoneto emit the plain JSON schema). auto_select_tiers: frozenset[TModel] | None-
Model tiers this backend may be auto-selected for;
None(the default) leaves it eligible for every tier, and an explicitbackend=reaches it regardless.
Methods
| Name | Description |
|---|---|
| accounting() |
Return the (cost_usd, usage) an attempt’s output reports, or (None, None) when it carries neither.
|
| aexecute() | Execute a single run asynchronously and resolve it to a Response. |
| check_status() | Check whether this backend is installed and authenticated. |
| core_plan() | Ask the core to plan this run’s invocation (an exec argv or an HTTP request). |
| env() | Return extra environment variables for the invocation, merged over the inherited environment. |
| execute() | Execute a single run synchronously and resolve it to a Response. |
| is_authenticated() | Probe whether the backend holds valid credentials for its provider. |
| openai_section() |
Return the openai_endpoint wire section; None for every backend but the endpoint backend.
|
| resolve_model() | Resolve an abstract tier to this backend’s provider model id, passing a concrete id through unchanged. |
| schema_for() | Serialize a Pydantic model into the JSON-schema string this backend expects. |
| to_response() |
Resolve a raw capture into a structured Response via the core resolve op.
|
| wire_schema() |
Return the portable schema object for spec, from a response_model or a raw schema.
|
| wire_spec() |
Serialize a RunSpec into the portable wire spec the core plan/resolve ops consume.
|
accounting()
Return the (cost_usd, usage) an attempt’s output reports, or (None, None) when it carries neither.
Usage
accounting(raw)The retry loop calls this on each transient failure it discards, so a caller reconciling spend can still see the cost. The default parses nothing; the CLI family reads the core’s resolve accounting.
Parameters
raw: str- The raw output read wherever the provider wrote it.
Returns
tuple[float | None, dict[str, object] | None]-
A
(cost_usd, usage)pair, eachNonewhen the output does not carry it.
aexecute()
Execute a single run asynchronously and resolve it to a Response.
Usage
aexecute(spec)The backend runs the process, reads its output wherever the provider writes it, detects failure, and validates against spec.response_model.
Parameters
spec: RunSpec- The configured run to execute.
Returns
check_status()
Check whether this backend is installed and authenticated.
Usage
check_status(*, timeout=10)Parameters
timeout: int = 10- Seconds to wait for the authentication probe.
Returns
BackendStatus-
BackendReady when authenticated, BackendNotInstalled when the
backend is not available, else BackendNotAuthenticated.
core_plan()
Ask the core to plan this run’s invocation (an exec argv or an HTTP request).
Usage
core_plan(spec)env()
Return extra environment variables for the invocation, merged over the inherited environment.
Usage
env(spec)Parameters
spec: RunSpec-
The configured run, so a backend can scope env overrides to
spec.isolated(e.g. a fresh config home only when isolating).
execute()
Execute a single run synchronously and resolve it to a Response.
Usage
execute(spec)Parameters
spec: RunSpec- The configured run to execute.
Returns
is_authenticated()
Probe whether the backend holds valid credentials for its provider.
Usage
is_authenticated(*, timeout)“Authenticated” means the backend reports an active login session for the provider, not merely that an executable is present on PATH.
Parameters
timeout: int- Seconds to wait for the credential probe.
Returns
bool-
Truewhen the backend reports an authenticated session.
openai_section()
Return the openai_endpoint wire section; None for every backend but the endpoint backend.
Usage
openai_section()resolve_model()
Resolve an abstract tier to this backend’s provider model id, passing a concrete id through unchanged.
Usage
resolve_model(model)A tier (small/medium/large) maps through models; any other string — a literal provider model id such as claude-fable-5 — returns unchanged, so a caller can pin an exact model anywhere a tier is accepted.
Parameters
model: str- An abstract tier or a concrete provider model id.
Returns
str- The provider model id to run.
schema_for()
Serialize a Pydantic model into the JSON-schema string this backend expects.
Usage
schema_for(model)The core’s strict_schema op applies the backend’s schema_dialect transform over the model’s plain JSON schema; a None dialect emits the schema unchanged.
Parameters
model: type[BaseModel]- The Pydantic model describing the structured output.
Returns
str- A JSON-schema string suitable for this backend’s structured-output argument.
to_response()
Resolve a raw capture into a structured Response via the core resolve op.
Usage
to_response(raw, *, returncode, stderr, spec)output always carries the full raw stream. The core detects failure (a nonzero exit or an error envelope) and extracts the final text and, when spec.response_model is set, the structured value; a pydantic.ValidationError from a non-conforming model routes through error.
Parameters
raw: str-
The raw output read wherever the provider wrote it.
returncode: int-
The process exit code.
stderr: str-
The captured stderr.
spec: RunSpec-
The configured run, carrying the optional
response_modelorschema.
Returns
wire_schema()
Return the portable schema object for spec, from a response_model or a raw schema.
Usage
wire_schema(spec)wire_spec()
Serialize a RunSpec into the portable wire spec the core plan/resolve ops consume.
Usage
wire_spec(spec)CliBackend
Execution contract for the subprocess-backed LLM family.
Usage
CliBackend()The core plans each invocation’s argv, files, env, and result source; this class materializes that plan into temp files and a subprocess, runs it, and resolves the result. Concrete CLI backends supply only ClassVars.
Attributes
binary: str-
Name of the backend’s CLI executable on PATH.
install_hint: str- Suggested shell command to install the CLI.
Methods
| Name | Description |
|---|---|
| accounting() |
Return the (cost_usd, usage) the core’s resolve op reads from raw.
|
| binary_path() |
Return the command every exec of this backend runs as argv[0].
|
| build_command() |
Return the core-planned argv (with ${file:id} placeholders) for a single invocation.
|
| check_status() | Check whether this backend’s CLI is installed and authenticated. |
| claude_isolation() |
Return the isolated config home a claude run substitutes into ${isolated_config_dir}.
|
| env() | Return the core-planned env, substituting the isolated config home into a claude run’s values. |
| invocation() |
Materialize the core’s exec plan into a runnable Invocation.
|
| is_authenticated() | Report whether any of the core’s auth probes for this provider succeeds. |
accounting()
Return the (cost_usd, usage) the core’s resolve op reads from raw.
Usage
accounting(raw)binary_path()
Return the command every exec of this backend runs as argv[0].
Usage
binary_path()The default is binary itself, left for the operating system to resolve through PATH; a backend whose executable ships inside the wheel returns that path instead.
build_command()
Return the core-planned argv (with ${file:id} placeholders) for a single invocation.
Usage
build_command(spec)check_status()
Check whether this backend’s CLI is installed and authenticated.
Usage
check_status(*, timeout=10)Parameters
timeout: int = 10- Seconds to wait for the authentication probe.
Returns
BackendStatus-
BackendReady when authenticated, BackendNotInstalled when
binary_path names no executable file, else BackendNotAuthenticated.
Raises
subprocess.TimeoutExpired-
If is_authenticated exceeds
timeout.
claude_isolation()
Return the isolated config home a claude run substitutes into ${isolated_config_dir}.
Usage
claude_isolation()env()
Return the core-planned env, substituting the isolated config home into a claude run’s values.
Usage
env(spec)Parameters
spec: RunSpec-
The configured run; the plan gates isolation on
spec.isolated.
Returns
dict[str, str]-
The plan’s env map with
${isolated_config_dir}resolved, or{}.
invocation()
Materialize the core’s exec plan into a runnable Invocation.
Usage
invocation(spec)Mints a temp file per files[] entry (writing its content when non-null), points argv[0] at binary_path, substitutes ${file:id} placeholders in the argv and — for a claude run — ${isolated_config_dir}, and wires the plan’s stdout/result source and the minted paths into the returned Invocation for cleanup.
Parameters
spec: RunSpec- The configured run to translate into an invocation.
Returns
Invocation-
An
Invocationcarrying the materialized argv, stdin, and result source.
is_authenticated()
Report whether any of the core’s auth probes for this provider succeeds.
Usage
is_authenticated(*, timeout)Parameters
timeout: int- Seconds to wait for each subprocess-backed probe.
Returns
bool-
Truewhen a probe reports an authenticated session.
ClaudeCliBackend
CliBackend for the Anthropic claude CLI.
Usage
ClaudeCliBackend()The core plans the claude -p argv (prompt delivered over stdin, result read from a stdout file) and lays out the host-free config home this backend seeds with only the active-account pointer and claude.ai OAuth token.
Attributes
models: dict[TModel, str]-
Mapping from abstract model size to a Claude model alias (
haiku/sonnet/opus).
Example
>>> from spawnllm.spec import RunSpec
>>> ClaudeCliBackend().invocation(RunSpec(prompt="hi", model="haiku")).argv[:5][‘claude’, ‘-p’, ‘–no-session-persistence’, ‘–model’, ‘haiku’]
Methods
| Name | Description |
|---|---|
| claude_isolation() | Return the process-lifetime isolated config home, creating and seeding it once. |
claude_isolation()
Return the process-lifetime isolated config home, creating and seeding it once.
Usage
claude_isolation()The core’s claude_isolation_sources op resolves the account pointer, credentials file, and Keychain service from the caller’s effective config home; this host reads those sources (falling back to the Keychain when the credentials file is absent), hands them to claude_isolation_seed for the exact files-and-modes to write, and materializes them into a fresh temp dir removed at interpreter exit. The dir is cached on the backend.
ClaudeSdkBackend
Claude backend hosted through the optional claude-agent-sdk package.
Usage
ClaudeSdkBackend()The SDK bundles Claude Code, so pip install 'spawnllm[sdk]' is the only installation step. Authentication comes from the ambient subscription OAuth session: the bundled CLI resolves /login credentials from the platform keychain or uses CLAUDE_CODE_OAUTH_TOKEN. Here isolated=True makes settings and MCP configuration hermetic through setting_sources=[] and strict_mcp_config, but unlike ClaudeCliBackend it does not seed a fresh CLAUDE_CONFIG_DIR, so credentials come from the ambient config home or CLAUDE_CODE_OAUTH_TOKEN.
api_auth=False blanks Claude’s API-key environment variables because the SDK can only overlay its subprocess environment, not truly unset inherited keys. An explicit RunSpec.env value still wins. ClaudeConfig.mcp_config passes through the SDK’s path-capable mcp_servers field. Its output_format and verbose fields are ignored because the SDK transport always uses stream-JSON with verbose output internally.
Attributes
models: dict[TModel, str]-
Mapping from abstract model size to a Claude model alias.
provider: ProviderName-
Python-only backend identifier used for selection and config.
schema_dialect: str | None- Anthropic strict-schema dialect applied by the core.
Methods
| Name | Description |
|---|---|
| accounting() | Return cost and usage from a reconstructed event via Claude resolution. |
| aexecute() | Execute one SDK query asynchronously and resolve its Claude result event. |
| build_options() | Translate a RunSpec into the SDK options matching the Claude CLI plan. |
| check_status() | Check whether the optional SDK is installed and its bundled CLI authenticated. |
| env() | Return no host-side environment; the SDK owns its subprocess environment. |
| execute() |
Execute one SDK query synchronously via asyncio.run.
|
| is_authenticated() | Report whether the SDK’s Claude Code executable has an active login. |
| to_response() | Resolve the SDK’s reconstructed event through the core’s Claude path. |
accounting()
Return cost and usage from a reconstructed event via Claude resolution.
Usage
accounting(raw)Parameters
raw: str- The reconstructed Claude result event.
Returns
tuple[float | None, dict[str, object] | None]-
The event’s
(cost_usd, usage)pair.
aexecute()
Execute one SDK query asynchronously and resolve its Claude result event.
Usage
aexecute(spec)Parameters
spec: RunSpec- The configured run to execute.
Returns
build_options()
Translate a RunSpec into the SDK options matching the Claude CLI plan.
Usage
build_options(spec)Parameters
spec: RunSpec- The configured run to translate.
Returns
ClaudeAgentOptions-
Options for the SDK’s one-shot
querygenerator.
check_status()
Check whether the optional SDK is installed and its bundled CLI authenticated.
Usage
check_status(*, timeout=10)Parameters
timeout: int = 10- Seconds to wait for the authentication probe.
Returns
BackendStatus-
BackendReady when authenticated, BackendNotInstalled without the
SDK extra, else BackendNotAuthenticated.
env()
Return no host-side environment; the SDK owns its subprocess environment.
Usage
env(_spec)execute()
Execute one SDK query synchronously via asyncio.run.
Usage
execute(spec)Parameters
spec: RunSpec- The configured run to execute.
Returns
Raises
RuntimeError- If called from a thread already running an event loop.
is_authenticated()
Report whether the SDK’s Claude Code executable has an active login.
Usage
is_authenticated(*, timeout)Parameters
timeout: int-
Seconds to wait for
claude auth status.
Returns
bool-
Truewhen the authentication probe exits successfully.
to_response()
Resolve the SDK’s reconstructed event through the core’s Claude path.
Usage
to_response(raw, *, returncode, stderr, spec)Parameters
raw: str-
The reconstructed Claude result event.
returncode: int-
The synthesized transport exit code.
stderr: str-
The SDK error text, when present.
spec: RunSpec- The configured run carrying any response model.
Returns
CodexCliBackend
CliBackend for the OpenAI codex CLI.
Usage
CodexCliBackend()The core plans a codex exec argv that runs an ephemeral session in a read-only sandbox, resolving the schema to an --output-schema file and the final message to an -o file. It pins service_tier=fast by default (an isolated run passes --ignore-user-config, dropping a user-level tier pin, and the standard tier turns long prompts into multi-minute runs).
Attributes
models: dict[TModel, str]- Mapping from abstract model size to an OpenAI model name.
Example
>>> from spawnllm.spec import RunSpec
>>> CodexCliBackend().invocation(RunSpec(prompt="hi", model="gpt-5.5")).argv[:4][‘codex’, ‘exec’, ‘–ephemeral’, ‘–sandbox’]
GeminiCliBackend
CliBackend for Google’s gemini CLI.
Usage
GeminiCliBackend()The core plans a gemini --model … -o json invocation with the prompt delivered inline via -p; structured output appends the JSON schema and an instruction to emit only conforming JSON. Authentication prefers cached OAuth credentials and falls back to a GEMINI_API_KEY/GOOGLE_API_KEY env key.
Attributes
models: dict[TModel, str]- Mapping from abstract model size to a Gemini model name.
Example
>>> from spawnllm.spec import RunSpec
>>> GeminiCliBackend().invocation(RunSpec(prompt="hi", model="gemini-2.5-flash")).argv[:5][‘gemini’, ‘–model’, ‘gemini-2.5-flash’, ‘-o’, ‘json’]
AntigravityCliBackend
CliBackend for the Antigravity agy CLI, a Gemini-family successor.
Usage
AntigravityCliBackend()The core plans an agy --model … -p invocation and reads its plain-text stdout. Authentication prefers an Antigravity login stored in the macOS keychain and falls back to a GEMINI_API_KEY/ANTIGRAVITY_API_KEY env key.
Attributes
models: dict[TModel, str]- Mapping from abstract model size to an Antigravity model name.
Example
>>> from spawnllm.spec import RunSpec
>>> AntigravityCliBackend().invocation(RunSpec(prompt="hi", model="gemini-3.5")).argv[:3][‘agy’, ‘–model’, ‘gemini-3.5’]
AppleBackend
CliBackend for the spawnllm-apple on-device Foundation Models sidecar.
Usage
AppleBackend()The core plans a single spawnllm-apple invocation carrying the whole request as one JSON object on stdin and reads one JSON envelope back from stdout. Generation is local to the device, so there is no credential: RunSpec’s model, isolated, api_auth, env, and cwd are all inert, and “authenticated” means the sidecar’s --probe reports Apple Intelligence available. AppleConfig carries the knobs that do apply — the use case and guardrails the session is built with, its instructions, and the decoding options.
The macOS wheel bundles the sidecar, so binary_path prefers it and falls back to PATH; where neither resolves — Linux, or a wheel built without it — the backend reports BackendNotInstalled and auto-selection passes it by.
Attributes
models: dict[TModel, str]-
Empty identity mapping; the device hosts exactly one model.
provider: ProviderName-
Provider identifier keying AppleConfig on a RunSpec.
binary: str-
Name of the sidecar executable.
install_hint: str-
Suggested shell command to build the sidecar.
schema_dialect: str | None-
Apple strict-schema dialect the core applies.
auto_select_tiers: frozenset[TModel] | None-
Only
small; the on-device model is a small model.
Example
>>> from spawnllm.spec import RunSpec
>>> AppleBackend().invocation(RunSpec(prompt="ping", model="small")).stdin[:16]‘{“prompt”:“ping”’
Methods
| Name | Description |
|---|---|
| binary_path() |
Return the wheel-bundled sidecar’s path, or the bare binary name for a PATH lookup.
|
binary_path()
Return the wheel-bundled sidecar’s path, or the bare binary name for a PATH lookup.
Usage
binary_path()MlxBackend
In-process backend that runs a prompt through a local MlxEngine.
Usage
MlxBackend(engine, *, max_tokens=512)Unlike the CLI backends this is never auto-selected; the consumer constructs it explicitly with a loaded MlxEngine. RunSpec.model is ignored — the engine is already bound to a fused model — and every provider config and CLI flag is irrelevant, so models is the empty identity mapping. The core knows no mlx provider, so resolution stays in-process here.
Example
>>> backend = MlxBackend(engine=engine, max_tokens=512)
>>> backend.execute(RunSpec(prompt="ping", model="local"))Methods
| Name | Description |
|---|---|
| check_status() | Report BackendReady; the engine is local and always available once loaded. |
| env() | Return no extra environment variables; MLX runs in-process with nothing to isolate. |
| is_authenticated() |
Report True; the engine is local and needs no credentials.
|
| result_value() |
Return the structured_output from a stream-json result event, else raw parsed as JSON.
|
| to_response() |
Resolve locally-generated text into a Response, validating a response_model when set.
|
check_status()
Report BackendReady; the engine is local and always available once loaded.
Usage
check_status(*, timeout=10)env()
Return no extra environment variables; MLX runs in-process with nothing to isolate.
Usage
env(_spec)is_authenticated()
Report True; the engine is local and needs no credentials.
Usage
is_authenticated(*, timeout)result_value()
Return the structured_output from a stream-json result event, else raw parsed as JSON.
Usage
result_value(raw)to_response()
Resolve locally-generated text into a Response, validating a response_model when set.
Usage
to_response(raw, *, returncode, stderr, spec)OpenAiEndpointBackend
LlmBackend that POSTs to an OpenAI-compatible /chat/completions endpoint.
Usage
OpenAiEndpointBackend(base_url, model, *, api_key="local", transport=None)Unlike the CLI backends this drives a raw httpx request rather than a subprocess, and it is never auto-selected; the consumer constructs it explicitly with a base_url and a literal model. The core plans the HTTP request (url, headers, body — a strict json_schema response_format when a response_model is set) and resolves the response; every abstract tier maps to the one pinned model, and RunSpec.model is ignored at request time.
Parameters
base_url: str-
Root URL of the server;
/chat/completionsis appended. model: str-
The literal model id sent in every request body.
api_key: str = "local"-
Bearer token for the
Authorizationheader; defaults to"local"for self-hosted servers that ignore it. transport: httpx.AsyncBaseTransport | None = None-
Async transport injected into the
httpx.AsyncClientused by aexecute — e.g. a record/replay caching transport;Noneuses httpx’s default transport. The synchronous execute path always uses the default transport.
Example
>>> backend = OpenAiEndpointBackend("http://localhost:8000/v1", "qwen3")
>>> backend.execute(RunSpec(prompt="ping", model="qwen3"))Methods
| Name | Description |
|---|---|
| check_status() | Report BackendReady; the endpoint is reached per-request and carries its own auth. |
| env() | Return no extra environment variables; the endpoint is reached over HTTP with nothing to isolate. |
| is_authenticated() |
Report True; credentials travel inline as the Authorization bearer token on every request.
|
| openai_section() |
Return the openai_endpoint wire section the core turns into the HTTP request.
|
check_status()
Report BackendReady; the endpoint is reached per-request and carries its own auth.
Usage
check_status(*, timeout=10)env()
Return no extra environment variables; the endpoint is reached over HTTP with nothing to isolate.
Usage
env(_spec)is_authenticated()
Report True; credentials travel inline as the Authorization bearer token on every request.
Usage
is_authenticated(*, timeout)openai_section()
Return the openai_endpoint wire section the core turns into the HTTP request.
Usage
openai_section()LlmBackends
Registry mapping each specialty to the LlmBackend that serves it.
Usage
LlmBackends()debugging and review route to CodexCliBackend; general routes to the first-priority ClaudeSdkBackend.
Attributes
LLM_BACKENDS: dict[TSpecialty, LlmBackend]- Mapping from specialty to its backend instance.
Methods
| Name | Description |
|---|---|
| for_specialty() | Return the backend registered for a specialty. |
for_specialty()
Return the backend registered for a specialty.
Usage
for_specialty(specialty)Parameters
specialty: TSpecialty-
One of
debugging,review, orgeneral.
Returns
LlmBackend-
The LlmBackend instance that serves
specialty.
select_backend()
Return the first installed, authenticated backend in priority order.
Usage
select_backend(*, specialty=None, model=None, timeout=10)A specialty, when given, promotes its registered backend to the front of the chain; the chain otherwise follows PRIORITY, minus GeminiCliBackend (its Code Assist OAuth tier is retired, so it reports ready yet fails at call time — reach it only via an explicit backend=) and minus any backend whose auto_select_tiers excludes model. The first backend whose check_status reports BackendReady wins, short-circuiting the rest; backends that time out are skipped.
Parameters
specialty: TSpecialty | None = None-
Specialty whose backend is tried first, or
None. model: TModel | str | None = None-
The abstract tier the run wants, gating tier-restricted backends; a concrete provider model id or
Noneexcludes every one of them. timeout: int = 10- Seconds to wait for each backend’s authentication probe.
Returns
LlmBackend- The first ready LlmBackend.
Raises
BackendUnavailable- When no backend is installed and authenticated.
BackendStatus
Result of LlmBackend.check_status: BackendReady, BackendNotInstalled, or BackendNotAuthenticated.
BackendStatus=BackendReady | BackendNotInstalled | BackendNotAuthenticated
BackendReady
A backend whose CLI is installed and authenticated.
Usage
BackendReady(binary)Attributes
binary: str- Name of the backend’s CLI executable on PATH.
BackendNotInstalled
A backend whose CLI is not on PATH.
Usage
BackendNotInstalled(binary, install_hint)Attributes
binary: str-
Name of the backend’s CLI executable.
install_hint: str- Suggested shell command to install the CLI.
BackendNotAuthenticated
A backend whose CLI is installed but not authenticated.
Usage
BackendNotAuthenticated(binary)Attributes
binary: str- Name of the backend’s CLI executable on PATH.