## Serve


## serve.RapidMlxSettings


The `[serve.rapid-mlx]` section: the daily-pinned rapid-mlx version, model, and port.


Usage

``` python
serve.RapidMlxSettings()
```


The default text lane. On ordinary tool calling it and [LlamaServerSettings](serve.md#athome.serve.LlamaServerSettings) are equivalent: serving the same Qwen3.6-35B-A3B to both, 42 calls each, every call dispatched and every argument came back valid and schema-conformant -- flat, enum, nested, arrays of objects, multi-tool selection, and numeric-verbatim alike. Pick between them on the two edges below, not on general quality.

rapid-mlx serves MLX-native weights only, and honors `tool_choice="required"`.


#### Warning

rapid-mlx extracts tool calls with a delimiter-based parser (it advertises `qwen3_coder_xml`), and a string argument containing that delimiter is truncated at it: send `</function>` inside a payload and the argument arrives silently short, as well-formed JSON no downstream check can catch. Serve [LlamaServerSettings](serve.md#athome.serve.LlamaServerSettings) when arguments can carry tool-call syntax -- quoted model output, agent transcripts, scraped markup.


## serve.MlxVlmSettings


The `[serve.mlx-vlm]` section: the daily-pinned mlx-vlm version, vision model, and port.


Usage

``` python
serve.MlxVlmSettings()
```


## serve.LlamaServerSettings


The `[serve.llama-server]` section: a full llama-server command string and its port.


Usage

``` python
serve.LlamaServerSettings()
```


The opt-in GGUF lane, equivalent to the default [RapidMlxSettings](serve.md#athome.serve.RapidMlxSettings) on ordinary tool calling. Its one advantage is string fidelity: it returns arguments containing tool-call delimiter syntax verbatim, where rapid-mlx's parser truncates them. That is the reason to select it.

Takes a full command string rather than a model and port, because a llama-server invocation carries flags no recipe can generalize.


#### Warning

llama-server ignores `tool_choice="required"` -- it applies no grammar constraint, so instead of forcing a call it can answer in prose and run to `max_tokens`, returning no tool call at all. Code that depends on being handed one must stay on rapid-mlx or treat a missing call as a real outcome.


## serve.ManagedServer


A recipe-configured OpenAI-compatible server with lifecycle + health, over any backend.


Usage

``` python
serve.ManagedServer(recipe, *, model=None, port=None)
```


The recipe picks the backend: a local uvx subprocess (`rapid-mlx`, `mlx-vlm`, `llama-server`) or a scale-to-zero hosted Modal vLLM endpoint (`modal-vllm`). Health, model verification, readiness, and the client are identical across both -- only where the endpoint lives and how it starts and stops differ, which is all a `ServeBackend` answers.

`model` and `port` override the recipe's configured pair for one server -- serving an arbitrary artifact (a freshly fused MLX directory, say) without touching the process-global settings. An overridden server is a distinct managed process: it carries its own detach run name and launchd label, so it neither adopts nor tears down the config-driven one.


#### Parameter Attributes


`recipe: Recipe`  

`model: str | None = None`  

`port: int | None = None`  


#### Example

``` python
>>> server = ManagedServer("rapid-mlx")
>>> handle = await server.ensure()
>>> fused = ManagedServer("rapid-mlx", model="/runs/watcher/fused", port=8410)
```


#### Attributes

| Name | Description |
|----|----|
| [api_key](#athome.serve.ManagedServer.api_key) | The bearer credential that reaches this server's endpoint. |
| [backend](#athome.serve.ManagedServer.backend) | The substrate this recipe runs on: a local subprocess, or a hosted Modal endpoint. |
| [base_url](#athome.serve.ManagedServer.base_url) | The OpenAI-compatible `/v1` endpoint this server answers on. |
| [identity](#athome.serve.ManagedServer.identity) | What distinguishes an overridden server: the port it listens on and the model it serves. |
| [label](#athome.serve.ManagedServer.label) | The launchd label owning this server's agent. |
| [required_models](#athome.serve.ManagedServer.required_models) | Every model id the endpoint must report, not just the base. |
| [run_name](#athome.serve.ManagedServer.run_name) | The detach run name owning this server's process. |
| [served_model](#athome.serve.ManagedServer.served_model) | The model this server serves: the override, else the recipe's configured model. |
| [served_port](#athome.serve.ManagedServer.served_port) | The port this server listens on: the override, else the configured port; `None` when hosted. |


##### api_key


The bearer credential that reaches this server's endpoint.


`api_key: str`


##### backend


The substrate this recipe runs on: a local subprocess, or a hosted Modal endpoint.


`backend: ServeBackend`


##### base_url


The OpenAI-compatible `/v1` endpoint this server answers on.


`base_url: str`


##### identity


What distinguishes an overridden server: the port it listens on and the model it serves.


`identity: str`


The model belongs in the identity, not just the port: two runs that land on one port would otherwise share a run name and a launchd label, and each would tear down the other's process while verifying it served the wrong model.


##### label


The launchd label owning this server's agent.


`label: str`


##### required_models


Every model id the endpoint must report, not just the base.


`required_models: set[str]`


An override serves exactly the one model it names. A hosted vLLM recipe serves its base model *and* the LoRA adapter it exists to score under a second id -- an endpoint serving only the base would pass a base-only identity check while missing the adapter. A `llama-server` command advertises nothing checkable, so it requires none.


##### run_name


The detach run name owning this server's process.


`run_name: str`


##### served_model


The model this server serves: the override, else the recipe's configured model.


`served_model: str | None`


##### served_port


The port this server listens on: the override, else the configured port; `None` when hosted.


`served_port: int | None`


A hosted recipe answers on a workspace-scoped HTTPS URL, not a loopback port, so it has none.


#### Methods

| Name | Description |
|----|----|
| [client()](#athome.serve.ManagedServer.client) | Return an `AsyncOpenAI` client bound to this recipe's endpoint. |
| [ensure()](#athome.serve.ManagedServer.ensure) | Return a healthy server for this recipe, starting it if necessary. |
| [health()](#athome.serve.ManagedServer.health) | Return `True` when `GET /v1/models` answers 2xx within the backend's probe timeout. |
| [idle()](#athome.serve.ManagedServer.idle) | An `~athome.idle.IdleResource` that spawns this server on first use and stops it once idle. |
| [probe()](#athome.serve.ManagedServer.probe) | Report health without waking a scale-to-zero backend -- a passive, side-effect-free check. |
| [stop()](#athome.serve.ManagedServer.stop) | Stop this recipe by tearing down its backend. |
| [verify_served_model()](#athome.serve.ManagedServer.verify_served_model) | Raise [ServeError](serve.md#athome.serve.ServeError) unless `GET /v1/models` lists every model this server requires. |


##### client()


Return an `AsyncOpenAI` client bound to this recipe's endpoint.


Usage

``` python
client(*, cached=False)
```


##### Parameters


`cached: bool = ``False`  
Route requests through [athome.llmcache.transport()](llm-cache.md#athome.llmcache.transport) for record/replay.


##### Returns


`AsyncOpenAI`  
A client with [base_url](serve.md#athome.serve.ManagedServer.base_url) set to the recipe's `/v1` endpoint and its api key.


##### ensure()


Return a healthy server for this recipe, starting it if necessary.


Usage

``` python
ensure(*, persistent=False)
```


A recipe already answering healthy returns its handle without restarting. Otherwise the backend brings it up -- a local recipe launches its command vector, detached via `athome.detach` (`persistent=False`) or as a launchd [KeepAlive](launchd.md#athome.launchd.KeepAlive) agent (`persistent=True`); a hosted recipe deploys its Modal app -- and the call blocks until health polling succeeds.


##### Parameters


`persistent: bool = ``False`  
Install a launchd KeepAlive agent instead of a detached run (local recipes only).


##### Returns


<a href="serve.html#athome.serve.ServerHandle" class="gdls-link gdls-code"><code>ServerHandle</code></a>  
The running server's handle.


##### Raises


<a href="serve.html#athome.serve.ServeError" class="gdls-link gdls-code"><code>ServeError</code></a>  
The adopted or launched server serves a different model.

<a href="serve.html#athome.serve.HealthTimeout" class="gdls-link gdls-code"><code>HealthTimeout</code></a>  
The server did not report healthy within the ready timeout.


##### health()


Return `True` when `GET /v1/models` answers 2xx within the backend's probe timeout.


Usage

``` python
health()
```


##### idle()


An `~athome.idle.IdleResource` that spawns this server on first use and stops it once idle.


Usage

``` python
idle(*, ttl_s)
```


Each `~athome.idle.IdleResource.use()` ensures the server is healthy and yields its handle; the resource's reaper stops the server once it has sat unused past `ttl_s`.


##### Parameters


`ttl_s: float`  
Idle seconds before the reaper stops the server.


##### Returns


<a href="idle.html#athome.idle.IdleResource" class="gdls-link gdls-code"><code>IdleResource</code></a>`[`<a href="serve.html#athome.serve.ServerHandle" class="gdls-link gdls-code"><code>ServerHandle</code></a>`]`  
A resource wired to [ensure()](serve.md#athome.serve.ManagedServer.ensure) and [stop()](serve.md#athome.serve.ManagedServer.stop).


##### probe()


Report health without waking a scale-to-zero backend -- a passive, side-effect-free check.


Usage

``` python
probe()
```


A local recipe probes its loopback endpoint (free); a hosted recipe observes its deployment through Modal's control plane, never an HTTP request that would cold-boot and bill a container.


##### stop()


Stop this recipe by tearing down its backend.


Usage

``` python
stop()
```


A local recipe boots out its launchd agent *and* kills its detached process group -- the two run independently, so a launchd uninstall failure never skips the process kill. A hosted recipe has no local process: Modal's scaledown reaps it, so this is a no-op.


##### verify_served_model()


Raise [ServeError](serve.md#athome.serve.ServeError) unless `GET /v1/models` lists every model this server requires.


Usage

``` python
verify_served_model()
```


This is an identity check, not authentication: a process holding the endpoint could spoof the model ids, and trusting the resolved endpoint is inherent to the design.


## serve.ServerHandle


A running recipe's port (`None` for a hosted endpoint), pid (`None` for a launchd


Usage

``` python
serve.ServerHandle(recipe, port, pid, base_url, api_key)
```


service or a hosted endpoint), OpenAI base URL, and the bearer credential that reaches it.

[api_key](serve.md#athome.serve.ManagedServer.api_key) is `"local"` for a loopback recipe and the recipe's configured key for a hosted one, so a consumer holding only the handle can authenticate against the endpoint.


#### Parameter Attributes


`recipe: Recipe`  

`port: int | None`  

`pid: int | None`  

`base_url: str`  

`api_key: str`  


#### Example

``` python
>>> handle = await up("mlx-vlm")
>>> handle.base_url
```

'http://127.0.0.1:8401/v1'


## serve.up()


Ensure `recipe` is running and healthy, returning its handle.


Usage

``` python
serve.up(recipe, *, persistent=False)
```


## serve.down()


Stop `recipe` -- launchd uninstall or detached-process kill.


Usage

``` python
serve.down(recipe)
```


## serve.settings_for()


Usage

``` python
serve.settings_for(recipe)
```


## serve.command_for()


The recipe's command vector; `model` and `port` override its configured pair.


Usage

``` python
serve.command_for(recipe, *, model=None, port=None)
```


## serve.probe_all()


Report the health of every configured recipe, without waking a scale-to-zero endpoint.


Usage

``` python
serve.probe_all()
```


## serve.ServeError


Raised when a managed server cannot be started, stopped, or reached.


Usage

``` python
serve.ServeError()
```


## serve.HealthTimeout


Raised when a spawned server does not report healthy within the ready timeout.


Usage

``` python
serve.HealthTimeout()
```