Idle

idle.IdleResource

A lazily-loaded value that unloads itself once idle past its TTL.

Usage

Source

idle.IdleResource(
    load,
    unload,
    *,
    ttl_s,
)

load is awaited at most once per loaded lifetime, under a lock, so concurrent first uses trigger exactly one load; a load that raises leaves the resource unloaded and the next use() retries fresh. unload closes over the consumer’s own state — the resource holds only the loaded T — and runs from the reaper (sweep() / run()) once no use is in flight and the idle window has elapsed, or immediately from discard().

Attributes

load: Callable[[], Awaitable[T]]

Awaited to produce the value; called once per loaded lifetime.

unload: Callable[[], Awaitable[None]]

Awaited to release the consumer’s state after the value reference is dropped.

ttl_s: float

Idle seconds a loaded value must sit unused before the reaper unloads it.

Example

>>> resource = IdleResource(spawn_server, teardown_server, ttl_s=300.0)
>>> async with resource.use() as server:
...     await server.query(prompt)

Attributes

Name	Description
loaded	Whether a value is currently held.

loaded

Whether a value is currently held.

loaded: bool

Methods

Name	Description
discard()	Drop the value and await unload now, bypassing both the TTL and the in-flight count.
run()	Reaper loop: every interval_s seconds, sweep the resource. Runs until cancelled.
sweep()	Unload the value iff it is loaded, no use is in flight, and it has been idle past ttl_s.
use()	Yield the loaded value, loading it first if necessary and counting the use in flight.

discard()

Drop the value and await unload now, bypassing both the TTL and the in-flight count.

Usage

Source

discard()

For a value known bad: in-flight holders keep their own local references and fail on their own, so the reaper’s guards are deliberately skipped here.

run()

Reaper loop: every interval_s seconds, sweep the resource. Runs until cancelled.

Usage

Source

run(*, interval_s=DEFAULT_REAP_INTERVAL_S)

Each iteration isolates its own sweep: a sweep that raises — a failed unload, most likely — is logged and the loop keeps running, so one transient failure never silently kills the reaper. Cancellation still ends the loop: anyio raises it as a BaseException, which except Exception cannot catch.

sweep()

Unload the value iff it is loaded, no use is in flight, and it has been idle past ttl_s.

Usage

Source

sweep(*, now=None)

The value reference is dropped before unload is awaited, with no backup local ref (a ref would pin MLX weights and defeat eviction). So a failed unload leaves the resource unloaded while the underlying resource is possibly still alive; the next load must tolerate that by re-adopting or reloading it.

Parameters

now: float | None = None

The monotonic time to measure the idle window against; resolves the clock itself when omitted.

use()

Yield the loaded value, loading it first if necessary and counting the use in flight.

Usage

Source

use()

Loads under the lock on first use so concurrent callers share one load. The in-flight count is held for the body’s duration, blocking the reaper; on exit it records the completion time against the monotonic clock so the idle window starts from the last release.