Two GPUs, arbitrated by a lease-based control plane.

Most local AI setups solve model loading and leave arbitration to the human: remembering which model is resident, on which card, and killing processes by hand before starting the next one. That holds until a second application exists. Then two workloads land on the same card, allocation fails deep inside a runtime, and the error that comes back has nothing to do with the actual cause.

Warden replaces that with a control plane that treats VRAM as a declared budget and hands out time-bounded capability leases instead of start and stop commands. A client says that it needs the chat model for five minutes, and Warden decides which card it runs on, starts the right binary for that card, returns the URL it chose, and stops the process once nobody is using it. A client that crashes cannot strand a model on a GPU, because its lease simply expires.

Once inference is a lease rather than a process, an application can build on top of it. DoubleHarness runs a planner, a worker and a verifier as three separately routed models with different privileges, so a local model can change a real repository under enforced limits. The planner reads only a frozen commit. The worker writes only the files its task listed, inside a disposable Git worktree. The verifier sees the diff and check output, never the tools.

AI-Home-Lab-Writer is the MacBook-side client. It never assumes a port, never launches a runtime, and treats a capacity refusal as a normal outcome with a stated reason rather than an error to retry blindly. It exists to keep the Warden interface small enough that a UI can sit on top of it without duplicating any GPU logic.

The result is that local hardware gets what a shared GPU host usually lacks: declared VRAM budgets, explicit per-task file contracts, refusals that name their cause, and an append-only evidence trail for every change that reaches a repository.

Everything below is running code on the hardware in the spec panel. The ports, budgets, model routes and failure rules on this page are the ones in the committed configuration.

A profile is a role such as chat, worker or comfy. A target is a place it can run. A profile declares one variant per target, each with its own command line, environment and VRAM estimate, because the real unit of difference is not a config flag: the CUDA and ROCm builds of llama.cpp are different binaries, and the two ComfyUI portables are different folders on disk. Swapping a card is therefore not a special case, it is just picking a different variant.

The same idea carries a third kind of target. vulkan is not a card, it is a span across both of them, used when one model is too large for either. It owns no memory of its own, so the arbiter admits it against each physical card it touches, which means a span request can evict on two cards at once and is correctly refused if either one is full.

Admission runs on declared estimates rather than measurements, because the decision has to be made before the process exists, when there is nothing to measure yet. The estimate is a worst case footprint that includes model residency, the KV allocation for the full accepted context, the parallel slot multiplier, and an MTP reserve where the model supports it. Measured peaks are still recorded, but only so the dashboard can show you the gap between what a profile reserved and what it actually used. Telemetry never feeds an admission decision, so a card whose driver reports nothing useful still gets supervised correctly.

When a card is full, Warden tries eviction first, and only considers profiles that are unleased, unpinned and strictly lower priority. If that is not enough it returns 409 with the reason: which profiles are holding the memory, why each one could not be moved, and which other target could serve the request. It never queues silently, because a queue turns a capacity problem into a latency mystery, and the caller is the only party that knows whether waiting is acceptable.

# warden.yaml, abbreviated
profiles:
  chat:
    kind: llm
    priority: 100
    default_device: cuda0
    default_context: 32768
    mtp: true
    health: {type: http, path: /v1/models}

    # one entry per model, priced per target
    models:
      - label: Qwen3.6 35B-A3B
        supports_mtp: true
        hip0:   {arg: "...Q5_K_M.gguf", vram_mb: 26000}
        cuda0:  {arg: qwen_qwen3.6-35b-a3b, vram_mb: 27000}
        vulkan: {arg: qwen_qwen3.6-35b-a3b,
                 split: {cuda0: 14000, hip0: 14000}}

    # one launch recipe per target
    variants:
      cuda0:  # LM Studio shim, CUDA engine
        port: 1234
        cmd: [py, -3.11, -m, warden.lmstudio, ...]
      hip0:   # ROCm llama.cpp, its own binary
        port: 1235
        cwd: E:/llama-rocm-lemonade/bin
        cmd: [llama-server.exe, -m, "{model}", ...]

The same profile, two vendors, one price list. footprint() reads this and returns megabytes per physical card, which is the only number the arbiter trusts.

The planner runs on the W7800 at Q8, the worker on the RTX 5090 at Q6, and the verifier follows the planner route so that it never shares a model instance with the code it is judging. Those are configured defaults rather than embedded rules: the console offers only the models Warden reports as admissible for each role's card, validates the choice against the live catalog, and refuses routing changes while an execution is running.

Splitting the roles across two cards keeps the verifier independent as well as parallel. A verifier sharing the worker's model instance can only report that a model agrees with itself. The router in DoubleHarness warns explicitly when a configuration would collapse that separation, and again when planner and worker land on the same physical card and lose their concurrency.

The core of the design is the transaction boundary. Planning is read-only against a commit SHA recorded once at session start, so the plan cannot drift while it is written. Execution happens in a disposable Git worktree, never the operator's checkout. Deterministic checks (tests, lint, build) run before any model is asked for a judgement. Integration into the session branch is the last step and stays an operator decision.

The key gate is the reconciliation in step three of the figure below. Neither the worker's own report nor Git alone is trusted: the set of files Git reports as changed must equal the set the worker claims it wrote, and both must sit inside the allowlist the plan issued for that task. Disagreement in either direction fails the task, so a model that quietly edits an undeclared file and a model that claims work it did not do are caught by the same check.

# src/codeharness/worker.py
def _reconcile(self, task, result, actual):
    claimed = set(result.files_written)
    allowed = set(task.writable_files)

    # the worker does not get the benefit
    # of the doubt in either direction
    if claimed != set(actual):
        raise TaskTransactionError(
            f"worker claim mismatch: "
            f"changed={sorted(actual)}, "
            f"claimed={sorted(claimed)}")

    outside = set(actual) - allowed
    if outside:
        raise TaskTransactionError(
            f"worker changed files outside "
            f"allowlist: {sorted(outside)}")

Run twice: once after the worker finishes, and again after the deterministic checks, so a formatter or a test run cannot smuggle a file past the contract.

The writer client runs on the MacBook, which has its own Apple Silicon GPU and 24 GB of unified memory but hosts no models; the workstation's two cards do that. The client does not know that the chat profile might be on the NVIDIA card behind LM Studio today and on the AMD card behind llama.cpp tomorrow. It asks for a profile, uses the base_url the lease returned, and renews in the background while long work continues. When the lease closes, the VRAM is released without the client issuing a stop.

It also handles Warden's error contract precisely. A 409 is surfaced immediately as a capacity condition with the reason Warden gave, not retried in a loop that will fail identically. A 503 means the workstation is paused because the operator pressed Release all GPUs, which is a human decision, not a transient fault. The two are reported differently instead of both becoming a hang.

Everything the client needs is seven environment variables and one context manager. That constraint is the point: if the client had needed a path, a port or a binary name, the Warden abstraction would have been leaking workstation detail it is meant to hide.

# the entire client contract
async with warden.lease(
        "chat", ttl_s=300) as lease:
    reply = await chat(lease.base_url, messages)

# renewal runs in the background at ttl/2,
# release happens on the way out, and a
# crash needs no cleanup path at all

# failures are named, not swallowed
class NoCapacityError(WardenError): ...
class WardenPausedError(WardenError): ...

The same shape is reused inside DoubleHarness, where a lease handle is held per agent role and lease transitions are serialised per Warden profile.