clanker-saddle

clanker-saddle

ClankerSaddle is an LLM task orchestration platform: a Statum + Spry web application that queues work for humans and dispatches it to saddle daemons — one-container agents that work on real git checkouts through bash sessions with full auditability. You design tasks during the day; the agent works while you sleep.

Architecture

Browser ←→ clanker-saddle (Spry app: SQLite queue, Statum UI, dispatcher)
                          ↕ HTTP (daemons push; asks/wakes only go out)
               saddle daemons (Vala: bash session + LLM loop, port 7100)
                          ↕ curl
               any OpenAI-compatible chat-completions endpoint

The web app owns projects, goals, flat tasks and the saddle registry. Work acquisition is ask-based (plan 10): daemons sleep when idle and ask POST /api/saddle/next-task (per-saddle bearer token); the allocation is one write transaction that pairs the asking saddle with the best offerable task — workspace affinity first (a task parked on this saddle is offered only to it), then capability-tag matching. The dispatcher wakes sleeping daemons when work becomes ready (POST /wake, the affine saddle first for parked tasks) and declares a saddle offline when no ask arrives within the lost-wake guard window. Telemetry is push-based (phase 2): daemons flush raw PTY batches to POST /api/saddle/chunks and round results with state transitions to POST /api/saddle/results — blocked raises the question as an answerable block and parks the saddle, suspended parks until the goal agent's next terminal-event resume, done/failed complete or fail the task. Nothing polls for output; GET /status against a daemon is exceptional only — the staleness probe (a silent runner owes one probe after SADDLE_STALE_S; a failed probe requeues its task and marks the saddle offline) and the manual refresh button. A recovered daemon's ask flips an offline row back to idle. Which saddle runs what derives from open sessions — busy iff an open session exists, idle otherwise; only the offline probe state is stored. Capability awareness: every assignment carries a capabilities context — saddle_tags (the allocated saddle's own tags) and fleet_tags (every tag name known across the fleet) — which the daemon renders into its system prompt; GET /api/saddle/capabilities (per-saddle token) feeds its saddle capabilities CLI, and the goal agent's task-create callback attaches explicit, validated capability requirements (unknown tags reject with the valid list). The home dashboard pins OPEN blocks (questions, RFCs, reviews) as newest-first link-through cards above a cross-task activity timeline (last ~50 events, newest first); the task page leads with its status block and orders blocking sections newest-first.

Component	Lives	Role
Web app	src/	pages, actions, services, dispatcher, saddle API
Saddle daemon	../saddle	bash PTY session, LLM loop, control server
Mock LLM	../saddle/test/mock_llm.py	scripted endpoint for testing

Orchestration behaviour to know:

Setting	Meaning
SADDLE_LLM_BASE_URL	LLM endpoint sent with every assignment; unset pauses dispatch
SADDLE_LLM_API_KEY / SADDLE_LLM_MODEL	Optional credentials/model for the above
SADDLE_LOST_WAKE_S	Lost-wake guard window (default 15s): no ask after a wake → the saddle goes offline and the next capable one is woken
SADDLE_STALE_S	Running-liveness window (default 90s): no chunks or results for this long → one exceptional GET /status probe; failure requeues the task and marks the saddle offline
SADDLE_PROJECTS_ROOT	Checkouts are addressed as root/<project-name> inside the saddle container (default /workspace)

Quick start (local)

export WS_PREFIX="$HOME/.local"
export PKG_CONFIG_PATH="$WS_PREFIX/lib64/pkgconfig${PKG_CONFIG_PATH:+:$PKG_CONFIG_PATH}"
export XDG_DATA_DIRS="$WS_PREFIX/share${XDG_DATA_DIRS:+:$XDG_DATA_DIRS}"
export LD_LIBRARY_PATH="$WS_PREFIX/lib64${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"

meson setup builddir && ninja -C builddir

Run the app, a saddle daemon and a mock LLM in three shells (the mock's port must be reachable from the daemon; project checkouts live under SADDLE_PROJECTS_ROOT):

export SADDLE_LLM_BASE_URL=http://127.0.0.1:9999 SADDLE_LLM_MODEL=mock
export SADDLE_PROJECTS_ROOT=/tmp/clanker-e2e/workspace
./builddir/clanker-saddle 8080

SPRY_SEED_ADMIN_USERNAME/SPRY_SEED_ADMIN_PASSWORD default to admin/change-me when unset; export them to seed different credentials.

../saddle/builddir/src/saddle --port 7100

python3 ../saddle/test/mock_llm.py --port 9999

Browse to `http://localhost:8080`, log in with the seeded admin account,
register the saddle on `/saddles` with host `localhost:7100`, and create
your first project and task. `spry dev` wraps the first shell (rebuild +
restart on save); `web-config.json` (from `spry keys`) keeps sessions
alive across restarts and is never packaged or baked into images.

The database is `./clanker-saddle.sqlite`, or `SPRY_DB_PATH` when set.

## Container deployment (spry deploy / usm)

Both halves package as USM packages and build container images from
their own checkouts — no Dockerfiles outside the repos:

bash

this app (a Spry application — spry deploy wraps usm manifest deploy):

spry deploy --verbose

--repository https://packages.astrologue.nz/repo/web-stack/repo.usmr

the daemon (from ../saddle — usm manifest deploy directly; --spm dnf

is required for the in-container manage dependencies):

cd ../saddle && usm manifest deploy --spm dnf

--repository https://packages.astrologue.nz/repo/web-stack/repo.usmr

Each writes `<name>-<version>.image.tar.xz` (single-stage, built in
container against the canonical Web-Stack repository — allow time for
the toolchain build on first run). Load and run:

bash podman load -i clanker-saddle-0.1.image.tar.xz podman load -i saddle-0.1.image.tar.xz

podman network create clanker-net podman run -d --name clanker-web --userns=keep-id --network clanker-net

-p 8080:8080 -v "$PWD/data:/var/lib/clanker-saddle:Z" \
-e SPRY_SEED_ADMIN_USERNAME=admin -e SPRY_SEED_ADMIN_PASSWORD=change-me \
-e SADDLE_LLM_BASE_URL=http://127.0.0.1:9999 \
-e SADDLE_PROJECTS_ROOT=/workspace \
-e SADDLE_CALLBACK_BASE_URL=http://clanker-web:8080 \
localhost/clanker-saddle:0.1

podman run -d --name clanker-saddle --userns=keep-id --network clanker-net

-p 7100:7100 -v "$PWD/workspace:/workspace:Z" \
localhost/saddle:0.1

Register the saddle with its **network** address (`clanker-saddle:7100`)
— the web app talks to the daemon container-to-container, while the LLM
base URL is dialled by the daemon itself (a mock can run inside the
saddle container via `podman exec -d clanker-saddle …`, reachable at
`http://127.0.0.1:9999`). `SADDLE_CALLBACK_BASE_URL` (or the
`web-config.json` `saddle_callback_base_url` key) must be the app's
address **as the daemon dials it** — every registered daemon pushes
telemetry, task callbacks and notes there. Put the SQLite store on a volume
(`SPRY_DB_PATH` defaults to `/var/lib/clanker-saddle/clanker-saddle.sqlite`)
and mount git checkouts under `/workspace`. On SELinux hosts the bind
mounts need `:Z`, and rootless podman needs `--userns=keep-id` for the
mounts to be writable. For durable sessions mount `web-config.json`
read-only and point `ASTRALIS_CONFIG_PATH` at it; otherwise the app runs
with ephemeral keys.

## Saddles with dev tools (the extendable agent container)

The stock `saddle` image carries the daemon, its CLI and git — nothing
else. Build specialised agents by extending it with the toolchains of
your choosing (see `../saddle`'s README, *Extending the saddle image*,
for the full guide): a two-line Containerfile `FROM` the loaded base
plus `dnf install` of your tools (C, Vala, GTK, Node, .NET, Android,
QEMU — anything), then tag the capability names you gave those tools in
the web UI's saddle page so only tasks requiring them route there.
Goal task worktrees need every container that may run a goal's tasks to
share the `/workspace` volume.

## Creating your first project and task

1. **Project** — `/projects` → name, git URL, default branch. The
   checkout must exist at `$SADDLE_PROJECTS_ROOT/<name>` on the saddle's
   filesystem (clone it into the workspace volume; the agent does not
   clone). Link related projects as *co-developed* (agent may modify) or
   *reference* (read-only) from the project page.
2. **Task** — `/tasks` → project, title, **mandate** (the description),
   mode (implement / investigate / review / test) and priority (1 is
   the highest). The task is created in **draft**: edit every field,
   attach rules and capabilities, then press **Enqueue** (only the
   creator enqueues — humans their own, the goal agent its own).
   Long-form fields edit in place: an **Edit** button beside the task
   page's Mandate card (drafts only) swaps the rendered markdown for a
   prefilled editor; the same pattern on a draft goal's Initial Brief
   card edits title, brief and priority before you start it.
   Chain human work with **auto-enqueue after**: pick a predecessor
   from the global task list; the dependent enqueues itself when the
   predecessor completes (a failed or cancelled predecessor leaves it
   in draft).
3. **Saddle** — `/saddles` (admin) → name plus the daemon's host:port.
   Registration mints a per-saddle token and delivers it to the daemon's
   `POST /register` (with this app's base URL); the daemon then asks
   for work whenever idle-awake and sleeps on a 204 answer, so a ready
   task reaches it within one dispatcher tick (wake → ask → allocate). The saddles page
   renders the daemon's `ssh_public_key` in a dark panel with the text
   colour lifted for readable contrast.

**Goals** (`/goals`) are the orchestrated path: write the Initial Brief
and a branch name, start it, and a goal agent investigates (asking
pre-plan questions liberally), submits its plan for review, fans flat
tasks out through the `POST /api/saddle/task/{create,delete,enqueue}`
callbacks (a per-task worktree per involved repo on an ephemeral
branch), merges finished branches into the goal branch, pushes, and
submits a completion report for your review. Push-style daemons report
through `POST /api/saddle/results` (round entries plus state
transitions) and `POST /api/saddle/chunks` (raw PTY batches) — every
`/api/saddle/*` call is authenticated by the per-saddle bearer token
(removing the saddle revokes it).

## Users, roles and your profile

Two roles decide what a logged-in user can do (the account model is
Spry's; this app surfaces it):

- **user** — manages projects, tasks, goals and rules.
- **admin** — additionally manages **users** (`/users`), **saddles**
  (register/remove, git identity, capability tags) and the
  **capability catalogue**. Viewing `/saddles` stays open to every
  logged-in user; the mutation controls are hidden and server-refused
  for non-admins.

`/users` (admin-only) lists accounts newest-first with a role badge
and enabled state: create users, edit profile fields, reset passwords,
enable/disable, grant/revoke admin, delete. Guards keep you from
locking yourself out: you cannot delete or disable your own account,
cannot revoke your own admin permission, and the last enabled admin
cannot be removed.

Every logged-in user has **`/profile`** (linked beside the logout
button in the nav corner): shows your account details and changes your
own password — current, new, confirm. Change the seeded admin's
`change-me` password there before exposing the app.

## Monitoring agent progress

- **`/tasks`** — live queue: status chips (`draft → queued → assigned →
  working → blocked → done/failed`), mode badges, goal links and the
  standalone/by-goal filter.
- **`/tasks/{id}`** — the control room; it refreshes on every state
  pulse while a saddle works: comments thread (yours and the agent's),
  blocks, goal context, session history, and the agent's final summary.
  Your mutations on this page (and the goal page's) answer with the
  refreshed state slots themselves rather than a navigate or reload —
  the page lands in its post-mutation shape in one round trip.
- **`/tasks/{id}/terminal`** — the full bash session log: the raw PTY
  stream the daemon pushes (chunk batches land in `session_raw_log` and
  stream over SSE byte-exact), with the typed entry view as the fallback
  for legacy sessions. The session picker honours `?session_id=` across
  every run a task ever had.
- **`/goals/{id}`** — the goal control room: lifecycle head, locked
  brief, plan expander with revision history, task strip, the
  supervision narrative and the completion review.
- **`/`** — dashboard: the "needs your attention" pile (goal AND task
  blocks interleaved, newest-first) above the activity timeline, saddle
  cards and queue summary. The Queue card's **queued** figure counts
  waiting-for-attention work: ready-to-dispatch queued tasks, parked
  (suspended) tasks — suspended goal-agent runs included — and tasks
  whose raised blocks are all answered and now wait for a saddle to
  resume them; queued rows gated on an unfinished auto-enqueue
  predecessor stay out (the predecessor itself is already counted).
  **blocked** stays human-answerable rows only.

## Answering block questions

When an agent needs a human decision it runs `saddle block "<question>"`;
the daemon pushes the `blocked` state with the block payload, the task
flips to **blocked** (the pulse bump lands the question on live pages
without a reload) and the saddle parks. Type the answer, submit —
answering IS allocating: the answered task becomes resume-offerable,
the dispatcher wakes the affine saddle, and the daemon's ask returns
the assignment carrying the verdict as its `resume_notice` (the stable
wordings — "Human selected: …", "Plan approved. Proceed with
implementation.", "Review approved. Continue with the task." &c.) with
a **fresh session id**; the parked session closes as `parked`. A goal
agent's blocks ride its RUN TASK's single block row — one answer
resolves it and resumes the run. `saddle plan` parks with the goal's
plan-review block: Approve answers "Plan approved. Proceed with
implementation." and applies the markdown as the goal's plan (documents
land on the owning entity only at acceptance — the pending report lives
in its block alone); the feedback textarea returns the agent to
planning to revise. The completion review renders as the rich attention
block (highlight-to-remark annotations queue onto the rejection);
answering "Review approved. Continue with the task." completes the goal
and stores the report.

Resumed assignments also carry an optional `context_update` (markdown
string): the delta of the instruction context that changed since the
run's last round — the goal's plan approved in the meantime (the
approved markdown rides along), rules created, edited or attached to
the task/its projects, or new project notes. The daemon appends it as
a TURN on resume (prompt-cache friendly) instead of refreshing the
system prompt; the member is absent when nothing changed.

## Force Wake (human-triggered resume)

A parked agent normally resumes only when the machinery owes it a wake
(a block answered, an owned task's terminal transition). When a run
parks with nothing pending — the goal agent suspends itself right
after a typo'd enqueue, say — no wake ever fires and it would sit
suspended forever. **Force wake** is the human lever. On an active
goal whose run task is suspended, the goal page's lifecycle head
offers a **Force wake** button (beside Pause/Resume); on a suspended
plain task the task page offers the same, and on a suspended goal-run
task row it routes to the same goal force-wake path — one lever, no
duplicates. One press sets the flag; the next dispatcher pass makes
the run offerable immediately (no coalesce wait), and the resumed
assignment's `resume_notice` leads with an explicit line — "The user
explicitly force-woke this goal's supervision — you were parked with
no pending wake." — followed by the standard batched report, which
names draft-status tasks as "draft — NOT enqueued" so a parked agent
that typo'd an enqueue sees it immediately. The flag clears on the
wake (one wake per press). Paused goals and non-suspended tasks hide
the button and the API refuses them (400).

## Browser notifications (Web Push)

The app speaks the real Web Push protocol (VAPID + RFC 8291 payload
encryption, no third-party service). The VAPID keypair is generated at
startup on first run and persisted in **`web-config.json`** under the
`vapid` section (`public_key` / `private_key`, base64url P-256) — that
file is the single home for the keys; delete the section to rotate
(regenerate = restart). `SADDLE_VAPID_SUBJECT` overrides the JWT's
`mailto:` contact (default `mailto:clanker-saddle@localhost`).

To enable notifications in a browser: open the dashboard and press
**Enable notifications** — the page requests permission, registers the
service worker (`GET /sw.js`, scope `/`) and subscribes with the app's
VAPID key. "Turn off notifications" reverses it. The bell is
**per-browser**: it compares THIS browser's subscription (by a
SHA-256 fingerprint of its endpoint — the endpoint URL itself never
leaves the server) against your subscribed browsers, so each browser
shows its own state; when you are subscribed elsewhere it says so and
still offers Enable here. **Send test notification** (visible when
this browser is subscribed) pushes a real notification through the
full protocol and reports the true per-endpoint result, so a broken
setup surfaces its actual reason instead of silence. Per task,
**Notify me** / **Don't notify me** (on the task page's status card)
opts you into that task's first start, failure and completion; task
questions, RFCs and review requests notify every subscribed user, and
goal blocks plus goal success/failure ALWAYS notify every subscribed
user. Clicking a notification opens the task's or goal's page.
Delivery is best-effort and truthful: failed sends retry on the
dispatcher tick (up to five attempts — after which the drop is logged
loudly, never silently claimed as delivered), endpoints answering
410/404 are pruned automatically, and other 4xx refusals (e.g. a VAPID
rejection) keep the subscription row and retry, since the endpoint
may recover.

## Rules and project notes

**Rules** (`/rules`) are reusable instructions dispatched with every task
that carries them: create them on the rules page, then attach/detach them
on the project page (they lead every task of the project) and the task
page (they lead the task's own set). A dispatch ships the effective set —
task rules, then the primary project's, then the co-developed projects'
in link order, deduplicated; reference projects contribute nothing.

**Project notes** are written by agents mid-task via `saddle note` and
curated by humans on the project page (delete only — there is no manual
create). Daemons POST notes to this app's
`POST /api/saddle/note` callback, authenticated by the per-saddle bearer
token the daemon persisted at registration (plan 10; the shared
`saddle-callback-…` token is gone). The callback base
URL daemons must dial is **this app's own externally reachable address**
(it differs from the browser's when the daemon runs in a container) and
is configured with the top-level `saddle_callback_base_url` key in
`web-config.json`, falling back to the `SADDLE_CALLBACK_BASE_URL`
environment variable and then `http://localhost:8080`:

json { "statum": { "…": "…" }, "saddle_callback_base_url": "http://clanker-web:8080" }

Dispatches also carry the primary and co-developed projects' notes (up to
30 entries / 20 000 characters, oldest first) so agents start with the
accumulated project knowledge.

## Integration testing

`test/integration_test.py` drives BOTH real binaries — this app and a
real saddle daemon over a real git checkout — through the full goal-era
arc (Statum-style PASS/FAIL): goal create → start → pre-plan question
(one answer resolving the goal-run task's single task_blocks row) →
plan reject-with-feedback → revision → approval on the agreed
verdict words → task fan-out through the real `/api/saddle/task/*`
callbacks → the worker's ephemeral worktree (auto-pruned, branch kept)
→ the coalesced terminal-event resume → merge + branch delete +
push-before-review → completion review accept (gated on the live
per-repo push flags) → goal complete, plus the `/status`
`goal_branches` live probe flags, SSE over the agent's raw terminal,
the DERIVED goal narrative, the push wire surface (typed entries and
block raises riding POST /api/saddle/results, the deleted poll-era
endpoints 404ing) and the stm-markdown renderer. Host run (the harness
prepares a `demo` repo
with a pushable bare origin when the workspace is empty; the database
must be FRESH — the harness refuses a used one):

bash export SPRY_SEED_ADMIN_USERNAME=admin SPRY_SEED_ADMIN_PASSWORD=adminpass123 export SADDLE_LLM_BASE_URL=http://127.0.0.1:9999 SADDLE_LLM_MODEL=mock export SADDLE_PROJECTS_ROOT=/tmp/clanker-e2e/workspace export SPRY_DB_PATH=/tmp/clanker-e2e/web/clanker-saddle.sqlite export SADDLE_GOAL_RESUME_S=2 # shrink the coalescing window for tests ./builddir/clanker-saddle 8180 & # 8180 — 8080 is the production port

SADDLE_WORKSPACE_ROOT=/tmp/clanker-e2e/workspace

../saddle/builddir/src/saddle --port 7100 &

WEB=http://localhost:8180 SADDLE=http://localhost:7100 \ WORKSPACE=/tmp/clanker-e2e/workspace \ DB_PATH=/tmp/clanker-e2e/web/clanker-saddle.sqlite

python3 test/integration_test.py

The e2e callback base URL comes from the mounted `web-config.json`'s
`saddle_callback_base_url` key (`http://localhost:8180` for a host run)
— the config beats the `SADDLE_CALLBACK_BASE_URL` environment
variable, so set it there rather than exporting a second source of
truth. The harness-prepared `demo` repo and its bare origin both live
on branch `main` (the project registration's default branch matches).

The daemon's `SADDLE_WORKSPACE_ROOT` and the app's
`SADDLE_PROJECTS_ROOT` point at the SAME directory (the goal clone the
daemon cuts must anchor the worker worktrees — the shared-volume
deployment assumption). For a container run point `WEB`/`SADDLE` at the
published ports, pass `SADDLE_HOST=clanker-saddle:7100`, the container's
SQLite file as `DB_PATH`, and start the in-container mock via
`MOCK_CMD='podman exec -d clanker-saddle python3
/usr/libexec/saddle/mock_llm.py --port 9999'` (the test appends the
same scripted responses to whatever command `MOCK_CMD` prefixes). The
suite releases a daemon left blocked by an earlier run via `/stop`
before it starts.

`meson test` additionally runs `test/web_stack_test.py` — a
self-contained suite (no daemon needed) that spawns the app in a scratch
directory plus an in-process fake saddle daemon speaking the wire
contract (thirteen scenario bundles — pages, goalui, push, dispatch,
answers, replay, subtasks, lineage, users, forcewake, goals, lifecycle,
allocation), and asserts the web-stack features: plan-review body hiding,
the effective `rules` array of assignments, project notes with the
`/api/saddle/note` callback and the payload's `project_notes`/`callback`
caps, the ask/allocation flip (concurrent asks race one winner, affinity
pinning, the lost-wake guard), the phase-2 telemetry flip (pushed
chunks idempotent and ordered into the terminal view, results pushes
with blocked/suspended/done/failed state transitions, entry-redelivery
idempotency, the one-probe staleness policy with its dead-daemon
requeue, and the retired `/output`//`/log`//`/start` daemon routes 404ing
while the web side never calls them), and the phase-3 capability
awareness + UI (the assignment `capabilities` context,
`GET /api/saddle/capabilities`, task-callback `--capability` validation
with the valid-tag list in the error, the task page's top status block
and newest-first blocking sections, the dashboard's pile and timeline,
and the slow-ask lost-wake regression), and the plan-12 goal era
(lifecycle, plan/completion reviews with the run-task's single block row
and the
agreed verdict wordings, task callbacks, coalesced terminal-event
resumes, pause queueing, cancel cascade, auto-enqueue-after, goal
notifications and the goal context of worker assignments).

## Realtime channels and the browser's SSE budget

Browsers cap ~6 concurrent connections per origin on HTTP/1.1, and SSE
streams hold theirs forever: Statum's realtime channel (`/_statum/channel`,
one per browser via its SharedWorker — one per tab where no SharedWorker
is available) plus one `/terminal-stream` per open terminal tab. Deep
tab-stacking on HTTP/1.1 can starve the pulse channel silently (task
pages then miss live block updates). Mitigations built in: hidden
terminal tabs close their stream until focused, and task pages refresh
their state on focus in addition to each pulse. Serving the app over
HTTP/2 removes the cap entirely.

## Deployment assumptions (goal era)

- **Shared workspace volume across saddles.** A goal's clone workspace
  (`<root>/goal-<id>`, one clone per involved repo on the goal branch)
  anchors every task worktree (`<root>/task-<task-id>`, ephemeral
  branch `task/<goal-id>-<task-id>`). Whichever saddle runs a goal task
  must see the same `SADDLE_WORKSPACE_ROOT` volume as the saddle that
  runs the goal agent — today the fleet is single-saddle per goal by
  construction (one goal agent, one worker at a time on the one
  registered saddle); multi-saddle fleets must mount the same volume
  everywhere before fanning goal work out.
- **Database tear-down-and-recreate.** The schema is a single flattened
  `M0001_Initial` (goals with links/comments, the flattened tasks, and
  the plan-16 deletions already applied: no `api_tokens`, no
  `working_branch`, no saddle-row work cache — blocks, comments and
  narrative ride the run task's own structures). There is NO upgrade
  path from an earlier era's database — delete the SQLite file (or
  point `SPRY_DB_PATH` at a fresh path) and let the app recreate it.
- **Restart both sides.** The daemon (`saddle`) and this app must both
  restart onto the goal-era builds together — the wire contract
  (assignment `kind`/`mode`/`creator`/`goal`/`ephemeral_branch`, the
  task callbacks, goal-keyed blocks, `/status` `goal_branches`) is only
  honoured by matching pairs, and the daemon's registration/token
  persists across its restarts while the app re-seeds on a fresh DB.

## Packaging (USM)

`MANIFEST.usm` + `usm-scripts/` package the app as
`bin:clanker-saddle` for the Web-Stack repository:

bash usm manifest package # → clanker-saddle-.usmc ```

Publishing to a USM repository is a repository-side operation (see the USM tooling; usm repository add from the repository's checkout). web-config.json is never packaged (.usmignore); keys are mounted at runtime, never baked into images.