integration: README — goal-era workflows, both-binaries harness, deployment assumptions

First-project/goal/task walkthrough rewritten for the flattened era
(goals, mandates/modes, enqueue-after; subtask machinery and the
plan-11 options gone), monitoring sections cover the goal control room
and the dashboard pile, block answering documents the dual-row goal
blocks with the agreed verdict wordings, the integration section
documents the 83-check both-real-binaries run (fresh DB, pushable
origin, shared workspace root, SADDLE_CALLBACK_BASE_URL +
SADDLE_GOAL_RESUME_S), and a new Deployment assumptions section pins
the shared /workspace volume across saddles, the DB
tear-down-and-recreate (flattened M0001) and the restart-both-sides
rule.

README.md

@@ -16,7 +16,7 @@ Browser ←→ clanker-saddle (Spry app: SQLite queue, Statum UI, dispatcher)
                any OpenAI-compatible chat-completions endpoint
 ```
 
-The web app owns projects, tasks, dependencies and the saddle registry.
+The web app owns projects, goals, flat tasks and the saddle registry.
 Work acquisition is ask-based (plan 10): daemons sleep when idle and ask
 `POST /api/saddle/next-task` (per-saddle bearer token); the allocation is
 one write transaction that pairs the asking saddle with the best
@@ -111,6 +111,7 @@ podman run -d --name clanker-web --userns=keep-id --network clanker-net \
     -e SPRY_SEED_ADMIN_USERNAME=admin -e SPRY_SEED_ADMIN_PASSWORD=change-me \
     -e SADDLE_LLM_BASE_URL=http://127.0.0.1:9999 \
     -e SADDLE_PROJECTS_ROOT=/workspace \
+    -e SADDLE_CALLBACK_BASE_URL=http://clanker-web:8080 \
     localhost/clanker-saddle:0.1
 podman run -d --name clanker-saddle --userns=keep-id --network clanker-net \
     -p 7100:7100 -v "$PWD/workspace:/workspace:Z" \
@@ -121,7 +122,10 @@ Register the saddle with its **network** address (`clanker-saddle:7100`)
 — the web app talks to the daemon container-to-container, while the LLM
 base URL is dialled by the daemon itself (a mock can run inside the
 saddle container via `podman exec -d clanker-saddle …`, reachable at
-`http://127.0.0.1:9999`). Put the SQLite store on a volume
+`http://127.0.0.1:9999`). `SADDLE_CALLBACK_BASE_URL` (or the
+`web-config.json` `saddle_callback_base_url` key) must be the app's
+address **as the daemon dials it** — every registered daemon pushes
+telemetry, task callbacks and notes there. Put the SQLite store on a volume
 (`SPRY_DB_PATH` defaults to `/var/lib/clanker-saddle/clanker-saddle.sqlite`)
 and mount git checkouts under `/workspace`. On SELinux hosts the bind
 mounts need `:Z`, and rootless podman needs `--userns=keep-id` for the
@@ -163,15 +167,22 @@ revokes it).
 ## Monitoring agent progress
 
 - **`/tasks`** — live queue: status chips (`draft → queued → assigned →
-  working → blocked → done/failed`), dependency states and ready flags.
+  working → blocked → done/failed`), mode badges, goal links and the
+  standalone/by-goal filter.
 - **`/tasks/{id}`** — the control room; it refreshes on every state
   pulse while a saddle works: comments thread (yours and the agent's),
-  blocks, subtasks, session history, and the agent's final summary.
+  blocks, goal context, session history, and the agent's final summary.
 - **`/tasks/{id}/terminal`** — the full bash session log: the raw PTY
   stream the daemon pushes (chunk batches land in `session_raw_log` and
   stream over SSE byte-exact), with the typed entry view as the fallback
-  for legacy sessions.
-- **`/`** — dashboard: saddle cards, queue summary, recent activity.
+  for legacy sessions. The session picker honours `?session_id=` across
+  every run a task ever had.
+- **`/goals/{id}`** — the goal control room: lifecycle head, locked
+  brief, plan expander with revision history, task strip, the
+  supervision narrative and the completion review.
+- **`/`** — dashboard: the "needs your attention" pile (goal AND task
+  blocks interleaved, newest-first) above the activity timeline, saddle
+  cards and queue summary.
 
 ## Answering block questions
 
@@ -184,12 +195,14 @@ the dispatcher wakes the affine saddle, and the daemon's ask returns
 the assignment carrying the verdict as its `resume_notice` (the stable
 wordings — "Human selected: …", "Plan approved. Proceed with
 implementation.", "Review approved. Continue with the task." &c.) with
-a **fresh session id**; the parked session closes as `resumed`. `saddle
-plan` parks the same way with a type=`plan` block ("Plan submitted for
-review"): Accept applies the markdown to the task's plan, Reject
-returns "Plan feedback received:\n…" — and the per-task auto-approve
-option (or an ancestor's subtask auto-approve toggle) answers plan
-blocks instantly web-side.
+a **fresh session id**; the parked session closes as `resumed`. A goal
+agent's blocks attach to the GOAL (dual-rowed: the goal block plus a
+quiet mirror on the run task) — one answer resolves both and resumes
+the run. `saddle plan` parks with the goal's plan-review block:
+Approve answers "Plan approved. Proceed with implementation." and
+applies the markdown as the goal's plan; the feedback textarea returns
+the agent to planning to revise. The completion review answers "Review
+approved. Continue with the task." to complete the goal.
 
 ## Browser notifications (Web Push)
 
@@ -206,9 +219,10 @@ To enable notifications in a browser: open the dashboard and press
 service worker (`GET /sw.js`, scope `/`) and subscribes with the app's
 VAPID key. "Turn off notifications" reverses it. Per task, **Notify
 me** / **Don't notify me** (on the task page's status card) opts you
-into that task's first start, failure and completion; questions, RFCs,
-review requests and the suspended-waiting-on-drafts condition notify
-every subscribed user. Clicking a notification opens the task's page.
+into that task's first start, failure and completion; task questions,
+RFCs and review requests notify every subscribed user, and goal blocks
+plus goal success/failure ALWAYS notify every subscribed user. Clicking
+a notification opens the task's or goal's page.
 Delivery is best-effort: failed sends retry on the dispatcher tick and
 endpoints answering 410/404 are pruned automatically.
 
@@ -246,19 +260,32 @@ accumulated project knowledge.
 
 ## Integration testing
 
-`test/integration_test.py` drives the whole loop against a running
-app + daemon pair (88 checks, Statum-style PASS/FAIL): explore task,
-dependency gating, then the Part 2 feature tour — `saddle plan`,
-`saddle query` with piped stdin, progress updates, a multiple-choice
-block answered through the web UI, an RFC block with a markdown body,
-the `saddle review` banner approved via the UI, a quoted user comment
-delivered over /notify, terminal SSE (replay + live), the bearer-token
-ingest API and the stm-markdown renderer. Host run:
+`test/integration_test.py` drives BOTH real binaries — this app and a
+real saddle daemon over a real git checkout — through the full goal-era
+arc (83 checks, Statum-style PASS/FAIL): goal create → start → pre-plan
+question (one answer resolving the goal row AND the mirrored run-task
+row) → plan reject-with-feedback → revision → approval on the agreed
+verdict words → task fan-out through the real `/api/saddle/task/*`
+callbacks → the worker's ephemeral worktree (auto-pruned, branch kept)
+→ the coalesced terminal-event resume → merge + branch delete +
+push-before-review → completion review accept → goal complete, plus the
+`/status` `goal_branches` cache, SSE over the agent's raw terminal, the
+`goal_events` narrative, the legacy ingest API surface and the
+stm-markdown renderer. Host run (the harness prepares a `demo` repo
+with a pushable bare origin when the workspace is empty; the database
+must be FRESH — the harness refuses a used one):
 
 ```bash
-mkdir -p /tmp/clanker-e2e/workspace && cd /tmp/clanker-e2e/workspace \
-    && git init demo && cd demo && git commit --allow-empty -m one \
-    && git commit --allow-empty -m two
+export SPRY_SEED_ADMIN_USERNAME=admin SPRY_SEED_ADMIN_PASSWORD=adminpass123
+export SADDLE_LLM_BASE_URL=http://127.0.0.1:9999 SADDLE_LLM_MODEL=mock
+export SADDLE_PROJECTS_ROOT=/tmp/clanker-e2e/workspace
+export SADDLE_CALLBACK_BASE_URL=http://localhost:8080
+export SPRY_DB_PATH=/tmp/clanker-e2e/web/clanker-saddle.sqlite
+export SADDLE_GOAL_RESUME_S=2   # shrink the coalescing window for tests
+./builddir/clanker-saddle 8080 &
+
+SADDLE_WORKSPACE_ROOT=/tmp/clanker-e2e/workspace \
+    ../saddle/builddir/src/saddle --port 7100 &
 
 WEB=http://localhost:8080 SADDLE=http://localhost:7100 \
 WORKSPACE=/tmp/clanker-e2e/workspace \
@@ -266,13 +293,17 @@ DB_PATH=/tmp/clanker-e2e/web/clanker-saddle.sqlite \
     python3 test/integration_test.py
 ```
 
-For a container run point `WEB`/`SADDLE` at the published ports, pass
-`SADDLE_HOST=clanker-saddle:7100`, the container's SQLite file as
-`DB_PATH`, and start the in-container mock via `MOCK_CMD='podman exec -d
-clanker-saddle python3 /usr/libexec/saddle/mock_llm.py --port 9999'`
-(the test appends the same scripted responses to whatever command
-`MOCK_CMD` prefixes). The suite releases a daemon left blocked by an
-earlier run via `/stop` before it starts.
+The daemon's `SADDLE_WORKSPACE_ROOT` and the app's
+`SADDLE_PROJECTS_ROOT` point at the SAME directory (the goal clone the
+daemon cuts must anchor the worker worktrees — the shared-volume
+deployment assumption). For a container run point `WEB`/`SADDLE` at the
+published ports, pass `SADDLE_HOST=clanker-saddle:7100`, the container's
+SQLite file as `DB_PATH`, and start the in-container mock via
+`MOCK_CMD='podman exec -d clanker-saddle python3
+/usr/libexec/saddle/mock_llm.py --port 9999'` (the test appends the
+same scripted responses to whatever command `MOCK_CMD` prefixes). The
+suite releases a daemon left blocked by an earlier run via `/stop`
+before it starts.
 
 `meson test` additionally runs `test/web_stack_test.py` — a
 self-contained suite (no daemon needed) that spawns the app in a scratch
@@ -288,11 +319,14 @@ idempotency, the one-probe staleness policy with its dead-daemon
 requeue, and the retired `/output`//`/log`//`/start` daemon routes 404ing
 while the web side never calls them), and the phase-3 capability
 awareness + UI (the assignment `capabilities` context,
-`GET /api/saddle/capabilities`, subtask `--capability` validation with
-the valid-tag list in the error, the task page's top status block and
-newest-first blocking sections, the dashboard's pinned link-through
-cards above the activity timeline, and the slow-ask lost-wake
-regression).
+`GET /api/saddle/capabilities`, task-callback `--capability` validation
+with the valid-tag list in the error, the task page's top status block
+and newest-first blocking sections, the dashboard's pile and timeline,
+and the slow-ask lost-wake regression), and the plan-12 goal era
+(lifecycle, plan/completion reviews with dual-rowed goal blocks and the
+agreed verdict wordings, task callbacks, coalesced terminal-event
+resumes, pause queueing, cancel cascade, auto-enqueue-after, goal
+notifications and the goal context of worker assignments).
 
 ## Realtime channels and the browser's SSE budget
 
@@ -306,6 +340,29 @@ terminal tabs close their stream until focused, and task pages refresh
 their state on focus in addition to each pulse. Serving the app over
 HTTP/2 removes the cap entirely.
 
+## Deployment assumptions (goal era)
+
+- **Shared workspace volume across saddles.** A goal's clone workspace
+  (`<root>/goal-<id>`, one clone per involved repo on the goal branch)
+  anchors every task worktree (`<root>/task-<task-id>`, ephemeral
+  branch `task/<goal-id>-<task-id>`). Whichever saddle runs a goal task
+  must see the same `SADDLE_WORKSPACE_ROOT` volume as the saddle that
+  runs the goal agent — today the fleet is single-saddle per goal by
+  construction (one goal agent, one worker at a time on the one
+  registered saddle); multi-saddle fleets must mount the same volume
+  everywhere before fanning goal work out.
+- **Database tear-down-and-recreate.** The plan-12 migrations flatten
+  the schema again (a fresh `M0001_Initial`: goals, goal links/comments/
+  blocks/events, the flattened tasks). There is NO upgrade path from a
+  plan-11 database — delete the SQLite file (or point `SPRY_DB_PATH` at
+  a fresh path) and let the app recreate it.
+- **Restart both sides.** The daemon (`saddle`) and this app must both
+  restart onto the goal-era builds together — the wire contract
+  (assignment `kind`/`mode`/`creator`/`goal`/`ephemeral_branch`, the
+  task callbacks, goal-keyed blocks, `/status` `goal_branches`) is only
+  honoured by matching pairs, and the daemon's registration/token
+  persists across its restarts while the app re-seeds on a fresh DB.
+
 ## Deployment (USM)
 
 `MANIFEST.usm` + `usm-scripts/` package the app as `bin:clanker-saddle`