Statum — Astralis Runtime Integration + `pstm-` Pre-rendering Pipeline

Goal

Wire Statum into Astralis/Inversion as two coupled subsystems:

Subsystem A (runtime): the HTTP endpoints from model.md (entrypoint, slots, channel + channel/{id}, resource/{name}), StateService as a singleton, realtime channel service, action conventions, and a StatumConfigurator.
Subsystem B (pstm- pipeline): a build-time generator that turns app HTML into precompressed Vala page classes (predicate-variant selection, template/fragment composition, guards, resource rewriting) + a resource generator + meson wiring.

Behavioral reference: frontend-demo/server.php (PHP reference backend) + model.md. Pattern reference: Spry (SpryModule, ComponentEndpoint, SseEndpoint, StaticResourceProvider/ConstantStaticResource, spry-mkssr/spry-mkconst tools wired as meson custom_target).

Confirmed Design Decisions (from interview)

No Gee. Use Invercargill.DataStructures everywhere. ⇒ migrate existing DirectiveDto Gee usage (Gee.List/Gee.ArrayList → Lot/Series/Enumerable).
Entrypoint handler = type-based subclass. StatumEntrypoint with async Lot<DirectiveDto> handle(); injected StatumRequest (uri, matched route params, held slots by type_name, query) + StateService + ChannelService. Same StatumRequest/header parsing reused by action handlers.
Held slots exposed by type_name, with decrypted public AND private data, scoped to only what the client sent in X-Statum-Slot headers.
Slot keys are random/unpredictable (session-like). Never app-chosen (auth state lives in slots = trust model). update(key, state) mutates in place keeping the key; client round-trips the slot; StateService is a short-lived cache for the transmit_after window. Server is effectively stateless (slot-as-session).
Realtime push is StateService-driven on mutation. StateService.update(key, state) (and clear_slot) auto-push the resulting frame to any channels subscribed to that key; the push is effectively a no-op when no stream is subscribed. Selective fan-out emerges naturally from subscriptions (a counter with no subscribers → no push; a live slot with subscribers → push), matching server.php without manual fan-out calls. update returns the signed FrameDto so the handler reuses it for the response set directive (one sign → push + response); set-directive signature idempotency dedupes the acting tab if it is also subscribed. ChannelService.push_set/push_clear remain as the low-level mechanism (used internally; still callable for advanced cases).
add_page<TPage, TEntrypoint>() registers BOTH the generated StatumPage (Endpoint for each <pstm-uri> route) AND the entrypoint handler in the framework's internal entrypoint route table (dispatched by /_statum/entrypoint?uri=…). Also add_page<TPage>() (static, no entrypoint) and add_resource<T>(). No standalone add_entrypoint.
pstm- variant selection is pure. Page handler evaluates pstm-if/pstm-else-if/pstm-else/pstm-guard against the request's held slots only (no app/DB logic in the page handler). Entrypoints are separate handler classes that send state for the template's type_names. Flow: template served → JS requests entrypoint → state hydrates. No SSR beyond guards/ifs.
Predicate roots: each type_name binds to its held public data, or null if absent; plus a virtual .private member exposing the decrypted private data. e.g. auth != null && auth.role == "admin" && auth.private.secret_rating > 30. .private is server-only (never sent to client).
pstm-guard has predicate + require (a type_name whose absence fails the guard). Failure → optional redirect (HTTP 302) or the app's globally-registered guard error page at status (default 403).
Template composition: <pstm-name> names this file; <pstm-template> names the parent template to wrap with (by its <pstm-name>); templates chain upward (a template may itself declare <pstm-template>); <pstm-content> is the child outlet; resolved at compile time, not route-based. Title prefix/suffix merge up the chain.
StatumPage base in Subsystem A; generator emits subclasses carrying variant byte arrays + compiled predicate/guard Vala. Author registers each page with one add_page call (routes read from the generated class).

Architecture / Data Flow

Browser GETs <pstm-uri> (e.g. /dashboard) → Astralis router → generated StatumPage Endpoint → parses X-Statum-Slot headers into held slots (public+private) → evaluates guards/pstm-if matrix → serves the matching precompressed variant (encoding negotiation + ETag), or 302/error-page on guard failure.
Page HTML boots Statum JS → JS GETs /_statum/entrypoint?uri=/dashboard → EntrypointEndpoint matches uri in internal table → resolves scoped StatumEntrypoint → handle() returns set/subscribe/… directives (application/vnd.statum+json).
Actions (app-defined URIs) → StatumAction handlers → mutate state via yield StateService.update(key, state) (returns the signed frame; auto-pushes to subscribed channels), build the response set directive from that frame, return directives.
Stale slots → client POSTs frames to /_statum/slots → restore_slot + re-sign → refreshed set directives.
Realtime → GET /_statum/channel (SSE, first event channel: <id>); PATCH /_statum/channel/{id} {subscribe,unsubscribe}; server pushes set/clear events to subscribed channels.

Subsystem A — Runtime (implement & validate first)

A1. Gee → Invercargill migration

src/Model/DirectiveDto.vala: drop using Gee;; change serialize_array(Gee.List<DirectiveDto>)/deserialize_array to Invercargill Lot/Enumerable + Series/Iterate. Update any callers.

A2. `StateService` changes (`src/StateService.vala`)

Register as singleton (done in StatumModule).
Inject ChannelService (singleton; one-way — ChannelService does NOT inject StateService, so no DI cycle).
Add async FrameDto update(string key, State state) throws Error: set slot.current_state = state; bump last_touched; sign (sign_slot logic) → FrameDto; serialize frame → yield ChannelService.push_set(key, frame_json) (no-op if no subscribers); return the frame (handler reuses it for the response set directive).
Add async void clear_slot(string key): remove slot; yield ChannelService.push_clear(key) (no-op if no subscribers).
Make get_slot a pure read (stop mutating last_touched there — liveness is now driven by transmit_after/invalid_after and explicit update).
Fix cleanup(): currently removes recently-touched slots (last_touched > cutoff is backwards). Evict slots whose transmit_after/invalid_after has passed (client will resend the frame via /_statum/slots).

A3. Held-slot resolution service (new)

HeldSlot record: key, Scope, public (Properties), private (Properties, decrypted), as_at, transmit_after?, invalid_after?.
HeldSlotResolver (singleton): parses each X-Statum-Slot header:
- Reference form key=…; as_at=… → StateService.get_slot(key) → build HeldSlot from current_state (skip if absent/expired).
- Base64 frame form → Base64.decode → JSON {content,signer,signature} → SigningProvider.verify_frame → parse content as SnapshotDto → EncryptionProvider.read_properties(slot.key, snapshot.private) → build HeldSlot.
- Index result by snapshot.slot.slot_type.
Returns ReadOnlyAssociative<string, HeldSlot> (type_name → held).

A4. `StatumRequest` + handler base classes (new)

StatumRequest: uri, route_params (ReadOnlyAssociative<string,string>), held (type_name → HeldSlot), query (Catalogue<string,string>).
StatumEntrypoint (abstract): injected StatumRequest, StateService, ChannelService; abstract async Lot<DirectiveDto> handle().
StatumAction (abstract, for app action endpoints): same + decrypts X-Statum-Private via EncryptionProvider.read_properties("action-private", blob) → action-private Properties; parses form (FormDataParser) + query; abstract async Lot<DirectiveDto> handle().
Directive-response helper: StatumDirectives.to_result(Lot<DirectiveDto>) → HttpStringResult(json) with Content-Type: application/vnd.statum+json + Cache-Control: no-store.

A5. Framework endpoints (new, registered in `StatumModule`)

EntrypointEndpoint (GET /_statum/entrypoint, scoped): read uri query; match against EntrypointRouteTable (singleton list of (EndpointRoute, Type)) using EndpointRoute-style segment matching; build StatumRequest (register scoped); resolve + handle(); return directive result. 404 if no match.
SlotPostEndpoint (POST /_statum/slots, scoped): read JSON array of frames; per frame verify_frame + restore_slot; re-sign (sign_slot) → set directive; return directives (matches server.php refresh).
ChannelEndpoint (GET /_statum/channel, singleton, extends SseEndpoint): new_connection generates random channel id, sends event: channel\ndata: <id>, tracks id→SseStream + id→Set<string> subs; exposes push_set(key, frame_json) / push_clear(key) sending event: set/event: clear to subscribed streams; retry_interval ≈ 3000.
ChannelSubscriptionEndpoint (PATCH /_statum/channel/{id}, scoped): inject ChannelEndpoint; parse {subscribe:[], unsubscribe:[]}; update subs; on each newly-subscribed key, if StateService has the slot cached, push its current frame (get_slot+sign_slot) via push_set; return HttpEmptyResult(204).
ResourceEndpoint (GET /_statum/resource/{name}, scoped): mirror Spry StaticResourceProvider — inject_all<StatumResource>; pick best encoding by Accept-Encoding; ETag/If-None-Match; serve. (Pin singular vs plural path: model.md/demo use /_statum/resource/{name}; user example said resources. Default to resource.)

A6. Resource types (new)

StatumResource interface (mirror Spry StaticResource): name, get_best_encoding, get_etag_for, to_result.
ConstantStatumResource base (mirror Spry): name/file_hash/content_type, get_best_encoding, get_encoding, byte-array consts.

A7. `StatumModule` + `StatumConfigurator` (`src/Statum.vala`)

StatumModule.register_components: singletons StateService, SigningProvider, EncryptionProvider, ChannelEndpoint (as ChannelService), EntrypointRouteTable, HeldSlotResolver; framework endpoints A5 with EndpointRoutes (ChannelEndpoint as singleton endpoint; rest scoped).
StatumConfigurator (like SpryConfigurator):
- add_page<TPage>() / add_page<TPage,TEntrypoint>(): read TPage's <pstm-uri> route(s); register TPage as scoped Endpoint per route; if TEntrypoint, add (route, typeof(TEntrypoint)) to EntrypointRouteTable.
- add_resource<T>(): register T as StatumResource.

A8. Validate Subsystem A

Port frontend-demo/server.php semantics to a Vala example (counter/products/cart/user/notice + live broadcast) exercising entrypoint, slots pre-flight, channel subscribe/push, actions, X-Statum-Private. Confirm against js/statum.js + js/statum-worker.js.

Subsystem B — `pstm-` Compile-time Pipeline

B1. `statum-mkpstm` generator tool (new, Vala executable)

Deps: astralis, invercargill, invercargill-json, libxml2, inversion (uses Astralis compressors + Invercargill expressions + libxml2 HTML parsing).
Input: app HTML files — pages (have <pstm-uri>), templates (<pstm-name> + <pstm-content>, no uri), fragments (<pstm-name> + optional <pstm-input>s, no uri).
Output: one Vala StatumPage subclass per page.

B2. Composition (compile-time)

Resolve <pstm-template> chain upward (page → template → …); inline child body into each ancestor's <pstm-content> outlet.
Apply pstm-materialise-as (default div) when embedding a fragment/template-child <body>.
Inline <pstm-fragment name="…"> includes; map attributes matching the fragment's <pstm-input>s to stm-def-<x>-as attributes on the materialised element.
Compose <pstm-title prefix|suffix> up the chain → final <title>.
Rewrite pstm-res-<x>="name" → <x>="/_statum/resource/<name>".
Strip all pstm-* directives/elements from emitted HTML (they must not reach the browser — see model.md pstm- warning).

B3. Predicate variant matrix + codegen

Collect pstm-if/pstm-else-if/pstm-else chains; enumerate the cross-product ⇒ one rendered HTML variant per combination (each branch kept, others removed).
For each variant: fully compose (B2), then compress with identity/gzip(9)/zstd(19)/brotli(11); store as IDENTITY/GZIP/ZSTD/BR byte arrays.
Emit compiled Vala: a select_variant(held) that (a) evaluates pstm-guards (presence require + predicate → GuardFailure(redirect|status)), then (b) evaluates the pstm-if matrix against the held context → variant index → returns that variant's bytes.
Predicate codegen: parse each predicate with Invercargill ExpressionParser (AST) and emit Vala reading the held context — type_name root → held public Properties or null; .private → held private Properties (virtual); member access → Properties lookup w/ coercion; != null → presence; ==/!=/>/etc., &&/||/!, literals.
- Risk: requires the Invercargill expression AST to be walkable for source emission. Fallback: emit Vala that calls the runtime evaluator (ExpressionParser.parse(str).evaluate(ctx)) — functional but not "compiled ifs". Decide during B3 based on AST introspectability; document the supported predicate subset either way (root type access, .private, null checks, member access, comparisons, logical ops, literals).

B4. Generated `StatumPage` subclass shape

Extends StatumPage (Subsystem A) which provides header parsing → held slots (public+private), encoding negotiation, ETag, guard redirect/error-page serving.
Subclass provides: variant byte arrays (B3), compiled select_variant, and a static list of its <pstm-uri> routes (for add_page).

B5. `statum-mkres` resource generator (new, mirror `spry-mkssr`)

File → Vala ConstantStatumResource subclass with IDENTITY/GZIP/ZSTD/BR byte arrays + get_best_encoding/get_encoding/hash/content-type.

B6. Meson wiring

tools/meson.build: build statum-mkpstm + statum-mkres executables (install).
App meson.build: custom_target per page HTML → generated .vala; custom_target per resource via statum-mkres; include generated sources in the app executable. (Mirror Spry demo/meson.build docs_css_resource pattern.)

B7. Validate Subsystem B

Example app: a homepage with pstm-if="auth != null" (signup CTA vs dashboard variants), a pstm-guard require="auth" protected page (redirect on failure), a template chain (<pstm-template>), a fragment with <pstm-input>, and a pstm-res-src image. Verify: correct variant served by held state, hidden branches absent from response source, guard redirect, resource served compressed.

Cross-cutting

Storage & crypto keys (v1 scope)

In-memory StateService store; ephemeral SigningProvider/EncryptionProvider keys (random per process). ⇒ all outstanding slots/sessions invalidate on restart. SQL persistence (invercargill-sql/sqlite3 deps already wired) + key persistence = out of scope / future.

Naming consistency

Mirror stm-* for pstm-* control flow: pstm-if, pstm-else-if, pstm-else (user wrote pstm-if-else once; align to stm-else-if).

Risks / Open Items

Predicate→Vala codegen depends on Invercargill expression AST walkability (B3); runtime-eval fallback documented.
Channel SSE concurrency: SseEndpoint base broadcasts to all streams; per-channel subscription filtering must be implemented in the ChannelEndpoint subclass (iterate get_open_streams(), filter by subs, send_event). Validate send failures close streams.
Singleton endpoint injecting scoped services: Channel/Resource/Entrypoint endpoints must inject only singletons (StateService, ChannelService, HeldSlotResolver, EntrypointRouteTable) — never scoped Scope/HttpContext at construction (those are per-request). StatumRequest is built per-request and registered scoped inside the endpoint.
Resource path singular vs plural (default /_statum/resource/{name}).
<pstm-uri> templated routes (e.g. /cats/{breed}/info) must register identically for both the page Endpoint and the entrypoint table so uri matching agrees.

Validation (overall)

Subsystem A: server.php-equivalent Vala example passes against the real JS client (entrypoint hydration, slots pre-flight after transmit_after, channel live push across tabs, action X-Statum-Private, directive content-type).
Subsystem B: variant-selection/guard/template-chain/fragment/resource example produces correct, source-minimized, precompressed responses.
Build: meson compile succeeds with generated sources; tools install.

phase-1-plan.md 17 KB Histórico Raw

Statum — Astralis Runtime Integration + pstm- Pre-rendering Pipeline