|
|
@@ -0,0 +1,328 @@
|
|
|
+# Statum issues — fix in this repository
|
|
|
+
|
|
|
+Friction points from the Percipio migration that should be addressed **directly in
|
|
|
+Statum** (client JS, `GObjectMapping`, action/handler base classes, `DirectiveBuilder`,
|
|
|
+`TopicSystem`, crypto providers, key config, and docs). Items that are actually
|
|
|
+upstream `invercargill-json` bugs (numeric/bool serialization crashes) are in
|
|
|
+`invercargill-upstream-issues.md`; this file references them only where a Statum-side
|
|
|
+mitigation is also worth doing. Item numbers in parentheses refer to the original
|
|
|
+Percipio friction report.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Already tracked (don't drop)
|
|
|
+
|
|
|
+### S-0a. `stm-data-form` — let a trigger reference a sibling `<form>`'s inputs
|
|
|
+- **What:** `statum.js` `collectData` only reads form inputs when the trigger
|
|
|
+ *is* a `<form>`, so a co-equal "Save draft" `<button>` sent an empty body.
|
|
|
+- **Fix:** an attribute letting an action element (e.g. a button) reference a
|
|
|
+ `<form>` whose named inputs to send with the action. (Already covered by
|
|
|
+ another report — keep it on the list.)
|
|
|
+
|
|
|
+### S-0b. `stm-selected` — set the selected value on a `<select>`
|
|
|
+- **What:** `stm-attribute.value` on a `<select>` only calls
|
|
|
+ `setAttribute('value', …)`, which does not change the selected option.
|
|
|
+- **Fix:** a `stm-selected` attribute (or make `stm-attribute.value` on a
|
|
|
+ `<select>` set the selected option). Worked around today with per-option
|
|
|
+ `stm-attribute.selected="q.skill == '…'"`. (Already covered — keep it.)
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Client / JS side (`js/statum.js`)
|
|
|
+
|
|
|
+### S-1. Slot names that shadow JS globals silently break `stm-if` → blank page — SEVERE (orig #1)
|
|
|
+- **What:** naming a slot `eval` made `stm-if="… eval == null …"` evaluate
|
|
|
+ wrong. Inside `with(scope){ … }`, an identifier not in scope resolves to the
|
|
|
+ **global** (`eval` is the built-in), so `eval == null` was always `false`.
|
|
|
+ Result: fully blank page, no console error.
|
|
|
+- **Impact:** hours of debugging with zero diagnostics.
|
|
|
+- **Fix:** reserve/validate slot type names against JS globals & reserved words
|
|
|
+ at registration (warn or reject), **and/or** make the expression evaluator
|
|
|
+ resolve unknown identifiers to `undefined` instead of falling through to
|
|
|
+ `window` (this is the same evaluator change as S-2).
|
|
|
+
|
|
|
+### S-2. Referencing an *unset* slot makes the whole expression silently falsy — SEVERE (orig #2)
|
|
|
+- **What:** `evalExpr` is
|
|
|
+ `new Function('scope','with(scope){try{return (EXPR);}catch(e){return undefined;}}')`.
|
|
|
+ If `EXPR` mentions a slot that isn't held, the reference throws
|
|
|
+ `ReferenceError`, the inner `catch` returns `undefined`, so `stm-if` is falsy
|
|
|
+ and `stm-text` is empty — silently. `unsetSlot == null` does **not** evaluate
|
|
|
+ to `true` (as plain JS `undefined == null` would), because the *entire*
|
|
|
+ expression is swallowed by the catch.
|
|
|
+- **Impact:** the challenge page was blank because stage-A's `stm-if` mentioned
|
|
|
+ `evaluation` (unset for a fresh attempt). Very hard to spot — no error.
|
|
|
+- **Fix:** use an evaluator that resolves missing scope members to `undefined`
|
|
|
+ (not throw), so `unsetSlot == null` → `undefined == null` → `true`. Only
|
|
|
+ property access on a truly-`undefined` value should be caught. Pair with the
|
|
|
+ debug mode in S-16.
|
|
|
+
|
|
|
+### S-3. `[hidden]` is defeated by ordinary author CSS — SEVERE, sneaky (orig #3)
|
|
|
+- **What:** Statum toggles `stm-if`/`stm-for` branches with the `hidden`
|
|
|
+ attribute, but the UA `[hidden]{display:none}` is lower specificity than a
|
|
|
+ normal rule like `.challenge-list li{display:flex}`, so the hidden loop
|
|
|
+ *holder* leaked as a persistent empty row.
|
|
|
+- **Impact:** "phantom empty entry" with no clue it was a CSS-specificity issue.
|
|
|
+- **Fix:** inject `[hidden]{display:none !important}` from `statum.js` itself
|
|
|
+ (so it can't be overridden), or switch to a dedicated `stm-hidden` class with
|
|
|
+ `!important`. At minimum, document this prominently.
|
|
|
+
|
|
|
+### S-4. `stm-text` on an editable `<input>`/`<textarea>` resets the user's typing (orig #4)
|
|
|
+- **What:** binding a form field's value with `stm-text` is one-way state→DOM;
|
|
|
+ on any re-evaluation (e.g. after a co-equal action's `notify`) it wipes what
|
|
|
+ the user just typed, resetting it to the stale state value.
|
|
|
+- **Impact:** "Save draft clears my answers" until worked around by echoing the
|
|
|
+ saved values back via an `update`.
|
|
|
+- **Fix:** an "initial-value / set-once" binding (e.g. `stm-value`) that sets
|
|
|
+ the value when first bound and then leaves it alone while the field is
|
|
|
+ editable (or focused), and/or document the editable-field pattern.
|
|
|
+
|
|
|
+### S-5. `collectData` ignores the enclosing form for non-`<form>` triggers (orig #5)
|
|
|
+- **What:** a `<button stm-action=…>` inside a `<form>` collects only its own
|
|
|
+ `stm-data-*` attributes, not the form's named inputs.
|
|
|
+- **Impact:** multiple data-collecting actions on one form are impossible today
|
|
|
+ without a hidden `intent` field + `onclick` hack.
|
|
|
+- **Fix:** resolved by S-0a (`stm-data-form`).
|
|
|
+
|
|
|
+### S-6. No first-class spinner/busy binding (orig #6)
|
|
|
+- **What:** `setBusyState` adds `stm-busy-class` classes to the action element
|
|
|
+ and disables children, but showing a spinner requires hand-rolled
|
|
|
+ `<span class="spinner">` + CSS keyed off `.btn-loading`.
|
|
|
+- **Fix:** ship a built-in `stm-spinner` (inject a spinner while busy).
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Server / Vala side (Statum)
|
|
|
+
|
|
|
+### S-7. `GObjectMapping` cannot serialise array/list/collection fields — FOUNDATIONAL (orig #10)
|
|
|
+> **FIXED (2026-08-31, WP1 server-side effort).** `GObjectMapping.to_properties`
|
|
|
+> now serialises `string[]` properties as JSON arrays (read-back supported via
|
|
|
+> `from_properties`) and `Object`-typed properties holding an Invercargill
|
|
|
+> collection (`Series<T>`/`Lot<T>`/`Vector<T>`) as JSON arrays — registered
|
|
|
+> `Object` elements map via the existing mapper registry, fundamental elements
|
|
|
+> as primitives. Note: `int[]`/`int64[]`/`double[]`/`bool[]`/`Object[]` cannot
|
|
|
+> exist as GObject properties at all on Vala 0.56 (the compiler drops them), so
|
|
|
+> `string[]` + typed collections are the supported shapes. Collection read-back
|
|
|
+> is write-side only (element types are erased from the property GType) —
|
|
|
+> documented prominently in the `to_properties`/`from_properties` valadoc.
|
|
|
+> `update_held` now carries such array fields forward instead of dropping them
|
|
|
+> (see S-14).
|
|
|
+
|
|
|
+- **What:** `Object[]` / `Lot<T>` / `Gee.List` properties are silently skipped
|
|
|
+ by `to_properties` (`src/GObjectMapping.vala:80-83` skips unregistered
|
|
|
+ `Object`-typed properties; collections become non-primitives). **Every**
|
|
|
+ list-bearing slot must be hand-built as a `PropertyDictionary` with a
|
|
|
+ `JsonElement` array, and `set_typed<T>` / `update_held<T>` are unusable for any
|
|
|
+ model containing a list.
|
|
|
+- **Impact:** the dominant source of boilerplate for a real (list-heavy) app;
|
|
|
+ wrote a custom `StateJson.list<T>(lot)` helper; `update_held` effectively dead.
|
|
|
+- **Fix:** support list/collection fields in `GObjectMapping` (emit JSON arrays),
|
|
|
+ and/or ship a built-in `State.json_array<T>(lot)` / `State.nested(model)`
|
|
|
+ helper so the "list of row models" pattern is one call instead of ~10 lines of
|
|
|
+ `PropertyDictionary` plumbing. (Independent of the upstream numeric/bool bugs
|
|
|
+ — Invercargill can already build array `JsonElement`s.)
|
|
|
+
|
|
|
+### S-8. `TypedStatumAction<TPrivate>.request_private` is broken (`typeof(TPrivate)` → `void`) (orig #11)
|
|
|
+> **FIXED (2026-08-31, WP1 server-side effort).** Root cause confirmed: valac
|
|
|
+> carries generic type arguments as hidden construct properties, so actions
|
|
|
+> instantiated reflectively (`scope.resolve_type` — the normal path) get
|
|
|
+> `G_TYPE_NONE` for `typeof(TPrivate)`; the erased `(TPrivate)` transfer also
|
|
|
+> dropped the only reference to the materialised model. Fix:
|
|
|
+> `TypedStatumAction` now has a `protected virtual Type private_type`
|
|
|
+> (overridden in subclasses to name the private model — the example's
|
|
|
+> `BumpCounterAction` shows the pattern), an eroded type logs a warning naming
|
|
|
+> the action type instead of crashing, and the model is held via a plain
|
|
|
+> `Object` field so the reference survives the erased-generic return. Verified
|
|
|
+> by a cross-namespace reproduction in `tests/TestMain.vala`.
|
|
|
+
|
|
|
+- **What:** `request_private` calls
|
|
|
+ `GObjectMapping.from_properties(typeof(TPrivate), …)`
|
|
|
+ (`src/StatumHandlers.vala:196`), but at runtime `typeof(TPrivate)` resolves to
|
|
|
+ `void` (Vala generic erasure) → `g_object_new_with_properties: assertion
|
|
|
+ G_TYPE_IS_OBJECT failed` → private data maps to `null`. Had to bypass it and
|
|
|
+ read `request.action_private` manually.
|
|
|
+- **Impact:** every typed action's `request_private` silently returned `null`.
|
|
|
+- **Fix:** work around the Vala `typeof(T)`-in-generics issue — pass the GType
|
|
|
+ explicitly from the subclass (e.g. an `abstract Type private_type` override)
|
|
|
+ or capture it another way — so `request_private` works as documented. (The
|
|
|
+ example's counter action apparently works — worth diffing why; possibly
|
|
|
+ same-compile-unit vs cross-namespace type registration.)
|
|
|
+
|
|
|
+### S-9. Action-private decryption failures are silent (orig #12)
|
|
|
+> **Partially FIXED (2026-08-31, WP1 server-side effort):** decryption failures
|
|
|
+> now log a `warning()` naming the seal namespace and action type, and
|
|
|
+> `request_private` logs when the blob cannot be materialised. The
|
|
|
+> `requires_valid_private` default for plain actions is unchanged (still
|
|
|
+> `false`).
|
|
|
+
|
|
|
+- **What:** `decrypt_action_private` (`src/StatumHandlers.vala:151-164`) returns
|
|
|
+ an empty `PropertyDictionary` on any failure instead of throwing/logging, and
|
|
|
+ `requires_valid_private` defaults to `false`
|
|
|
+ (`src/StatumHandlers.vala:86`), so a mis-sealed action just runs with an empty
|
|
|
+ context (`attempt_id == 0`).
|
|
|
+- **Impact:** "retry button does nothing / bad_private" bugs are invisible.
|
|
|
+- **Fix:** default `requires_valid_private` to `true` for plain actions too (it
|
|
|
+ already is for `TypedStatumAction`, `StatumHandlers.vala:185`), and **log**
|
|
|
+ decryption failures with the namespace so mismatches are diagnosable.
|
|
|
+
|
|
|
+### S-10. The topic system can't push **list** payloads (orig #13)
|
|
|
+- **What:** a topic's `StateModifier.apply` (`src/TopicSystem.vala:111-143`)
|
|
|
+ serialises the derived state via `GObjectMapping.to_properties`
|
|
|
+ (`src/TopicSystem.vala:136`), which can only carry scalar/registered-object
|
|
|
+ state — not a per-question results list. Fell back to a manual
|
|
|
+ `state_service.update(key, …)` with a stashed slot key on the background worker.
|
|
|
+- **Impact:** couldn't use topics for live eval results.
|
|
|
+- **Fix:** let topic modifiers author a full `PropertyDictionary` (with
|
|
|
+ `JsonElement` arrays), or provide a blessed "background worker → held slot"
|
|
|
+ push keyed by a business id, so "job finishes → update the client's view"
|
|
|
+ doesn't require hand-stashing slot keys. (Cascades from S-7; fixing S-7's list
|
|
|
+ support removes most of the pain.)
|
|
|
+
|
|
|
+### S-11. No built-in auth / identity primitive (orig #14)
|
|
|
+- **What:** identity is entirely DIY — build an `auth` SESSION slot, read it back
|
|
|
+ from `request.held` in every handler, derive admin from the signed public
|
|
|
+ `role`, hand-roll guards.
|
|
|
+- **Impact:** getting the slot shape, scope, key persistence, and the
|
|
|
+ "rebuild-each-entrypoint" pattern right was a large part of the effort.
|
|
|
+- **Fix:** a blessed "identity/auth slot" pattern + helpers
|
|
|
+ (`request.current_user`, `require_admin()`, a server-side guard that issues a
|
|
|
+ `navigate`/error directive) and documented guidance on SESSION vs DEVICE scope
|
|
|
+ and static-key persistence (see S-12).
|
|
|
+
|
|
|
+### S-12. Static crypto keys are required for persistent sessions — but only a console whisper (orig #15)
|
|
|
+> **FIXED (2026-08-31, WP1 server-side effort).** The single-line whispers in
|
|
|
+> `create_signing_provider`/`create_encryption_provider` are now prominent
|
|
|
+> multi-line startup warnings (still warnings, not errors) that spell out the
|
|
|
+> consequence and point at `statum-genkeys`.
|
|
|
+
|
|
|
+- **What:** without `web-config.json` static keys, every signed SESSION/DEVICE
|
|
|
+ slot is invalidated on restart (ephemeral keys) → all users logged out. There
|
|
|
+ is only a quiet `warning(...)` at startup
|
|
|
+ (`src/Statum.vala:86` and `src/Statum.vala:104`).
|
|
|
+- **Fix:** fail loud (or a very prominent warning) when slots are used for
|
|
|
+ identity without persistent keys, or auto-generate + persist keys by default
|
|
|
+ (the `statum-genkeys` tool already exists in `tools/`).
|
|
|
+
|
|
|
+### S-13. `to_properties` returns `Properties`, forcing a `PropertyDictionary` cast (orig #16, primary)
|
|
|
+> **FIXED (2026-08-31, WP1 server-side effort).** `GObjectMapping.to_properties`
|
|
|
+> now returns `PropertyDictionary` directly; all Statum callers updated.
|
|
|
+
|
|
|
+- **What:** `GObjectMapping.to_properties()` returns the `Properties` interface
|
|
|
+ (`src/GObjectMapping.vala:57`); adding a list/nested value requires casting to
|
|
|
+ the concrete `PropertyDictionary` to use its indexer.
|
|
|
+- **Impact:** fragile, undocumented cast.
|
|
|
+- **Fix:** have `to_properties` return `PropertyDictionary` directly (it already
|
|
|
+ constructs one internally at `GObjectMapping.vala:58`). The complementary
|
|
|
+ upstream enhancement (give `Properties` a typed setter) is tracked as
|
|
|
+ `invercargill-upstream-issues.md` I-4.
|
|
|
+
|
|
|
+### S-14. `update_held<T>` is effectively unusable for real models (orig #17)
|
|
|
+> **FIXED (2026-08-31, WP1 server-side effort).** With S-7's array support,
|
|
|
+> `string[]` fields round-trip through `update_held`, and JSON-array fields the
|
|
|
+> typed model cannot read back (collection-typed properties) are carried
|
|
|
+> forward unchanged instead of vanishing (`GObjectMapping.preserving_arrays`;
|
|
|
+> `StateService.update_global_typed` behaves the same). Rebuilding from the
|
|
|
+> source of truth remains the canonical pattern for mutating collection
|
|
|
+> contents.
|
|
|
+
|
|
|
+- **What:** `update_held` (`src/DirectiveBuilder.vala:73`) round-trips through
|
|
|
+ `to_properties`/`from_properties`, so it can't preserve list fields — they'd
|
|
|
+ be dropped. The real pattern is "reload from DB and `update(key, State)`".
|
|
|
+ `update_held` looks temptingly usable until your list vanishes.
|
|
|
+- **Impact:** dead convenience API for any real slot.
|
|
|
+- **Fix:** either lift the list limitation (depends on S-7, making `update_held`
|
|
|
+ safe) or document loudly that `update_held` is scalar-only and the canonical
|
|
|
+ mutation pattern is rebuild-from-DB + `update`.
|
|
|
+
|
|
|
+### S-15. Page route params don't reach actions (orig #18)
|
|
|
+- **What:** actions resolve to GUID endpoints, so a page's `{slug}`/`{id}` aren't
|
|
|
+ in `request.route_params`. The documented workaround is to seal context into
|
|
|
+ the action's private data — verbose and easy to forget.
|
|
|
+- **Impact:** every parametrized action needs per-action sealing boilerplate.
|
|
|
+- **Fix:** a first-class "page context" mechanism that actions can read without
|
|
|
+ per-action sealing (e.g. automatically include the originating page's resolved
|
|
|
+ route params in `StatumRequest`, which is populated at
|
|
|
+ `src/StatumHandlers.vala:125-132`).
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Cross-cutting / DX
|
|
|
+
|
|
|
+### S-16. Blank pages with zero diagnostics — SEVERE (orig #19)
|
|
|
+- **What:** the combination of S-1, S-2 and S-3 produces *blank pages* (not even
|
|
|
+ the preloader) with no console error and no server error. The hardest class of
|
|
|
+ bug.
|
|
|
+- **Fix:** a "statum debug" mode that logs: which slots are in scope, which
|
|
|
+ `stm-if` evaluated falsy and why, and any `with(scope)` `ReferenceError`s.
|
|
|
+
|
|
|
+### S-17. `X-Statum-Slot` / `X-Statum-Private` headers are hard to construct for testing (orig #20)
|
|
|
+- **What:** figuring out the reference form (`key=…; as_at=…`) vs base64-frame
|
|
|
+ form, and that action private travels on `X-Statum-Private`, took
|
|
|
+ reverse-engineering the JS client.
|
|
|
+- **Fix:** a short "testing Statum handlers with curl" recipe in the docs, and/or
|
|
|
+ a helper that prints the exact headers a client would send.
|
|
|
+
|
|
|
+### S-18. Documentation gaps vs the example (orig #21)
|
|
|
+- **What:** the example app (`example/`) is minimal (counter + a scalar
|
|
|
+ announcement). It doesn't exercise list-bearing slots, encrypted private data,
|
|
|
+ `update_held`, typed action private, editable form fields, selects, or
|
|
|
+ background pushes — i.e. exactly the things that bit.
|
|
|
+- **Fix:** a richer example (or a "patterns" doc) covering lists + private +
|
|
|
+ actions.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Fixed during the 2026-08-31 stack effort (not in the Percipio report)
|
|
|
+
|
|
|
+Bugs found and fixed while landing WP1–WP4 of the stack refactor; recorded
|
|
|
+here so they stay tracked. All are fixed.
|
|
|
+
|
|
|
+### S-19. `from_properties` crashed reading cached `int64` `ValueElement`s
|
|
|
+> **FIXED (2026-08-31, WP1 server-side effort).** `GObjectMapping.from_properties`
|
|
|
+> now converts cached `ValueElement`s through the native GValue transformation
|
|
|
+> (`Value.type_transformable`/`transform`) instead of the generic
|
|
|
+> `Element.to_value` dispatch, whose `int64`/`uint64` branches mis-handle the
|
|
|
+> boxed representation and crashed on read-back. Verified by `Statum/tests`
|
|
|
+> round-trips.
|
|
|
+
|
|
|
+- **What:** a slot read back through `from_properties` whose properties still
|
|
|
+ held server-cached `ValueElement`s (rather than re-parsed JSON) crashed in
|
|
|
+ `Element.to_value` on `int64` values — the read-side twin of the upstream
|
|
|
+ I-1 write-side crash.
|
|
|
+- **Fix:** `element_value` in `src/GObjectMapping.vala` — the raw scalar never
|
|
|
+ passes through the generic element dispatch when a native transform exists.
|
|
|
+
|
|
|
+### S-20. `EncryptionProvider.read_properties` use-after-free
|
|
|
+> **FIXED (2026-08-31, WP1 server-side effort).** `read_properties` previously
|
|
|
+> returned a `JsonObject` wrapping memory owned by a temporary `JsonElement`;
|
|
|
+> once that element was finalized the returned properties pointed at freed
|
|
|
+> memory. It now re-parses the decrypted JSON into a `PropertyDictionary` it
|
|
|
+> owns, with values held by fresh `JsonElement`s. (Also noted under upstream
|
|
|
+> I-2, where the crash was first chased.)
|
|
|
+
|
|
|
+- **What:** the unseal path (`src/Cryptography/EncryptionProvider.vala`)
|
|
|
+ handed back a view over a temporary parse tree; consumers reading it later
|
|
|
+ read freed memory.
|
|
|
+- **Fix:** re-parse into an owned `PropertyDictionary`.
|
|
|
+
|
|
|
+### S-21. `Spry.Authentication.UserPermissionEntity` was missing its `Object` base
|
|
|
+> **FIXED (2026-08-31, WP2 Spry rewrite).** Found while wiring the rewritten
|
|
|
+> Spry (recorded here because the effort's issue log lives in these files):
|
|
|
+> the entity class was declared without `: Object`, so its GType was
|
|
|
+> fundamental and it could not be instantiated — the user-management wiring
|
|
|
+> crashed at startup. Fixed by deriving from `Object` like every other entity.
|
|
|
+
|
|
|
+---
|
|
|
+
|
|
|
+## Top 5 ranked by pain (Statum-side; see upstream file for the #1 process crash)
|
|
|
+
|
|
|
+1. **S-2** unset-slot → silent falsy → blank page — no diagnostics; affects every
|
|
|
+ page referencing an optional slot.
|
|
|
+2. **S-7** no list serialisation — makes the typed-model API unusable for real
|
|
|
+ apps and forces verbose `PropertyDictionary` plumbing everywhere.
|
|
|
+3. **S-3** `[hidden]` vs author CSS — sneaky, app-wide risk, trivial library fix.
|
|
|
+4. **S-8** broken `request_private` — a documented API that silently returns null.
|
|
|
+5. **S-16** blank pages with zero diagnostics — the hardest bug class.
|
|
|
+
|
|
|
+> Note: the #1 overall pain item — the `int64`/`bool` **segfault** — is upstream
|
|
|
+> (`invercargill-upstream-issues.md` I-1/I-2); it crashes the whole process and
|
|
|
+> cannot be fixed in Statum.
|
