Procházet zdrojové kódy

Add LFS bootstrap feasibility and effort report

Investigation report for goal 15: feasibility and effort of building a
USM-based Linux distro from LFS 13.1 through staged USM/DCBA repositories,
a chroot build, and OCI image assembly.

Covers: verified platform facts (capabilities, toolchain versions, host
gaps, disk layout), an audit of the USM mechanics the bootstrap relies on
(staged installs, ResourceFinder resolution semantics, single-pass
resolver, file:// repos, the DCBA artifact flow), per-item analysis of all
six brief items, a seven-stage architecture with the resolver argument for
staging, SBU-derived wall-clock and engineering-time estimates, a risk
register with mitigations, and a recommended task breakdown for the
implementation goal.

All USM behaviour claims are cited to file and line at master commit
b046afe; LFS facts cite the 13.1 book checkout. Report only - no code
changes.
saddle-Clanker-Saddle před 1 měsícem
rodič
revize
af9092cd2d
1 změnil soubory, kde provedl 648 přidání a 0 odebrání
  1. 648 0
      LFS_BOOTSTRAP_FEASIBILITY.md

+ 648 - 0
LFS_BOOTSTRAP_FEASIBILITY.md

@@ -0,0 +1,648 @@
+# USM Distro Bootstrap — Feasibility & Effort Report
+
+Investigation report for goal 15 ("a USM based Linux distro"), covering every
+item of the brief: a scaffold repository for LFS bootstrap packages, a
+bootstrap container, a USM repository of LFS packages, a DCBA repository of
+LFS build artifacts, a USM-driven chroot build, and a container of the final
+root with USM installed.
+
+Verdict up front: **feasible, with an explicitly staged architecture, and
+conditioned on the build host.** USM's existing mechanics (staged installs
+via `USM_DESTDIR`/`USM_CONFIGDIR`, destination-local resource resolution,
+`file://` repositories, the DCBA build-artifact flow) map well onto an LFS
+bootstrap — but USM's resolver is single-pass, while LFS is a three-pass
+toolchain construction (temporary → chroot → final), so the bootstrap cannot
+be one USM resolution. It must be staged: a handful of driver invocations,
+each a valid single-pass resolution, chained by an orchestrator. The dominant
+risks are environmental, not architectural: device nodes and `/proc` inside a
+capability-restricted chroot, systemd's build requirements, and self-hosting
+USM itself (valac + glib are not in the LFS book).
+
+Everything below was verified against the USM checkout at commit `b046afe`
+("Merge branch 'goal/dcba' of Tilo15/usm into master", branch `master`) and
+the LFS book checkout at branch `13.1` (LFS 13.1, `trunk` working copy),
+plus live probes of this build sandbox. File/line citations refer to those
+checkouts.
+
+---
+
+## 1. Executive summary
+
+- **Feasible.** Every one of the six brief items maps onto USM mechanics that
+  exist on `master` today; none requires new USM core features. What must be
+  built is an *orchestration layer* (a generator that turns the LFS book into
+  USM scaffolds, stage drivers, and OCI assembly scripts) plus a small
+  beyond-LFS package set so USM can build itself.
+- **The architecture must be staged.** `Resolver.resolve`
+  (src/lib/Resolver.vala:232) processes `manage`+`build`+`runtime` phases in
+  one pass and produces one topological order
+  (src/lib/Resolver.vala:422-428) — a single resolution cannot express LFS's
+  ch5/6 temporary tools → ch7 chroot tools → ch8 final system sequence, where
+  later packages must be built *with* the products of earlier ones. The
+  bootstrap is therefore N driven stages, each internally a normal USM
+  install; the stages are ordered by an orchestrator script, not by the
+  resolver.
+- **Host-conditioned.** On a sandbox without `CAP_SYS_ADMIN`/`CAP_MKNOD`
+  (this one), chroot builds work but `/dev` nodes are plain files and
+  `/proc`/`/sys` are empty. That is fine for the great majority of LFS
+  packages (proven: bison 3.8.2 configured and compiled inside a host-root
+  clone with exactly that setup) but endangers the systemd build and anything
+  reading `/proc/mounts`. Final artifacts should be produced on a full host
+  (or a rootful container); the sandbox remains excellent for development of
+  the pipeline itself.
+- **Disk layout matters**: `/` (and `/tmp`) is a 17 GB overlay while
+  `/workspace` has ~11 TB free — all LFS scratch trees must live under
+  `/workspace`. (The host also lacks `g++`/`patch` — two chapter-2
+  requirements — but the sandbox has root and dnf5, so stage 0's image
+  recipe closes that with one `dnf install` line; see §2.3.)
+- **Effort.** Roughly **6–9 engineer-weeks** for the pipeline (generator +
+  stage drivers + OCI scripts + key management + CI), of which the single
+  largest item is the book-to-scaffold generator (94 packages, ~640 command
+  blocks, needing curated per-package handling). Wall-clock for one full
+  bootstrap run on this 28-core sandbox: **~3–5 hours without test suites,
+  ~8–14 hours with** (SBU-derived; see §7). Artifacts published per run:
+  ~94 `.usmc` source packages, ~94 `.dcba` build artifacts, two signed
+  listings.
+
+## 2. Verified platform facts
+
+Every probe below was run live in this session (2026-09-23) unless marked
+"inherited from the goal-15 investigation"; re-running any of them is a
+one-liner.
+
+### 2.1 Capabilities and kernel
+
+- `grep CapEff /proc/self/status` → `00000000800405fb`;
+  `capsh --decode` → `cap_chown, cap_dac_override, cap_fowner, cap_fsetid,
+  cap_kill, cap_setgid, cap_setuid, cap_setpcap, cap_net_bind_service,
+  cap_sys_chroot, cap_setfcap`.
+  **`cap_sys_chroot` present; `cap_mknod`, `cap_sys_admin`,
+  `cap_net_admin` absent** (CapBnd equals CapEff — they cannot be gained).
+  Consequences: `chroot(2)` works; `mknod(2)` fails (`Operation not
+  permitted`); no mounts, no user/pid namespaces, no overlayfs.
+- Kernel `7.0.12-101.fc43.x86_64` (uname -r) — comfortably above the book's
+  minimum kernel requirement (the book specifies >= 5.10 for glibc's
+  `--enable-kernel` reasoning; `chapter02/hostreqs.xml`).
+- chroot re-proven this session: a minimal root (`/usr/bin/true` + its
+  `ldd` closure + the dynamic loader) executed via `chroot $W /usr/bin/true`
+  → "CHROOT EXEC OK".
+- Inherited from the goal-15 investigation (not re-run): a clone of the host
+  root with a plain-file `/dev/null` and empty `/proc` configured and
+  **compiled GNU bison 3.8.2 with gcc inside chroot**; the only fix needed
+  was `PATH` reconstruction under `env -i`. Also inherited: `tar` cannot
+  carry device nodes into this sandbox (`Cannot mknod: Operation not
+  permitted`).
+
+### 2.2 Resources
+
+- 28 cores (`nproc`).
+- Storage: `/workspace` on `/dev/sdc1` — 19 T total, **11 T free**;
+  `/` is an overlay with only **17 G free** (so `/tmp` is tight — LFS
+  scratch trees must live under `/workspace`).
+- Network: outbound HTTPS works (`curl -sI https://ftp.gnu.org/gnu/` →
+  `HTTP/1.1 200 OK`).
+
+### 2.3 Toolchain versions and version soundness
+
+- Host gcc `16.2.1 20260819 (Red Hat 16.2.1-2)`, host glibc `2.43`
+  (`ldd --version`), Python 3.14.7, perl 5.42.3, bison 3.8.2.
+- Book targets (reference/LFS-Stable `packages.ent`): gcc `16.2.0`, glibc
+  `2.44`, binutils `2.47`, sysvinit `3.14` / systemd `261.3` (both revisions
+  shipped in the book).
+- Soundness for a *native* (non-cross) bootstrap:
+  - host glibc 2.43 < book glibc 2.44 → the temporary tools never
+    accidentally link against a *newer* libc than they are building;
+  - host gcc 16.2 matches the book's 16.2.0 — and crucially the book warns
+    (`chapter02/hostreqs.xml`) that *newer* gcc/binutils than the book's are
+    untested and likely to break the build; equal is the intended case;
+  - kernel 7.0 > minimum → glibc headers configure cleanly.
+- **Host tool gaps (new this session):** `/usr/bin/g++` and `cc1plus` are
+  **absent** (`gcc -x c++` fails: "cannot execute 'cc1plus'"), and `patch`
+  is absent. `find`/`makeinfo` exist where needed. The book's ch2 software
+  list requires both a C++ compiler and patch. `dnf5 5.4.3.0` is present and
+  both `gcc-c++.x86_64 16.2.1-2.fc44` (updates) and `patch.x86_64
+  2.8-4.fc44` (fedora) are available → the Stage-0 image build must
+  `dnf install gcc-c++ patch` (plus any other missing ch2 items re-checked
+  at image-build time).
+
+### 2.4 Container runtimes
+
+- `podman`, `docker`, `buildah`, `skopeo`, `umoci`: **all absent**. So
+  containers must be *assembled* (tar layers + config + manifest, i.e. an
+  OCI image layout) by script here; they can be *consumed* anywhere a
+  runtime exists. `usm deploy`'s podman path (§4.6) remains available on
+  hosts that have podman.
+
+## 3. USM mechanics the bootstrap relies on (verified on master)
+
+These are the load-bearing behaviours; each was read at the cited location
+in this session.
+
+1. **Staged installs.** `Paths.override_from_environment`
+   (src/lib/Paths.vala:189-207) overrides `destination`, `prefix`, every
+   dir, and `usm_config_dir` from `USM_DESTDIR`, `USM_PREFIX`,
+   `USM_BINDIR`, … `USM_CONFIGDIR`. `Manifest.run_build` then calls
+   `paths.set_envs()` (src/lib/Manifest.vala:289-314), exporting
+   `DESTDIR/PREFIX/BINDIR/LIBDIR/…` to the build script, and
+   `Manifest.run_install` clones the paths and overrides `destination`
+   with the per-package install path (src/lib/Manifest.vala:441-475) before
+   spawning the install script. `scripts/dcba-e2e.sh` drives two fully
+   isolated roots this way end-to-end (its header documents the recipe:
+   `<root>/config` via USM_CONFIGDIR, `<state>/{lists,packages,installed}`
+   pre-created, `usm.config` with `is_managed=true` +
+   `managed.state_path`).
+2. **ResourceFinder local presence.** `ResourceFinder.locate_resource`
+   (src/lib/ResourceFinder.vala:13-58) resolves refs mostly **relative to
+   `paths.destination`** (`locate_path`, `locate_res`, `locate_cfg`,
+   `locate_bin` walks `$PATH` under destination, `locate_libres`,
+   `locate_pc` including `PKG_CONFIG_PATH` entries, `locate_vapi`,
+   `locate_typelib`, …), so inside a chroot (destination `/`) resolution
+   sees only the staged root. **Caveat (a real constraint, not a blocker):
+   several locators are host-absolute** — `locate_lib` consults
+   `ldconfig -p` and falls back to absolute `/lib /usr/lib /lib64
+   /usr/lib64` (src/lib/ResourceFinder.vala:88-126), and `locate_inc`,
+   `locate_libexec`, `locate_gir`, `locate_info/man/locale`,
+   `locate_app` use absolute `/usr/...` paths. Inside a chroot that is
+   correct (destination is `/`); *outside* a chroot with USM_DESTDIR set,
+   `lib:`/`include:` refs can be satisfied by the host — the stage drivers
+   must run usm inside the target chroot, or audited for it.
+3. **Resolver is single-pass.** `Resolver.resolve`
+   (src/lib/Resolver.vala:232-287) picks roots, does ONE batched SPM query,
+   processes all packages, and `resolution_phases` folds
+   `manage`+`build`+`runtime` (src/lib/Resolver.vala:422-428) into one
+   dependency graph with one topological install order. Missing resources
+   resolve: (a) local presence, (b) system packages, (c) USM packages —
+   exactly the semantics a stage needs ("what the previous stage left in
+   the root counts as present"), but it cannot reorder the world between
+   two packages in one transaction.
+4. **file:// repositories.** `Repository.get_client`
+   (src/lib/Repository/Repository.vala:34-53) routes `file` →
+   `GioRepositoryClient`, `http`/`https` → `CurlRepositoryClient`
+   (curl `-fsSL --max-time`). All repositories in the bootstrap can be
+   plain directories on the same filesystem — no server needed.
+5. **DCBA flow.** `usm dcba repository init/add` (src/cli/Dcba.vala):
+   `init` generates an ed25519 keypair under `keys/` (private 0600) and a
+   `.dcbar` descriptor; `add <file>.dcba` validates tar.xz, derives
+   name/version from the filename via `DcbaArtifact.parse_filename`
+   (src/lib/Repository/DcbaArtifact.vala:22-56) and rebuilds a signed
+   `ARTIFACTS.usml` (SHA-512 over entry lines, ed25519 signatures —
+   `DcbaListingWriter.write`, src/lib/Repository/DcbaArtifact.vala:82+).
+   The artifact itself is literally the package cache's `build.tar.xz`
+   renamed: `scripts/dcba-e2e.sh` copies
+   `<state>/packages/<n>-<v>/build.tar.xz` to `<n>-<v>.dcba` (its §3),
+   produced by `CachedPackage.archive_build`
+   (src/lib/State/CachedPackage.vala:101-131, digest-gated re-archive).
+   A consumer with the `.dcbar` configured gets the artifact as its build
+   cache ("Retrieved pre-built artifact"), records provenance in
+   `origin-info` (`"dcba"` field), and a tampered artifact is rejected with
+   a warning + fallback to source build — all asserted by the e2e script
+   (assertions a–f).
+6. **`usm deploy`.** `usm manifest deploy` (src/cli/Deploy.vala:223-473)
+   generates a Containerfile-based context and (unless `--no-build`) builds
+   with podman and saves an xz-compressed image. Default base is Fedora
+   (src/cli/Deploy.vala:16); `--base` overrides. This is the
+   "container from the chroot" path on hosts with podman; here we script
+   OCI assembly instead (Stage 6).
+7. **Scaffolds.** `usm scaffold <name> <template> [attrs]`
+   (src/cli/Scaffold.vala) generates `MANIFEST.usm` + `usm-scripts/` for
+   templates `basic`/`makefile`/`meson` with attributes `vala`/`c`/`acquire`
+   (the last adds a wget/tar acquire script + deps). Scaffolded manifests
+   express deps as resource refs (`bin:make`, `pc:glib-2.0.pc`, …), the
+   same vocabulary the LFS scaffolds will use — but LFS scaffolds will be
+   *generated*, not hand-scaffolded, and their scripts are bespoke per
+   package.
+
+## 4. The six brief items — analysis
+
+### 4.1 Item 1: a repository of USM scaffolds containing all packages needed to bootstrap LFS
+
+**Mapping.** One git repository ("the scaffold repo") holding one directory
+per LFS package, each a USM scaffold: `MANIFEST.usm` (name/version/licences/
+provides/depends/execs) + `usm-scripts/{acquire.sh,build.sh,install.sh}` +
+vendored patches. `usm manifest package` turns each into a `.usmc`; the
+collection is published with `usm repository init/add/build-list`. Nothing
+here needs new USM features — `Scaffold.vala` already writes this shape;
+the generator will write it directly (scaffold templates don't fit LFS's
+per-package configure flags anyway).
+
+**What's missing / must be built — the generator.** The book is DocBook
+XML; the machine-readable part is excellent (`packages.ent`: 718 entities —
+names, versions, URLs, md5s, SBU/disk estimates as text), but the ~640
+build steps are prose. Counts from this checkout:
+
+- `userinput remap="…"` command blocks across ch5–8: **558** — 105
+  `configure`, 118 `make`, 180 `install`, 76 `pre`, 76 `test`, 3 `adjust`;
+  plus **85** un-remapped `<userinput>` (chapter setup commands).
+- Package-ish pages: ch5 5, ch6 17, ch7 8, ch8 91 files (85 with build
+  commands) → **~94 source packages** distinct (some pages are
+  multi-pass: gcc appears in ch5 pass1/pass2 and ch8; binutils pass1/ch8).
+
+A naive `<userinput>`-to-shell-literal extraction produces *almost* right
+scripts — but not right: entities need expansion, `make -j1` /
+`make -k check` conventions need policy, ch5/6 scripts must hard-code
+`/tools` prefixes and env (`LFS`, `LFS_TGT`, `PATH=/tools/bin:...`), ch8
+scripts assume the chroot, and a few packages need hand-massaging (gcc's
+three passes and its `limitl`, glibc's `libc.picture`-era configure
+options, systemd's `meson -D*` policy, perl's `groff`-era hints). So: a
+generator (Python, ~1.5-2k lines) with a **curated overrides table** for
+the ~15 hairy packages, emitting both the scaffolds and a
+`stage-manifest.json` the orchestrator consumes. Generated scaffolds are
+checked into git, reviewed once, then regenerated only on book bumps.
+
+**Effort**: generator 2-3 engineer-weeks (including the overrides table and
+the ch5/6/ch7/ch8 script envs), scaffold review pass 2-4 days.
+
+### 4.2 Item 2: build a container into this bootstrap environment
+
+**Mapping.** The bootstrap container is the *build host* for stages 1-4: a
+root filesystem with a chapter-2-compliant toolchain, the `dnf`-installed
+gaps closed, `usm` + its runtime deps (curl, git, libsodium, glib, json-glib
+— via dnf on the Fedora base) preinstalled, and the LFS sources either
+pre-fetched or fetchable. Two ways to produce it:
+
+- **(a) dnf-based (recommended for stage 0).** Start from a Fedora
+  container base (or `dnf --installroot` into a directory on this host —
+  no runtime needed), `dnf install` the ch2 checklist + `gcc-c++ patch` +
+  usm's runtime closure, then tar it up. Fully reproducible-ish, fast.
+- **(b) host-clone (proven here).** Clone the sandbox host root, add the
+  missing pieces via dnf, prune to the ch2 list. Faster to stand up (the
+  bison proof used exactly this), less hermetic; fine for development,
+  not for the published artifact.
+
+**What's missing / must be built.** An OCI image *assembler* script: tar
+the rootfs into one (or a few) layers, write `oci-layout`, `index.json`,
+`blobs/sha256/…` config+manifest. ~200-300 lines of Python/shell; standard
+format, no runtime required to *produce*. (On podman hosts, `usm manifest
+deploy --no-build`'s context or a plain Containerfile is the alternative
+production path.)
+
+**Effort**: image build recipe + assembler 3-5 days, including a
+verify-by-chroot harness (unpack the image, chroot in, run the ch2
+version-check script from the book).
+
+### 4.3 Item 3: a USM repository of the LFS packages
+
+**Mapping.** Exactly `usm repository init lfs && usm repository add *.usmc
+&& usm repository build-list` (src/cli/Repository.vala): per-package
+`.usmc` from `usm manifest package`, a `PACKAGES.usml` signed listing
+(ed25519 keys under `keys/`, same mechanics as DCBA init), served as a
+`file://` (or later https) repository. Consumers `usm add-repo` it. The
+only structural question is *granularity of stages*: because the resolver
+is single-pass (§3.3), the repo should be **one repository with stage
+tags** — e.g. manifests carry a `stage` extra-property (or separate repos
+`lfs-tmp0`, `lfs-tmp1`, `lfs-final`; tags are lighter). The orchestrator
+installs stage by stage; each stage's install is one ordinary `usm install
+<pkg>...` transaction whose dependencies resolve against what earlier
+stages already left in the root (local presence, §3.2).
+
+**What's missing / must be built.** The stage tags (generator-emitted); a
+`provides` vocabulary decision for LFS (e.g. `lib:libc.so.6` from glibc,
+`bin:gcc` from gcc, `pc:…` where the book installs .pc files); discipline
+that ch5/6 temporary packages *self-contain* under `/tools` and do not
+`provides`-collide with ch8 finals (they must not — same package name,
+different version suffix, e.g. `gcc-tmp-16.2.0` vs `gcc-16.2.0`).
+
+**Effort**: 1 week, mostly the provides/depends modelling and validating
+resolution with the real resolver (the itemised `UNSATISFIABLE` reports
+make this pleasantly debuggable).
+
+### 4.4 Item 4: a DCBA repository of the LFS packages
+
+**Mapping.** Direct use of the DCBA flow (§3.5): after a build stage, each
+package cache's `build.tar.xz` becomes `<name>-<version>.dcba`
+(`cp`, exactly as `scripts/dcba-e2e.sh` §3 does), `usm dcba repository
+add` rebuilds the signed `ARTIFACTS.usml`, and consumers with the `.dcbar`
+configured install with pre-built artifacts ("Retrieved pre-built
+artifact") — turning later stages (and any third party) from compiles into
+cache-restores. This is precisely the artifact-distribution model the
+brief asks for, and it is what makes Stage 4 (final chroot build) cheap
+enough to iterate on.
+
+**What's missing / must be built.** A small publish driver (loop over the
+state dir, copy, add, verify — `dcba repository verify` exists). Real
+considerations: (i) **the artifact is only valid for one stage's
+environment** — a ch5 `build.tar.xz` is useless in ch8 — so artifacts must
+be namespaced (separate DCBA repos per stage, or name-embedded stage
+suffixes mirroring the `-tmp` package names); (ii) signing keys per repo
+with `keys/` 0600 and the public key distributed in the `.dcbar`; (iii)
+`.dcba` files are multi-hundred-MB for gcc — disk budgeting in the publish
+step (11 TB free here; a full set is ~15-25 GB xz).
+
+**Effort**: 2-4 days including the verify-by-fresh-root test (reuse the
+dcba-e2e.sh pattern).
+
+### 4.5 Item 5: build a chroot using USM from those repos
+
+**Mapping.** Stage 4 in the architecture below: enter the stage-0 container
+root (chroot on this sandbox; a container anywhere else), configure USM's
+`USM_CONFIGDIR`/state inside it, `usm add-repo` the file:// USM repo and
+`usm dcba add-repo` the DCBA repo(s), then run the orchestrator's stage
+installs — each `usm install <stage-packages>` resolves against the root's
+current contents, restores `.dcba` caches where available, builds where
+not, and stages outputs into the root. The result after the final stage is
+an LFS root *built by USM transactions* — the actual "USM distro" claim.
+
+**Constraints found.** (i) The chroot needs `/dev/null` (plain file is
+enough for most builds — proven) and ideally `/dev/zero`, `/dev/random`,
+`/dev/tty`; without `CAP_MKNOD` they stay plain files, so anything doing
+`ioctl` on them (systemd's build probes, some test suites) is at risk.
+(ii) `/proc` empty breaks packages that read `/proc/mounts` or
+`/proc/self/…` at build time; systemd again, plus `procps` tests. (iii)
+The ResourceFinder host-absolute locators (§3.2) are *correct inside* a
+chroot (destination `/`), so run `usm` inside the chroot for the build
+stages — which is also the honest end-to-end test of self-hosting. (iv)
+`USM never creates the state root itself` (dcba-e2e.sh header): the
+orchestrator pre-creates `<state>/{lists,packages,installed}` per root.
+
+**What's missing / must be built.** The stage orchestrator (~300-500
+lines): create root, populate /dev workaround files, run per-stage
+installs with the right env, checkpoint between stages (tar snapshot for
+fast iteration), and the sysvinit-vs-systemd switch (recommend building
+sysvinit revision first — no /proc dependency at build time — and
+systemd as a follow-up on a full host).
+
+**Effort**: 1-1.5 weeks including debugging the first full run.
+
+### 4.6 Item 6: a container based on that chroot with USM installed
+
+**Mapping.** The final root already *is* a USM-managed root (stages 4-5
+installed everything through USM transactions, and USM itself is installed
+by stage 5). Producing the container = the same OCI assembly as item 2
+(§4.2): tar the final root as one layer, write config (cmd `/bin/bash`,
+env `PATH=/usr/bin:/bin`, labels recording the repo commits + listing
+digests), manifest, `oci-layout`. On a podman host, `usm manifest deploy`
+of a meta-package (or just `podman import`) is the one-liner alternative.
+
+**What's missing / must be built.** Nothing structural. A "distro
+identity" decision (name, /etc/os-release — trivially a scaffold package),
+and the same assembler script with a different config blob. Verification:
+`podman run` on a capable host, or chroot-in-the-unpacked-image on this
+sandbox (bash + `usm --version` + `usm list`).
+
+**Effort**: 2-3 days, including the smoke-test matrix.
+
+## 5. The staged architecture
+
+One diagram, then per-stage detail. `H` is the build host (this sandbox or
+a full machine); every stage's output feeds the next.
+
+    Stage 0            Stage 1        Stage 2/3                 Stage 4              Stage 5        Stage 6
+    bootstrap image ← scaffolds  →   USM repo + DCBA repo  →   final LFS root   →   +beyond-LFS    → OCI image
+    (dnf + ch2 tool-   (generated     (built INSIDE a stage-0    (built by staged     + USM itself    (assembled by
+     chain + usm)       from book)     chroot, staged installs)  usm transactions)   (self-host)     script)
+
+### Stage 0 — bootstrap container/image
+
+Purpose: a hermetic chapter-2-compliant build host. Recipe (dnf-based):
+
+1. `dnf --installroot=$ROOT --releasever=44 install @build-tools` or an
+   explicit list from the book's ch2 table, **plus `gcc-c++` and `patch`**
+   (the two gaps found on this sandbox — see §2.3), plus usm's runtime
+   closure: `curl git libsodium glib2 json-glib` (Fedora names).
+2. Install `usm` itself (any build that runs on the host glibc — the
+   Fedora dnf set is fine; this stage-0 copy never needs to be the
+   self-hosted one).
+3. Fetch/verify the ~94 LFS sources (URLs+md5 from `packages.ent`), or
+   vendor them into the image; pre-seed the file:// repo layout.
+4. Assemble the OCI image (script) — or just keep it as a directory tree
+   for chroot use on this sandbox.
+
+Outputs: `$ROOT` (chroot-able), `stage0.img` (OCI layout).
+
+### Stage 1 — the scaffold repository
+
+The generator (§4.1) reads `packages.ent` + chapter XML from the LFS
+checkout and emits one scaffold dir per package-pass:
+
+- `MANIFEST.usm` — name (with `-tmp` suffixes for ch5/6 passes),
+  version, licence (from the book's licence notes), `depends`
+  (manage: `bin:bash`, `bin:coreutils`…; build: the tools the book's
+  commands call; runtime: the libs they install), `provides`
+  (`bin:gcc`, `lib:libc.so.6`, …), `execs` (acquire/build/install/test),
+  extra-property `stage` ∈ {tmp0, tmp1, chroot, final}.
+- `usm-scripts/build.sh` — the book's `configure`/`make` commands with
+  `PATH`/`LFS_TGT` env set by the stage driver, `make -j$(nproc)`.
+- `usm-scripts/install.sh` — the book's install commands; DESTDIR-style
+  staging is NOT what LFS does (it installs into `/`/`/tools` inside the
+  chroot), so install scripts write directly and USM's per-package
+  install staging is bypassed via `install_path == destination` (or the
+  scripts `cp` from a destdir; per-scaffold choice, recorded in the
+  generator's overrides).
+- vendored patches from the book.
+
+Output: the scaffold git repo (reviewed, committed) → `*.usmc` via
+`usm manifest package` → the USM repo (§4.3).
+
+### Stage 2/3 — build the USM + DCBA repositories inside a stage-0 chroot
+
+Why inside: the packages must be built by the stage-0 toolchain in the
+stage-0 environment, and USM must be able to see its own staged outputs
+(local presence) — which is exactly "run usm inside the chroot". Driver:
+
+1. chroot into stage-0's `$ROOT` (plain-file /dev/null workaround; empty
+   /proc acknowledged).
+2. `usm add-repo` the stage-1 USM repo; for each stage s ∈
+   {tmp0, tmp1, chroot, final}: `usm install <stage-s packages>` — each
+   install resolves against the root as-left-by-stage-(s-1).
+3. After each stage: publish `.dcba` artifacts (copy `build.tar.xz` →
+   `<name>-<version>.dcba`, `usm dcba repository add`) into per-stage DCBA
+   repos.
+
+Outputs: `PACKAGES.usml`-signed USM repo; per-stage `ARTIFACTS.usml`-signed
+DCBA repos.
+
+### Stage 4 — USM-driven chroot build of the final root
+
+Fresh root `$LFS`; populate `/dev` workaround files + empty `/proc`;
+chroot; configure a *new* USM state inside it; add the stage-1 USM repo +
+the per-stage DCBA repos; run the same stage sequence. Because the DCBA
+caches restore, this stage is CPU-cheap (cache restores + final-system
+gcc/glibc builds are the only heavy items). End state: a complete LFS
+system whose every file was placed by a USM transaction.
+
+### Stage 5 — beyond-LFS self-hosting set + USM itself
+
+The book does not contain USM's build deps. The gap set (from the plan's
+evidence, re-confirmed against the book's package list): the book HAS
+pcre2 (chapter08/pcre.xml), libffi, python, meson, ninja, pkgconf,
+elfutils/libelf — so the beyond-LFS set is small: **glib, libgee, valac,
+json-glib, curl** (+ optional git). Order matters: glib → libgee →
+json-glib → valac → USM. Then `usm install usm` (from a `.usmc` of USM's
+own repo — USM's MANIFEST.usm exists at the repo root) is the
+self-hosting milestone: the distro's package manager installs itself.
+
+### Stage 6 — OCI image assembly
+
+Scripted (no runtime here): tar `$LFS` (+ stage-5 additions) as one
+gzip/xz layer, emit `oci-layout`, `index.json`, config JSON with
+`cmd ["/bin/bash"]`, env, and labels carrying the repo commit + listing
+digests. Optionally split `/usr` vs `/etc/var` into two layers for
+cache-friendlier future updates. On podman hosts: `podman load`/`run` for
+verification, or skip assembly entirely and use `usm manifest deploy`.
+
+## 6. Why staged (the resolver argument, precisely)
+
+`Resolver.resolve` (src/lib/Resolver.vala:232) computes ONE resolved set
+with ONE topological order over `manage`+`build`+`runtime` refs
+(src/lib/Resolver.vala:422-428). LFS needs three orders interleaved:
+binutils-p1 → gcc-p1 → (ch5 libs) → gcc-p2 must complete *before* ch7's
+chroot tools build against them, and ch8's final glibc/gcc must replace
+the temporary ones — i.e. the same *resource* (`lib:libc.so.6`,
+`bin:gcc`) is provided by different packages at different times, with
+deliberate replacement semantics. USM's resolver treats one name-version
+per package and picks providers once; replacement-across-stages is a
+different problem (it has `usm downgrade`/transaction machinery, but not
+"same provides, different package, by stage"). Hence: stage boundaries are
+orchestrator-level, stage interiors are USM resolutions. This is not a
+USM defect — it is the honest boundary between a package manager and a
+distro builder, and it keeps every stage debuggable with stock `usm`
+commands and stock resolver error reports.
+
+## 7. Effort estimates
+
+### 7.1 Wall-clock (build time)
+
+SBU data is machine-readable in `packages.ent` (entities
+`<pkg>-<tmp|fin>-sbu`). Summed this session (test-suite entities excluded,
+"less than X" counted as X/2):
+
+- ch5/6 temporary tools (`-tmp-sbu`, 33 entries): **~21 SBU**
+  (gcc-p2 5.2, gcc-p1 4.3, tcl 2.9, gettext 1.5, glibc-tmp 1.3, …).
+- ch8 final system (`-fin-sbu`, 82 entries): **~100 SBU**
+  (gcc 53, glibc 11, vim 3.3, python 2.7, e2fsprogs 2.4, …).
+- Linux kernel (~32 SBU) is *not* part of the bootstrap (no boot
+  requirement for a container/chroot distro); building it doubles the
+  gcc-dominated budget if wanted.
+
+One SBU = one core's binutils-p1 time, typically **~1-2 min** on modern
+hardware (the book: gcc ~5 min "on the fastest systems"). With ~120 SBU
+total and `make -j28`, the parallelisable `make` phases compress roughly
+10-20x, but configure phases, install phases and single-threaded steps
+(glibc's `make install`, gcc's early stages) do not. Honest estimate for
+one full stage-0→4 pass **on this 28-core sandbox**:
+
+- without test suites: **~3-5 hours** (gcc dominates: its 53 SBU ≈
+  1.5-2 h alone at j28 with serial configure/install).
+- with the book's recommended test suites: **~8-14 hours** (tests are
+  mostly serial and several are multi-SBU).
+- Stage 5 (beyond-LFS + USM): ~30-45 min (glib ~0.5 SBU, valac ~2 SBU,
+  USM itself < 1 SBU at j28).
+- A DCBA-cached re-run of stage 4 (iteration loop): **~20-40 min** — this
+  is the number that matters for daily development.
+
+Disk per full run: sources ~1 GB, build trees ~40-80 GB, artifacts
+(build.tar.xz set) ~15-25 GB, final root ~4-8 GB. Budget ~200 GB per
+working copy; 11 TB free comfortably supports several parallel runs, and
+zero runs if any of it lands on `/` (17 GB — see §2.2).
+
+### 7.2 Engineering time
+
+Per-item figures from §4; consolidated (one engineer, familiar with both
+USM and LFS):
+
+- Stage 0 recipe + OCI assembler + verify harness: **3-5 days**
+- Generator (book → scaffolds, with overrides table): **2-3 weeks**
+- USM repo modelling (stages, provides vocab, resolution validation): **1 week**
+- DCBA publish driver + per-stage repos: **2-4 days**
+- Stage orchestrator (chroot mgmt, /dev workarounds, checkpoints):
+  **1-1.5 weeks**
+- Beyond-LFS set (6 packages, valac ordering) + USM self-host: **3-5 days**
+- OCI final image + smoke tests + documentation: **2-3 days**
+- Integration/debug of the first full run + book-drift tooling
+  (regeneration diff): **1 week**
+
+**Total: ~6-9 engineer-weeks** (the spread is almost entirely the
+generator: 94 packages × curated overrides vs mostly-mechanical).
+Uncertainty: a first full run will surface 5-15 per-package script bugs
+(extraction artefacts, env differences) — that's inside the integration
+line above; a systemd-revision build adds ~3-5 days if pursued (and needs
+a full host for /proc reasons, §8).
+
+## 8. Risk register
+
+- **Device nodes in /dev** (no CAP_MKNOD). Plain-file `/dev/null` works
+  for most builds (proven with bison); `/dev/zero`, `/dev/random`,
+  `/dev/tty` as plain files break anything doing ioctls. Mitigations:
+  stage-0 image carries *pre-created device nodes* (built where mknod is
+  allowed, e.g. the human's machine or a rootful builder — the image is
+  just a tarball, so nodes survive into it; only *creating* them here is
+  impossible); chroots unpacking that image inherit working /dev.
+- **Empty /proc and /sys** (no CAP_SYS_ADMIN to mount). Most packages
+  fine; systemd's build/configure and readers of `/proc/mounts` at risk.
+  Mitigations: sysvinit revision first (no /proc at build time);
+  systemd build on a full host; or bind-mount-free stub files where the
+  book's commands tolerate them.
+- **systemd vs sysvinit.** Both revisions are in the book (261.3 vs
+  3.14). Recommend shipping sysvinit first, systemd as a follow-up goal.
+- **ResourceFinder host-absolute locators** (`ldconfig -p`, `/usr/include`,
+  `/usr/libexec`, gir/man/info/locale — §3.2). Outside a chroot, staged
+  roots can be "satisfied" by host files. Mitigation: run build-stage usm
+  inside the chroot (destination `/`, so absolute == correct); for
+  outside-chroot tooling, audit or later patch ResourceFinder with a
+  strict-destination mode (candidate small USM improvement, out of scope
+  here).
+- **Acquire phase needs curl/git/wget** — but the bootstrap's acquire
+  scripts can instead use *vendored* sources already inside the .usmc or
+  the stage-0 image, removing the network dependency from stages 2-5
+  entirely (recommended; the book's md5s gate integrity).
+- **Book drift.** LFS 13.1 → 13.2+ bumps package versions and commands.
+  Mitigation: the generator is the only thing that reads the book;
+  regeneration diff (git diff of scaffolds) makes drift reviewable.
+- **valac bootstrap ordering.** valac needs glib+libgee; USM needs valac
+  + glib + json-glib. Fixed linear order in stage 5 (glib → libgee →
+  json-glib → valac → USM); no cycle exists because valac's own
+  compilation can use a stage-0-hosted valac (dnf-installed) — the dnf
+  set already includes one, so USM's first self-host can even avoid
+  valac-from-source if desired.
+- **ed25519 key management.** `keys/` 0600, private keys never leave the
+  producer; `.dcbar`/`.usmr` carry the public key. Risk: losing the key
+  orphanes a repo — keep one offline master per distro release line, and
+  document rotation.
+- **Disk.** `/` here is 17 GB — *nothing* large may touch /tmp; all
+  scratch under /workspace (11 TB). Checkpoint tars double usage
+  transiently; clean between stages.
+- **gcc/binutils newer than the book** are explicitly unsupported by LFS
+  (ch2 warning). This host's gcc 16.2.1 == book's 16.2.0 line — fine;
+  any host bump past the book needs a book bump first. Relatedly, the
+  sandbox host itself lacks `g++`/`patch` (ch2 requirements) — with root
+  and dnf5 this is a non-event: stage 0 installs them, and a ch2-check
+  script fails fast on any host still missing items.
+
+## 9. Recommended task breakdown (for the implementation goal)
+
+Ordered; each is independently verifiable and lands value on its own:
+
+1. **stage0 recipe + OCI assembler + ch2 check** (3-5 d) — produce a
+   bootstrap image from dnf; verify by chroot + book ch2 script.
+2. **Generator v1: packages.ent → scaffold skeletons** (3-5 d) — manifest
+   fields, URLs/md5s, acquire scripts; build.sh placeholders; no commands
+   yet. Deliverable: 94 scaffolds packaging cleanly (`usm manifest
+   package` all green).
+3. **Generator v2: chapter XML → build/install scripts + overrides
+   table** (1.5-2 wk) — ch5/6 (tmp passes, /tools env), ch7, ch8;
+   curated handling for gcc/binutils/glibc/systemd|sysvinit/perl/python.
+4. **USM repo modelling + stage tags + resolution dry-runs** (1 wk) —
+   resolver-validated dependency graph per stage, zero UNSATISFIABLE.
+5. **Stage orchestrator + stage-2/3 build of tmp/chroot stages** (1 wk) —
+   first real chroot builds inside stage-0; publish tmp/chroot `.dcba`
+   repos.
+6. **Stage 4: final-system build via USM** (3-5 d) — full ch8 through usm
+   transactions with DCBA caches.
+7. **Stage 5: beyond-LFS set + USM self-host** (3-5 d) — the milestone:
+   `usm install usm` inside the final root.
+8. **Stage 6: final OCI image + smoke tests** (2-3 d) — assembled image,
+   chroot/podman verification matrix, docs.
+9. **(Follow-up, separate goal) systemd revision, kernel build, https
+   repo hosting, ResourceFinder strict-destination mode.**
+
+## 10. Conclusion
+
+The goal is feasible with current USM master: the staged architecture
+uses only documented, e2e-tested mechanics (staged installs, file://
+repos, DCBA artifact distribution), the LFS book is machine-minable
+enough for a generator with a modest overrides table, and the two hard
+environmental constraints of this sandbox (no device-node creation, no
+mounts) are worked around for every stage except a systemd build. The
+recommended first implementation milestone is tasks 1+2: a verified
+bootstrap image and 94 packaging-clean scaffolds — roughly a week of
+work that de-risks everything downstream.