|
|
@@ -57,14 +57,30 @@ checkouts.
|
|
|
three chapter-2 requirements — but the sandbox has root and dnf5, so
|
|
|
stage 0's image
|
|
|
recipe closes that with one `dnf install` line; see §2.3.)
|
|
|
-- **Effort.** Roughly **7–10 engineer-weeks** for the pipeline (generator +
|
|
|
- stage drivers + OCI scripts + key management + CI), of which the single
|
|
|
- largest item is the book-to-scaffold generator (~90 packages, ~640 command
|
|
|
- blocks, needing curated per-package handling). Wall-clock for one full
|
|
|
- bootstrap run on this 28-core sandbox: **~3–5 hours without test suites,
|
|
|
- ~8–14 hours with** (SBU-derived; see §7). Artifacts published per run:
|
|
|
- ~90 `.usmc` source packages (one per package-pass) and as many `.dcba`
|
|
|
- build artifacts, plus the signed listings.
|
|
|
+- **Effort (LFS-only path).** Roughly **7–10 engineer-weeks** for the
|
|
|
+ pipeline (generator + stage drivers + OCI scripts + key management +
|
|
|
+ CI), of which the single largest item is the book-to-scaffold
|
|
|
+ generator (~90 packages, ~640 command blocks, needing curated
|
|
|
+ per-package handling). Wall-clock for one full bootstrap run on this
|
|
|
+ 28-core sandbox: **~3–5 hours without test suites, ~8–14 hours with**
|
|
|
+ (SBU-derived; see §8). Artifacts published per run: ~90 `.usmc`
|
|
|
+ source packages (one per package-pass) and as many `.dcba` build
|
|
|
+ artifacts, plus the signed listings.
|
|
|
+- **But a faster route exists — §5 recommends taking it.** An
|
|
|
+ alternatives analysis (added after review feedback) compares seeding
|
|
|
+ from a **Gentoo stage3** tarball and/or converting **Gentoo ebuilds**
|
|
|
+ against mining the LFS book. Recommendation: the **hybrid** — Gentoo
|
|
|
+ stage3 as the stage-0 build appliance + ebuild-derived package
|
|
|
+ definitions — reaches the *same* purity (every file of the final root
|
|
|
+ placed by a USM transaction; the seed is host scaffolding, exactly
|
|
|
+ like the dnf-built stage 0 of the LFS plan) in **~4.5–6 engineer-weeks**
|
|
|
+ (vs 7–10) and **~2.5–4 h wall clock per run** (vs 3–5 h; the
|
|
|
+ temporary-toolchain passes are only ~18% of the SBU table — the
|
|
|
+ saving is mostly engineering time, not build time). The
|
|
|
+ LFS plan remains fully valid as the fallback and the P3-purity
|
|
|
+ reference; stages 2–6 of the architecture are shared verbatim by both
|
|
|
+ paths. See §5 for evidence (live stage3/ebuild probes) and §5.3 for
|
|
|
+ the side-by-side numbers.
|
|
|
|
|
|
## 2. Verified platform facts
|
|
|
|
|
|
@@ -276,7 +292,7 @@ the ch5/6/ch7/ch8 script envs), scaffold review pass 2-4 days.
|
|
|
root filesystem with a chapter-2-compliant toolchain, the `dnf`-installed
|
|
|
gaps closed, `usm` + its runtime/build closure (curl, git, libsodium, glib,
|
|
|
libgee, json-glib, vala, gobject-introspection, and the two invercargill
|
|
|
-libraries — all via dnf on the Fedora base; see §5 Stage 5) preinstalled,
|
|
|
+libraries — all via dnf on the Fedora base; see §6 Stage 5) preinstalled,
|
|
|
and the LFS sources either pre-fetched or fetchable. Two ways to produce it:
|
|
|
|
|
|
- **(a) dnf-based (recommended for stage 0).** Start from a Fedora
|
|
|
@@ -401,7 +417,291 @@ sandbox (bash + `usm --version` + `usm list`).
|
|
|
|
|
|
**Effort**: 2-3 days, including the smoke-test matrix.
|
|
|
|
|
|
-## 5. The staged architecture
|
|
|
+## 5. Alternatives analysis: is LFS the best seed?
|
|
|
+
|
|
|
+The human's review asked exactly the right question: *"Is LFS the best
|
|
|
+approach? I like Gentoo and don't mind leveraging it, its base image, or
|
|
|
+its package definitions. Goal: rapidly stand something up that truly uses
|
|
|
+USM, deriving from whatever project works best."* This section answers it
|
|
|
+with probes run live in this session (2026-09-23) and re-derives the
|
|
|
+recommendation. The LFS analysis of §4 and the architecture of §6 are
|
|
|
+**kept unchanged** — they remain the purity baseline and the fallback —
|
|
|
+but the recommendation at the end of this section supersedes "start with
|
|
|
+LFS" as the fastest route to a genuinely-USM distro.
|
|
|
+
|
|
|
+### 5.1 What "truly uses USM" means — the measuring stick
|
|
|
+
|
|
|
+A spectrum, stated up front so every option can be scored honestly:
|
|
|
+
|
|
|
+- **P3 (full purity):** every file in the final root was placed by a USM
|
|
|
+ transaction from a USM package built by USM's own build pipeline. The
|
|
|
+ root is *reproducible from USM repos alone*.
|
|
|
+- **P2 (bootstrap-tolerant, what LFS §6 delivers):** the final root is
|
|
|
+ 100% USM-built and USM-managed, but the build *host* (stage 0) was not
|
|
|
+ USM-built — it is dnf-assembled. USM builds every package that ends up
|
|
|
+ in the distro; the host is scaffolding, discarded or kept as a build
|
|
|
+ appliance.
|
|
|
+- **P1 (thin layer):** an existing distro root is enrolled as-is and USM
|
|
|
+ only manages *new* packages on top. Fastest possible, but the base was
|
|
|
+ never built by USM — most of the root is not USM packages.
|
|
|
+
|
|
|
+The goal's phrase "truly uses USM" reads as at least P2. Everything below
|
|
|
+is scored against time-to-P2 and time-to-P3.
|
|
|
+
|
|
|
+### 5.2 The candidate paths, with evidence
|
|
|
+
|
|
|
+**A. LFS-from-scratch (the current report, §4/§6).** Build the toolchain
|
|
|
+the book's way (ch5/6 temporary tools → ch7 chroot tools → ch8 final),
|
|
|
+USM driving each stage. Effort **7–10 engineer-weeks**, wall clock
|
|
|
+**~3–5 h/run** (§8). Strengths: the book is a fixed, finite recipe
|
|
|
+(~90 packages, 13.1 stable, version-pinned); complete control of the
|
|
|
+package set; nothing foreign persists. Weaknesses (all flow from the
|
|
|
+multi-pass design): the single largest generator cost is mining ~646
|
|
|
+DocBook `<userinput>` command blocks into per-package scripts with a
|
|
|
+curated overrides table (~2–3 weeks of the total); the temporary-tools
|
|
|
+stages exist *only* to produce a toolchain that a complete seed already
|
|
|
+has; and stage-4 rebuilds must replay them on a fresh host.
|
|
|
+
|
|
|
+**B. Gentoo stage3 as the stage-0 seed.** Probed this session:
|
|
|
+`distfiles.gentoo.org/releases/amd64/autobuilds/latest-stage3-amd64-openrc.txt`
|
|
|
+(PGP-signed) points at
|
|
|
+`stage3-amd64-openrc-20260920T170055Z.tar.xz` (279 MB; a systemd variant
|
|
|
+exists, 291 MB). File list (`tar tf`, 66,195 entries) and in-chroot
|
|
|
+execution of the unpacked tree show:
|
|
|
+
|
|
|
+- A **complete native toolchain**: gcc 15.3.0, binutils 2.46.1, glibc
|
|
|
+ 2.43-r2, make 4.4.1, bison/flex, autoconf/automake/libtool, cmake
|
|
|
+ 4.3.4, **meson 1.11.2 + ninja 1.13.2 + pkgconf 3.0.7**, perl 5.44.0,
|
|
|
+ Python 3.14.7, texinfo 7.3, patch 2.8, gawk, groff, wget, curl 8.21.0,
|
|
|
+ openssh, rsync — **297 packages installed** (`var/db/pkg` census),
|
|
|
+ i.e. every chapter-2 host requirement is met with margin, plus
|
|
|
+ portage 3.0.82.2 and emerge itself. This is the dnf-based stage-0
|
|
|
+ recipe of §6 Stage 0 already built, tested and published by Gentoo,
|
|
|
+ weekly.
|
|
|
+- **Sandbox compatibility is the same story as §9**: the tarball's
|
|
|
+ `/dev` contains exactly two device nodes (`./dev/console`,
|
|
|
+ `./dev/null` — `tar tvf | grep -c '^[cb]'` → 2); unpacking here
|
|
|
+ demotes them to plain files (`Cannot mknod: Operation not permitted`),
|
|
|
+ and `emerge` refuses to run against a plain-file `/dev/null`
|
|
|
+ ("Failed to validate a sane '/dev'") — but **we never need emerge**:
|
|
|
+ USM replaces it as the package manager. gcc, python, ldd all execute
|
|
|
+ fine in-chroot with the plain-file workaround (verified this session).
|
|
|
+- Not in stage3 (checked): glib, vala, libgee, json-glib, libsodium, git
|
|
|
+ — i.e. **USM's own build closure is exactly the beyond-LFS set of
|
|
|
+ §6 Stage 5, unchanged**, and stage 0 still needs usm itself installed
|
|
|
+ (any host-glibc build, as before).
|
|
|
+
|
|
|
+What B buys: ch5/6/ch7 of the book (30 of the 115 package pages per
|
|
|
+the §4.1 count — including the three temporary gcc passes, the two
|
|
|
+temporary binutils passes, temporary glibc and libstdc++, and ~24
|
|
|
+temporary library/tool builds) and the majority of the curated-override
|
|
|
+pain **disappear entirely**. Every
|
|
|
+package becomes a single-pass native build into a fresh DESTDIR —
|
|
|
+exactly USM's staged-install model (§3.1) — from day one. The version
|
|
|
+soundness concern of §2.3 is satisfied in the other direction: stage3's
|
|
|
+glibc 2.43/gcc 15.3 are *older* than the targets we would build
|
|
|
+(glibc 2.44 exists as `sys-libs/glibc-2.44-r2.ebuild` in the repo), so
|
|
|
+building newer-from-older is the sound direction; no
|
|
|
+"accidental forward-dependence".
|
|
|
+
|
|
|
+**C. Gentoo ebuilds as the package-definition source.** The ebuild repo
|
|
|
+(`github.com/gentoo/gentoo`, cloned at master `c2cede5`, 2026-09-23) is
|
|
|
+the machine-readable layer LFS lacks. Read this session:
|
|
|
+
|
|
|
+- `app-shells/bash/bash-5.3_p20.ebuild` (486 lines): `DEPEND`/
|
|
|
+ `RDEPEND`/`BDEPEND` are declarative dependency atoms
|
|
|
+ (`>=sys-libs/ncurses-5.2-r2:=`, `readline? ( >=sys-libs/readline-8.3:= )`)
|
|
|
+ — an almost mechanical translation to USM's `depends` resource refs,
|
|
|
+ *after* USE-flag resolution. Phase functions are real shell:
|
|
|
+ `src_configure()` builds a `myconf` array from `$(use_enable …)`
|
|
|
+ expansions and calls `econf "${myconf[@]}"`.
|
|
|
+- `sys-libs/glibc/glibc-2.44-r2.ebuild` (1,934 lines): BDEPEND names
|
|
|
+ bison, pax-utils, texinfo, python (`${PYTHON_DEPS}`); the same file
|
|
|
+ drives native, cross (`cross-*` category) and headers-only builds —
|
|
|
+ evidence that the *final*-system recipe we need is one USE-case of it.
|
|
|
+- `dev-lang/vala-0.56.19`, `dev-libs/glib-2.88.3`, `dev-libs/libgee-0.20.8`,
|
|
|
+ `dev-libs/json-glib-1.10.8`, `net-misc/curl-8.22.0`,
|
|
|
+ `dev-libs/libsodium-1.0.22`, `sys-libs/libselinux-3.11-r1`,
|
|
|
+ `dev-libs/gobject-introspection-1.86.0`: **every beyond-LFS dependency
|
|
|
+ of §6 Stage 5 has a current ebuild** (only invercargill and
|
|
|
+ invercargill-json do not — they are USM's own libraries, packaged from
|
|
|
+ USM's tree either way). A census of the ~45 core system packages
|
|
|
+ (coreutils, shadow, util-linux, procps, kbd, e2fsprogs, sysvinit,
|
|
|
+ systemd, perl, python, texinfo, …) found all present as ebuilds.
|
|
|
+- **The catch, measured:** USE-flag conditional expansion. curl's
|
|
|
+ `IUSE` declares 16 flags whose conditionals (`flag? ( … )`) appear
|
|
|
+ 36 times across its dependency and configure lines; glib's
|
|
|
+ BDEPEND pulls docbook-xsl stylesheets, libxslt, setuptools,
|
|
|
+ `dev-python/docutils`. A converter must (i) parse bash-with-Gentoo-
|
|
|
+ helpers (portage is pip-installable — pypi.org/simple/portage returns
|
|
|
+ 200 — and its Python API can source/parse ebuilds), (ii) pick a
|
|
|
+ frozen USE profile per package, (iii) translate `econf`/`emake`/
|
|
|
+ `meson_src_install` (which already does `--destdir "${D}"`, the
|
|
|
+ DESTDIR-staging USM wants) into `usm-scripts/build.sh`+`install.sh`.
|
|
|
+ That is the *same order of effort* as the LFS book-mining generator
|
|
|
+ (~646 command blocks vs ~90 ebuilds × USE-resolution + eclass
|
|
|
+ semantics), **not** a free lunch.
|
|
|
+
|
|
|
+**D. Hybrid: stage3 seed + ebuild-derived definitions (the shape to
|
|
|
+beat).** Stage 0 = Gentoo stage3 (path B); package definitions for the
|
|
|
+~90-system-package set converted from ebuilds with a frozen-USE policy
|
|
|
+(path C, applied only to what the distro actually ships); stages 2–6 of
|
|
|
+§6 unchanged — they are already pure USM mechanics (USM repo, DCBA
|
|
|
+repos, USM-driven chroot build, beyond-LFS set, OCI assembly). Note
|
|
|
+what stays identical: the beyond-LFS stage-5 set, the DCBA publish
|
|
|
+driver, the orchestrator, the OCI assembler. Only two things change
|
|
|
+versus the LFS plan: stage 0 is a downloaded tarball instead of a dnf
|
|
|
+recipe, and the generator reads ebuilds instead of DocBook.
|
|
|
+
|
|
|
+**E. Other derivations, one paragraph each.**
|
|
|
+
|
|
|
+- **Arch PKGBUILDs** (probed: bash and pacman PKGBUILDs read from
|
|
|
+ gitlab.archlinux.org this session). The cleanest definitions of any
|
|
|
+ distro: `depends=(readline glibc …)`, `build()` and `package()` are
|
|
|
+ plain bash, `package()` installs via `DESTDIR="$pkgdir"` — the most
|
|
|
+ direct mechanical mapping to USM manifests + scripts of everything
|
|
|
+ surveyed. All of USM's stack is packaged in official repos (glib2
|
|
|
+ 2.88.3 core; vala 0.56.19, libgee 0.20.8, json-glib 1.10.8,
|
|
|
+ libsodium 1.0.22, curl 8.22.0 in extra — verified via the package
|
|
|
+ JSON API). Two problems: the definitions are *consumed* by
|
|
|
+ makepkg/pacman tooling that assumes a full Arch root (makedepends
|
|
|
+ resolved by pacman against binary repos — using them means either
|
|
|
+ bootstrapping from Arch binaries (path F) or reimplementing the
|
|
|
+ dependency resolution the ebuild/atom world already gives us), and
|
|
|
+ there is no Arch-published *toolchain-complete rootfs tarball*
|
|
|
+ equivalent to stage3 (archlinux-bootstrap exists but is unofficial
|
|
|
+ and minimal). As a definition source for a from-source build it is
|
|
|
+ genuinely attractive; as an ecosystem to lean on it is thinner than
|
|
|
+ Gentoo's.
|
|
|
+- **Alpine APKBUILDs** (probed: musl's APKBUILD read from
|
|
|
+ aports this session). Bash-based like PKGBUILD but with `abuild`
|
|
|
+ conventions and — visible right in the probed file — a `case
|
|
|
+ "$BOOTSTRAP"` block switching musl between `nocc`/`nolibc`/full
|
|
|
+ variants: Alpine's definitions are *themselves* multi-pass bootstrap
|
|
|
+ recipes, which reintroduces exactly the complexity path B removes.
|
|
|
+ Switching libc to musl would also diverge USM's binary compatibility
|
|
|
+ from the glibc world its current build/test hosts live in.
|
|
|
+- **Fedora spec files / dnf base.** Closest to the current host and
|
|
|
+ already the §6 stage-0 recipe; spec is RPM-centric (macros, rpmbuild
|
|
|
+ phases), and the definitions are less useful than ebuilds for
|
|
|
+ from-source staging because they assume RPM's file-listing and
|
|
|
+ scriptlet model. Fine as the *seed* of last resort, already covered.
|
|
|
+- **Debian debian/rules.** Slow-moving, debhelper abstraction layers
|
|
|
+ (dh_*) are heavily implicit — a converter would have to emulate
|
|
|
+ debhelper's implicit behaviour, a worse translation target than
|
|
|
+ eclasses or PKGBUILDs.
|
|
|
+- **Void templates, Nix expressions.** Void's templates are fine but
|
|
|
+ the ecosystem is small and xbps-bootstrap has the same multi-pass
|
|
|
+ character as Alpine. Nix is a category error here: expressions
|
|
|
+ produce a binary store with fixed paths, not staged file trees with
|
|
|
+ prefixes — the packaging model clash with USM's resource model
|
|
|
+ (destinations, provides refs) is total; only worth revisiting as a
|
|
|
+ *consumer* of USM artifacts.
|
|
|
+- **jhalfs / LFS automation tooling.** Automates the LFS book faithfully
|
|
|
+ — but it automates *the multi-pass book*, so it inherits path A's
|
|
|
+ cost structure while giving us no USM-native artifacts. It would
|
|
|
+ only ever be a reference implementation to crib commands from.
|
|
|
+
|
|
|
+**F. Thin USM layer (state the tradeoff, don't bury it).** Take any
|
|
|
+distro image (stage3, Fedora, Arch), enrol it, `usm` manages only new
|
|
|
+packages. Time to *something*: **days**. Purity: **P1** — the base was
|
|
|
+never USM-built; `usm list` manages a small delta. It is a legitimate
|
|
|
+*v0 demo* of USM-on-a-distro and everything learned (repos, DCBA, keys)
|
|
|
+carries forward, but it does not satisfy "a USM based distro" any more
|
|
|
+than a package manager running on Fedora makes Fedora USM-based. Worth
|
|
|
+doing as a one-day smoke milestone inside whichever path is chosen, not
|
|
|
+as the destination.
|
|
|
+
|
|
|
+### 5.3 Comparison
|
|
|
+
|
|
|
+- **A (LFS-only):** effort 7–10 wk; wall clock 3–5 h/run; risk: moderate
|
|
|
+ (book-mining + host gaps + multi-pass orchestration); purity: P2→P3
|
|
|
+ (every distro file USM-built; host is dnf-built); time-to-first-P2:
|
|
|
+ ~6–9 weeks.
|
|
|
+- **B (stage3 seed):** effort 4.5–6.5 wk; wall clock 2.5–4 h/run (the
|
|
|
+ deleted temporary-toolchain passes are disproportionately serial, so
|
|
|
+ the wall-clock saving exceeds their raw SBU share — derivation below);
|
|
|
+ risk: lower on toolchain (it arrives prebuilt and known-good), same on
|
|
|
+ /dev//proc sandbox items (§9); purity: identical to A (stage 0 is
|
|
|
+ build *host* scaffolding either way — dnf-assembled in A,
|
|
|
+ Gentoo-assembled in B); time-to-P2: ~4–5.5 weeks.
|
|
|
+- **C (ebuild definitions only, LFS-style host):** effort ~8–11 wk
|
|
|
+ (generator complexity moves, does not shrink); not recommended alone.
|
|
|
+- **D (hybrid B+C):** effort 4.5–6 wk; wall clock as B; risk: lowest of
|
|
|
+ the from-source paths that still yields P2; purity P2, and P3 remains
|
|
|
+ reachable later by swapping stage 0 for the distro's own stage-0
|
|
|
+ packages (the stage-0-for-stage-0 step, a known LFS-style sub-project);
|
|
|
+ time-to-P2: **~3.5–5 weeks** — the definitions arrive pre-reviewed by
|
|
|
+ the Gentoo ecosystem and the toolchain arrives prebuilt.
|
|
|
+- **E (other):** Arch is the best *definition* source but the weakest
|
|
|
+ *seed* ecosystem; Alpine/Fedora/Debian/Void/Nix/jhalfs each lose on
|
|
|
+ at least one axis (multi-pass reintroduction, abstraction impedance,
|
|
|
+ model clash) — see §5.2 E for the per-path verdicts.
|
|
|
+- **F (thin layer):** effort days; purity P1; a milestone inside D, not
|
|
|
+ a destination.
|
|
|
+
|
|
|
+The wall-clock numbers for B/D deserve their derivation, because the
|
|
|
+first-cut intuition (delete half the book, halve the time) is wrong and
|
|
|
+we should not plan on it. Summing the SBU entities in `packages.ent`
|
|
|
+(114 entities this session): the temporary passes (ch5/6/7) total
|
|
|
+**~21 SBU** against **~100 SBU** for the final system — only **~18%**
|
|
|
+of the SBU table, because ch8's final gcc (53 SBU) and glibc dominate
|
|
|
+and are built in *both* paths. The honest expectation is therefore
|
|
|
+**~15–25% wall-clock saving**, not half: 3–5 h → **~2.5–4 h**. The
|
|
|
+saving slightly exceeds the 18% SBU share because the temporary chain
|
|
|
+(binutils-p1 → gcc-p1 → ch5 libs → gcc-p2 → …) is the book's most
|
|
|
+serial, least `-j`-compressible stretch, with its own configure/install
|
|
|
+phases and the /tools adjust steps; but no more than that. Sanity
|
|
|
+bound: D's build set is a strict subset of A's, so it can never be
|
|
|
+*slower*. **The real advantage of B/D is engineering time, not build
|
|
|
+time** — the generator never has to express the multi-pass semantics
|
|
|
+at all.
|
|
|
+
|
|
|
+### 5.4 Recommendation
|
|
|
+
|
|
|
+**Switch the implementation plan to path D (Gentoo stage3 seed +
|
|
|
+ebuild-derived definitions), keeping the LFS plan as the documented
|
|
|
+fallback and the reference for book-grade command sequences.** Reasoning:
|
|
|
+
|
|
|
+1. **It is faster for the same purity, mostly in engineering time.**
|
|
|
+ B/D reach the same P2 outcome as A — a root whose every file was
|
|
|
+ placed by a USM transaction — in roughly **two-thirds of the
|
|
|
+ engineering time** (~4.5–6 wk vs 7–10 wk) and ~15–25% less wall
|
|
|
+ clock per run, by deleting the multi-pass toolchain construction
|
|
|
+ that consumes the generator's hardest inputs (gcc/binutils/glibc
|
|
|
+ multi-pass overrides) — the build-time saving is real but modest
|
|
|
+ (§5.3 derivation).
|
|
|
+2. **It matches the human's stated preferences.** "I like Gentoo and
|
|
|
+ don't mind leveraging it, its base image, or its package
|
|
|
+ definitions" — path D uses exactly its base image (stage3 as the
|
|
|
+ build appliance) and its package definitions (ebuilds as the
|
|
|
+ definition source), while the *distro itself* remains USM-native:
|
|
|
+ its packages are `.usmc` in USM repos, its binaries distributed via
|
|
|
+ DCBA, its root assembled by USM transactions. Gentoo is the
|
|
|
+ *parent's kitchen*, not the product.
|
|
|
+3. **It removes, rather than adds, risk.** The riskiest LFS items —
|
|
|
+ toolchain-multi-pass orchestration, host ch2 gaps, book-command
|
|
|
+ mining — are all stage-0/definition-source concerns that stage3 +
|
|
|
+ ebuilds make somebody else's maintained artifact. The remaining
|
|
|
+ risks (sandbox /dev, systemd-vs-sysvinit, ed25519 keys, valac
|
|
|
+ ordering) are identical in A and D (§9).
|
|
|
+4. **It is reversible.** Stages 2–6 of §6 are shared verbatim. If the
|
|
|
+ ebuild converter disappoints for some package, the LFS book's
|
|
|
+ commands for that package can be transplanted into its scaffold
|
|
|
+ one-for-one — the two definition sources are per-package
|
|
|
+ interchangeable, so the fallback is per-package, not wholesale.
|
|
|
+5. **A thin layer (F) first is still worth one day**: enrol the stage3
|
|
|
+ tarball, add the repos, install one trivial package — a
|
|
|
+ horizontal smoke test of the USM/DCBA machinery before the
|
|
|
+ from-source pipeline work begins.
|
|
|
+
|
|
|
+Purity note for the record: D ships a P2 distro. If P3 (a root whose
|
|
|
+*build host* was also USM-built) is ever required, the path is
|
|
|
+well-defined — rebuild the ~297-package stage3-equivalent set as USM
|
|
|
+packages using the LFS-plan machinery, i.e. fall back to path A for
|
|
|
+stage 0 only. That is a follow-up goal, not a blocker.
|
|
|
+
|
|
|
+## 6. The staged architecture
|
|
|
|
|
|
One diagram, then per-stage detail. `H` is the build host (this sandbox or
|
|
|
a full machine); every stage's output feeds the next.
|
|
|
@@ -534,7 +834,7 @@ digests. Optionally split `/usr` vs `/etc/var` into two layers for
|
|
|
cache-friendlier future updates. On podman hosts: `podman load`/`run` for
|
|
|
verification, or skip assembly entirely and use `usm manifest deploy`.
|
|
|
|
|
|
-## 6. Why staged (the resolver argument, precisely)
|
|
|
+## 7. Why staged (the resolver argument, precisely)
|
|
|
|
|
|
`Resolver.resolve` (src/lib/Resolver.vala:233) computes ONE resolved set
|
|
|
with ONE topological order over `manage`+`build`+`runtime` refs
|
|
|
@@ -552,7 +852,7 @@ USM defect — it is the honest boundary between a package manager and a
|
|
|
distro builder, and it keeps every stage debuggable with stock `usm`
|
|
|
commands and stock resolver error reports.
|
|
|
|
|
|
-## 7. Effort estimates
|
|
|
+## 8. Effort estimates
|
|
|
|
|
|
### 7.1 Wall-clock (build time)
|
|
|
|
|
|
@@ -591,6 +891,12 @@ one full stage-0→4 pass **on this 28-core sandbox**:
|
|
|
invercargills and USM itself well under 1 SBU each at j28).
|
|
|
- A DCBA-cached re-run of stage 4 (iteration loop): **~20-40 min** — this
|
|
|
is the number that matters for daily development.
|
|
|
+- Hybrid track (§5.3), for comparison: deleting the ch5/6/7
|
|
|
+ temporary-toolchain passes removes ~21 of ~122 total SBU (~18%:
|
|
|
+ 21.4/121.8 summed from `packages.ent`), and since that chain is the
|
|
|
+ most serial stretch of the build, the honest wall-clock saving is
|
|
|
+ ~15-25%: **~2.5-4 h without test suites** — a strict subset of this
|
|
|
+ plan's build set, hence bounded above by the 3-5 h figure.
|
|
|
|
|
|
Disk per full run: sources ~1 GB, build trees ~40-80 GB, artifacts
|
|
|
(build.tar.xz set) ~15-25 GB, final root ~4-8 GB. Budget ~200 GB per
|
|
|
@@ -599,8 +905,9 @@ zero runs if any of it lands on `/` (17 GB — see §2.2).
|
|
|
|
|
|
### 7.2 Engineering time
|
|
|
|
|
|
-Per-item figures from §4; consolidated (one engineer, familiar with both
|
|
|
-USM and LFS):
|
|
|
+Per-item figures from §4; consolidated for the **LFS track** (one
|
|
|
+engineer, familiar with both USM and LFS). The hybrid-track breakdown
|
|
|
+lives in §10; its total (~4.5-6 wk) is the §5.3 comparison figure.
|
|
|
|
|
|
- Stage 0 recipe + OCI assembler + verify harness: **3-5 days**
|
|
|
- Generator (book → scaffolds, with overrides table): **2-3 weeks**
|
|
|
@@ -620,9 +927,9 @@ packages × curated overrides vs mostly-mechanical).
|
|
|
Uncertainty: a first full run will surface 5-15 per-package script bugs
|
|
|
(extraction artefacts, env differences) — that's inside the integration
|
|
|
line above; a systemd-revision build adds ~3-5 days if pursued (and needs
|
|
|
-a full host for /proc reasons, §8).
|
|
|
+a full host for /proc reasons, §9).
|
|
|
|
|
|
-## 8. Risk register
|
|
|
+## 9. Risk register
|
|
|
|
|
|
- **Device nodes in /dev** (no CAP_MKNOD). Plain-file `/dev/null` works
|
|
|
for most builds (proven with bison); `/dev/zero`, `/dev/random`,
|
|
|
@@ -677,9 +984,53 @@ a full host for /proc reasons, §8).
|
|
|
root and dnf5 this is a non-event: stage 0 installs them, and a
|
|
|
ch2-check script fails fast on any host still missing items.
|
|
|
|
|
|
-## 9. Recommended task breakdown (for the implementation goal)
|
|
|
-
|
|
|
-Ordered; each is independently verifiable and lands value on its own:
|
|
|
+## 10. Recommended task breakdown (for the implementation goal)
|
|
|
+
|
|
|
+Two tracks, reflecting §5: the **recommended hybrid track** (Gentoo
|
|
|
+stage3 seed + ebuild-derived definitions; §5.4) and the **LFS fallback
|
|
|
+track** (the original plan, kept verbatim — it is also the P3-purity
|
|
|
+upgrade path). Tasks 4-8 are shared by both tracks (stages 2-6 of §6
|
|
|
+are definition-source-agnostic); only the seed (task 1) and the
|
|
|
+generator (tasks 2-3) differ.
|
|
|
+
|
|
|
+**Hybrid track (recommended, ~4.5-6 wk total):**
|
|
|
+
|
|
|
+1. **Stage-0 from Gentoo stage3 + enrol + OCI assembler** (2-4 d) —
|
|
|
+ download/verify the PGP-signed `latest-stage3-amd64-openrc` tarball,
|
|
|
+ unpack (device-node loss acknowledged, §5.2 B), install usm, enrol
|
|
|
+ the root as a build appliance; one-day "thin layer" smoke milestone
|
|
|
+ on top (§5.2 F): repos + keys + one trivial install, proving the
|
|
|
+ USM/DCBA machinery horizontally before any from-source work.
|
|
|
+2. **Ebuild converter v1: dependency atoms → MANIFEST.usm** (1-1.5 wk) —
|
|
|
+ portage (pip-installable) sources each ebuild; resolve USE against a
|
|
|
+ frozen per-package profile; emit DEPEND/BDEPEND/RDEPEND as USM
|
|
|
+ resource-ref depends. Deliverable: ~90 manifests + the ~10
|
|
|
+ beyond-LFS ones (glib, vala, libgee, json-glib, gobject-introspection,
|
|
|
+ libsodium, libselinux, curl; invercargill ×2 hand-packaged from USM's
|
|
|
+ tree) packaging cleanly.
|
|
|
+3. **Ebuild converter v2: phase functions → usm-scripts** (1-1.5 wk) —
|
|
|
+ translate `src_configure`/`src_compile`/`src_install` (econf/emake,
|
|
|
+ meson/ninja with `--destdir`, plain `./configure`) into
|
|
|
+ build.sh/install.sh with a per-package fallback to hand-curated
|
|
|
+ scripts cribbed from the LFS book where the eclass semantics resist
|
|
|
+ conversion (per-package fallback is cheap, §5.4 point 4).
|
|
|
+4. **USM repo modelling + resolution dry-runs** (1 wk) — shared;
|
|
|
+ resolver-validated dependency graph, zero UNSATISFIABLE.
|
|
|
+5. **Build all ~100 packages single-pass inside the stage-3 chroot**
|
|
|
+ (1 wk) — every package builds into a fresh DESTDIR from day one (no
|
|
|
+ /tools passes); publish the USM repo + `.dcba` repos as it goes.
|
|
|
+6. **Final root via USM transactions + beyond-LFS + USM self-host**
|
|
|
+ (4-6 d) — fresh root, DCBA-accelerated install of the whole set,
|
|
|
+ then stage-5's order (glib → … → USM); milestone: `usm install usm`
|
|
|
+ inside the final root.
|
|
|
+7. **Final OCI image + smoke tests + docs** (2-3 d) — assembled image,
|
|
|
+ chroot/podman verification matrix.
|
|
|
+8. **(Follow-up, separate goal) systemd revision, kernel package, https
|
|
|
+ repo hosting, ResourceFinder strict-destination mode, P3 stage 0
|
|
|
+ (USM-built build host, i.e. the LFS fallback applied to stage 0
|
|
|
+ only).**
|
|
|
+
|
|
|
+**LFS fallback track (original plan, ~7-10 wk total):**
|
|
|
|
|
|
1. **stage0 recipe + OCI assembler + ch2 check** (3-5 d) — produce a
|
|
|
bootstrap image from dnf; verify by chroot + book ch2 script.
|
|
|
@@ -691,7 +1042,7 @@ Ordered; each is independently verifiable and lands value on its own:
|
|
|
table** (1.5-2 wk) — ch5/6 (tmp passes, /tools env), ch7, ch8;
|
|
|
curated handling for gcc/binutils/glibc/systemd|sysvinit/perl/python.
|
|
|
4. **USM repo modelling + stage tags + resolution dry-runs** (1 wk) —
|
|
|
- resolver-validated dependency graph per stage, zero UNSATISFIABLE.
|
|
|
+ as above (plus stage tags tmp0/tmp1/chroot/final).
|
|
|
5. **Stage orchestrator + stage-2/3 build of tmp/chroot stages** (1 wk) —
|
|
|
first real chroot builds inside stage-0; publish tmp/chroot `.dcba`
|
|
|
repos.
|
|
|
@@ -704,14 +1055,28 @@ Ordered; each is independently verifiable and lands value on its own:
|
|
|
9. **(Follow-up, separate goal) systemd revision, kernel build, https
|
|
|
repo hosting, ResourceFinder strict-destination mode.**
|
|
|
|
|
|
-## 10. Conclusion
|
|
|
+## 11. Conclusion
|
|
|
|
|
|
The goal is feasible with current USM master: the staged architecture
|
|
|
uses only documented, e2e-tested mechanics (staged installs, file://
|
|
|
-repos, DCBA artifact distribution), the LFS book is machine-minable
|
|
|
-enough for a generator with a modest overrides table, and the two hard
|
|
|
-environmental constraints of this sandbox (no device-node creation, no
|
|
|
-mounts) are worked around for every stage except a systemd build. The
|
|
|
-recommended first implementation milestone is tasks 1+2: a verified
|
|
|
-bootstrap image and ~90 packaging-clean scaffolds — roughly a week of
|
|
|
-work that de-risks everything downstream.
|
|
|
+repos, DCBA artifact distribution), and the two hard environmental
|
|
|
+constraints of this sandbox (no device-node creation, no mounts) are
|
|
|
+worked around for every stage except a systemd build.
|
|
|
+
|
|
|
+Answering the review question directly — *is LFS the best approach?* —
|
|
|
+**no, not for the stated goal** ("rapidly stand something up that truly
|
|
|
+uses USM"). LFS remains the best *purity reference* and the fallback,
|
|
|
+and its book is machine-minable enough for a generator with a modest
|
|
|
+overrides table; but §5 shows the Gentoo hybrid reaches the same
|
|
|
+USM-built, USM-managed root in roughly two-thirds of the engineering
|
|
|
+time and ~15–25% less wall clock, using Gentoo's weekly-published
|
|
|
+stage3 as the build appliance and its ebuild repository as a
|
|
|
+pre-reviewed definition source, while every artifact of the distro
|
|
|
+itself is a USM package in a USM repo, distributed via DCBA, assembled
|
|
|
+by USM transactions. The
|
|
|
+recommended first implementation milestone is hybrid tasks 1+2: a
|
|
|
+verified stage3-based build appliance with the thin-layer smoke test
|
|
|
+passing, plus the dependency-atom converter producing packaging-clean
|
|
|
+manifests — roughly a week of work that de-risks everything downstream.
|
|
|
+If P3 purity (a USM-built build host) is later required, the LFS
|
|
|
+fallback track applies to stage 0 alone.
|