|
@@ -0,0 +1,648 @@
|
|
|
|
|
+# USM Distro Bootstrap — Feasibility & Effort Report
|
|
|
|
|
+
|
|
|
|
|
+Investigation report for goal 15 ("a USM based Linux distro"), covering every
|
|
|
|
|
+item of the brief: a scaffold repository for LFS bootstrap packages, a
|
|
|
|
|
+bootstrap container, a USM repository of LFS packages, a DCBA repository of
|
|
|
|
|
+LFS build artifacts, a USM-driven chroot build, and a container of the final
|
|
|
|
|
+root with USM installed.
|
|
|
|
|
+
|
|
|
|
|
+Verdict up front: **feasible, with an explicitly staged architecture, and
|
|
|
|
|
+conditioned on the build host.** USM's existing mechanics (staged installs
|
|
|
|
|
+via `USM_DESTDIR`/`USM_CONFIGDIR`, destination-local resource resolution,
|
|
|
|
|
+`file://` repositories, the DCBA build-artifact flow) map well onto an LFS
|
|
|
|
|
+bootstrap — but USM's resolver is single-pass, while LFS is a three-pass
|
|
|
|
|
+toolchain construction (temporary → chroot → final), so the bootstrap cannot
|
|
|
|
|
+be one USM resolution. It must be staged: a handful of driver invocations,
|
|
|
|
|
+each a valid single-pass resolution, chained by an orchestrator. The dominant
|
|
|
|
|
+risks are environmental, not architectural: device nodes and `/proc` inside a
|
|
|
|
|
+capability-restricted chroot, systemd's build requirements, and self-hosting
|
|
|
|
|
+USM itself (valac + glib are not in the LFS book).
|
|
|
|
|
+
|
|
|
|
|
+Everything below was verified against the USM checkout at commit `b046afe`
|
|
|
|
|
+("Merge branch 'goal/dcba' of Tilo15/usm into master", branch `master`) and
|
|
|
|
|
+the LFS book checkout at branch `13.1` (LFS 13.1, `trunk` working copy),
|
|
|
|
|
+plus live probes of this build sandbox. File/line citations refer to those
|
|
|
|
|
+checkouts.
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## 1. Executive summary
|
|
|
|
|
+
|
|
|
|
|
+- **Feasible.** Every one of the six brief items maps onto USM mechanics that
|
|
|
|
|
+ exist on `master` today; none requires new USM core features. What must be
|
|
|
|
|
+ built is an *orchestration layer* (a generator that turns the LFS book into
|
|
|
|
|
+ USM scaffolds, stage drivers, and OCI assembly scripts) plus a small
|
|
|
|
|
+ beyond-LFS package set so USM can build itself.
|
|
|
|
|
+- **The architecture must be staged.** `Resolver.resolve`
|
|
|
|
|
+ (src/lib/Resolver.vala:232) processes `manage`+`build`+`runtime` phases in
|
|
|
|
|
+ one pass and produces one topological order
|
|
|
|
|
+ (src/lib/Resolver.vala:422-428) — a single resolution cannot express LFS's
|
|
|
|
|
+ ch5/6 temporary tools → ch7 chroot tools → ch8 final system sequence, where
|
|
|
|
|
+ later packages must be built *with* the products of earlier ones. The
|
|
|
|
|
+ bootstrap is therefore N driven stages, each internally a normal USM
|
|
|
|
|
+ install; the stages are ordered by an orchestrator script, not by the
|
|
|
|
|
+ resolver.
|
|
|
|
|
+- **Host-conditioned.** On a sandbox without `CAP_SYS_ADMIN`/`CAP_MKNOD`
|
|
|
|
|
+ (this one), chroot builds work but `/dev` nodes are plain files and
|
|
|
|
|
+ `/proc`/`/sys` are empty. That is fine for the great majority of LFS
|
|
|
|
|
+ packages (proven: bison 3.8.2 configured and compiled inside a host-root
|
|
|
|
|
+ clone with exactly that setup) but endangers the systemd build and anything
|
|
|
|
|
+ reading `/proc/mounts`. Final artifacts should be produced on a full host
|
|
|
|
|
+ (or a rootful container); the sandbox remains excellent for development of
|
|
|
|
|
+ the pipeline itself.
|
|
|
|
|
+- **Disk layout matters**: `/` (and `/tmp`) is a 17 GB overlay while
|
|
|
|
|
+ `/workspace` has ~11 TB free — all LFS scratch trees must live under
|
|
|
|
|
+ `/workspace`. (The host also lacks `g++`/`patch` — two chapter-2
|
|
|
|
|
+ requirements — but the sandbox has root and dnf5, so stage 0's image
|
|
|
|
|
+ recipe closes that with one `dnf install` line; see §2.3.)
|
|
|
|
|
+- **Effort.** Roughly **6–9 engineer-weeks** for the pipeline (generator +
|
|
|
|
|
+ stage drivers + OCI scripts + key management + CI), of which the single
|
|
|
|
|
+ largest item is the book-to-scaffold generator (94 packages, ~640 command
|
|
|
|
|
+ blocks, needing curated per-package handling). Wall-clock for one full
|
|
|
|
|
+ bootstrap run on this 28-core sandbox: **~3–5 hours without test suites,
|
|
|
|
|
+ ~8–14 hours with** (SBU-derived; see §7). Artifacts published per run:
|
|
|
|
|
+ ~94 `.usmc` source packages, ~94 `.dcba` build artifacts, two signed
|
|
|
|
|
+ listings.
|
|
|
|
|
+
|
|
|
|
|
+## 2. Verified platform facts
|
|
|
|
|
+
|
|
|
|
|
+Every probe below was run live in this session (2026-09-23) unless marked
|
|
|
|
|
+"inherited from the goal-15 investigation"; re-running any of them is a
|
|
|
|
|
+one-liner.
|
|
|
|
|
+
|
|
|
|
|
+### 2.1 Capabilities and kernel
|
|
|
|
|
+
|
|
|
|
|
+- `grep CapEff /proc/self/status` → `00000000800405fb`;
|
|
|
|
|
+ `capsh --decode` → `cap_chown, cap_dac_override, cap_fowner, cap_fsetid,
|
|
|
|
|
+ cap_kill, cap_setgid, cap_setuid, cap_setpcap, cap_net_bind_service,
|
|
|
|
|
+ cap_sys_chroot, cap_setfcap`.
|
|
|
|
|
+ **`cap_sys_chroot` present; `cap_mknod`, `cap_sys_admin`,
|
|
|
|
|
+ `cap_net_admin` absent** (CapBnd equals CapEff — they cannot be gained).
|
|
|
|
|
+ Consequences: `chroot(2)` works; `mknod(2)` fails (`Operation not
|
|
|
|
|
+ permitted`); no mounts, no user/pid namespaces, no overlayfs.
|
|
|
|
|
+- Kernel `7.0.12-101.fc43.x86_64` (uname -r) — comfortably above the book's
|
|
|
|
|
+ minimum kernel requirement (the book specifies >= 5.10 for glibc's
|
|
|
|
|
+ `--enable-kernel` reasoning; `chapter02/hostreqs.xml`).
|
|
|
|
|
+- chroot re-proven this session: a minimal root (`/usr/bin/true` + its
|
|
|
|
|
+ `ldd` closure + the dynamic loader) executed via `chroot $W /usr/bin/true`
|
|
|
|
|
+ → "CHROOT EXEC OK".
|
|
|
|
|
+- Inherited from the goal-15 investigation (not re-run): a clone of the host
|
|
|
|
|
+ root with a plain-file `/dev/null` and empty `/proc` configured and
|
|
|
|
|
+ **compiled GNU bison 3.8.2 with gcc inside chroot**; the only fix needed
|
|
|
|
|
+ was `PATH` reconstruction under `env -i`. Also inherited: `tar` cannot
|
|
|
|
|
+ carry device nodes into this sandbox (`Cannot mknod: Operation not
|
|
|
|
|
+ permitted`).
|
|
|
|
|
+
|
|
|
|
|
+### 2.2 Resources
|
|
|
|
|
+
|
|
|
|
|
+- 28 cores (`nproc`).
|
|
|
|
|
+- Storage: `/workspace` on `/dev/sdc1` — 19 T total, **11 T free**;
|
|
|
|
|
+ `/` is an overlay with only **17 G free** (so `/tmp` is tight — LFS
|
|
|
|
|
+ scratch trees must live under `/workspace`).
|
|
|
|
|
+- Network: outbound HTTPS works (`curl -sI https://ftp.gnu.org/gnu/` →
|
|
|
|
|
+ `HTTP/1.1 200 OK`).
|
|
|
|
|
+
|
|
|
|
|
+### 2.3 Toolchain versions and version soundness
|
|
|
|
|
+
|
|
|
|
|
+- Host gcc `16.2.1 20260819 (Red Hat 16.2.1-2)`, host glibc `2.43`
|
|
|
|
|
+ (`ldd --version`), Python 3.14.7, perl 5.42.3, bison 3.8.2.
|
|
|
|
|
+- Book targets (reference/LFS-Stable `packages.ent`): gcc `16.2.0`, glibc
|
|
|
|
|
+ `2.44`, binutils `2.47`, sysvinit `3.14` / systemd `261.3` (both revisions
|
|
|
|
|
+ shipped in the book).
|
|
|
|
|
+- Soundness for a *native* (non-cross) bootstrap:
|
|
|
|
|
+ - host glibc 2.43 < book glibc 2.44 → the temporary tools never
|
|
|
|
|
+ accidentally link against a *newer* libc than they are building;
|
|
|
|
|
+ - host gcc 16.2 matches the book's 16.2.0 — and crucially the book warns
|
|
|
|
|
+ (`chapter02/hostreqs.xml`) that *newer* gcc/binutils than the book's are
|
|
|
|
|
+ untested and likely to break the build; equal is the intended case;
|
|
|
|
|
+ - kernel 7.0 > minimum → glibc headers configure cleanly.
|
|
|
|
|
+- **Host tool gaps (new this session):** `/usr/bin/g++` and `cc1plus` are
|
|
|
|
|
+ **absent** (`gcc -x c++` fails: "cannot execute 'cc1plus'"), and `patch`
|
|
|
|
|
+ is absent. `find`/`makeinfo` exist where needed. The book's ch2 software
|
|
|
|
|
+ list requires both a C++ compiler and patch. `dnf5 5.4.3.0` is present and
|
|
|
|
|
+ both `gcc-c++.x86_64 16.2.1-2.fc44` (updates) and `patch.x86_64
|
|
|
|
|
+ 2.8-4.fc44` (fedora) are available → the Stage-0 image build must
|
|
|
|
|
+ `dnf install gcc-c++ patch` (plus any other missing ch2 items re-checked
|
|
|
|
|
+ at image-build time).
|
|
|
|
|
+
|
|
|
|
|
+### 2.4 Container runtimes
|
|
|
|
|
+
|
|
|
|
|
+- `podman`, `docker`, `buildah`, `skopeo`, `umoci`: **all absent**. So
|
|
|
|
|
+ containers must be *assembled* (tar layers + config + manifest, i.e. an
|
|
|
|
|
+ OCI image layout) by script here; they can be *consumed* anywhere a
|
|
|
|
|
+ runtime exists. `usm deploy`'s podman path (§4.6) remains available on
|
|
|
|
|
+ hosts that have podman.
|
|
|
|
|
+
|
|
|
|
|
+## 3. USM mechanics the bootstrap relies on (verified on master)
|
|
|
|
|
+
|
|
|
|
|
+These are the load-bearing behaviours; each was read at the cited location
|
|
|
|
|
+in this session.
|
|
|
|
|
+
|
|
|
|
|
+1. **Staged installs.** `Paths.override_from_environment`
|
|
|
|
|
+ (src/lib/Paths.vala:189-207) overrides `destination`, `prefix`, every
|
|
|
|
|
+ dir, and `usm_config_dir` from `USM_DESTDIR`, `USM_PREFIX`,
|
|
|
|
|
+ `USM_BINDIR`, … `USM_CONFIGDIR`. `Manifest.run_build` then calls
|
|
|
|
|
+ `paths.set_envs()` (src/lib/Manifest.vala:289-314), exporting
|
|
|
|
|
+ `DESTDIR/PREFIX/BINDIR/LIBDIR/…` to the build script, and
|
|
|
|
|
+ `Manifest.run_install` clones the paths and overrides `destination`
|
|
|
|
|
+ with the per-package install path (src/lib/Manifest.vala:441-475) before
|
|
|
|
|
+ spawning the install script. `scripts/dcba-e2e.sh` drives two fully
|
|
|
|
|
+ isolated roots this way end-to-end (its header documents the recipe:
|
|
|
|
|
+ `<root>/config` via USM_CONFIGDIR, `<state>/{lists,packages,installed}`
|
|
|
|
|
+ pre-created, `usm.config` with `is_managed=true` +
|
|
|
|
|
+ `managed.state_path`).
|
|
|
|
|
+2. **ResourceFinder local presence.** `ResourceFinder.locate_resource`
|
|
|
|
|
+ (src/lib/ResourceFinder.vala:13-58) resolves refs mostly **relative to
|
|
|
|
|
+ `paths.destination`** (`locate_path`, `locate_res`, `locate_cfg`,
|
|
|
|
|
+ `locate_bin` walks `$PATH` under destination, `locate_libres`,
|
|
|
|
|
+ `locate_pc` including `PKG_CONFIG_PATH` entries, `locate_vapi`,
|
|
|
|
|
+ `locate_typelib`, …), so inside a chroot (destination `/`) resolution
|
|
|
|
|
+ sees only the staged root. **Caveat (a real constraint, not a blocker):
|
|
|
|
|
+ several locators are host-absolute** — `locate_lib` consults
|
|
|
|
|
+ `ldconfig -p` and falls back to absolute `/lib /usr/lib /lib64
|
|
|
|
|
+ /usr/lib64` (src/lib/ResourceFinder.vala:88-126), and `locate_inc`,
|
|
|
|
|
+ `locate_libexec`, `locate_gir`, `locate_info/man/locale`,
|
|
|
|
|
+ `locate_app` use absolute `/usr/...` paths. Inside a chroot that is
|
|
|
|
|
+ correct (destination is `/`); *outside* a chroot with USM_DESTDIR set,
|
|
|
|
|
+ `lib:`/`include:` refs can be satisfied by the host — the stage drivers
|
|
|
|
|
+ must run usm inside the target chroot, or audited for it.
|
|
|
|
|
+3. **Resolver is single-pass.** `Resolver.resolve`
|
|
|
|
|
+ (src/lib/Resolver.vala:232-287) picks roots, does ONE batched SPM query,
|
|
|
|
|
+ processes all packages, and `resolution_phases` folds
|
|
|
|
|
+ `manage`+`build`+`runtime` (src/lib/Resolver.vala:422-428) into one
|
|
|
|
|
+ dependency graph with one topological install order. Missing resources
|
|
|
|
|
+ resolve: (a) local presence, (b) system packages, (c) USM packages —
|
|
|
|
|
+ exactly the semantics a stage needs ("what the previous stage left in
|
|
|
|
|
+ the root counts as present"), but it cannot reorder the world between
|
|
|
|
|
+ two packages in one transaction.
|
|
|
|
|
+4. **file:// repositories.** `Repository.get_client`
|
|
|
|
|
+ (src/lib/Repository/Repository.vala:34-53) routes `file` →
|
|
|
|
|
+ `GioRepositoryClient`, `http`/`https` → `CurlRepositoryClient`
|
|
|
|
|
+ (curl `-fsSL --max-time`). All repositories in the bootstrap can be
|
|
|
|
|
+ plain directories on the same filesystem — no server needed.
|
|
|
|
|
+5. **DCBA flow.** `usm dcba repository init/add` (src/cli/Dcba.vala):
|
|
|
|
|
+ `init` generates an ed25519 keypair under `keys/` (private 0600) and a
|
|
|
|
|
+ `.dcbar` descriptor; `add <file>.dcba` validates tar.xz, derives
|
|
|
|
|
+ name/version from the filename via `DcbaArtifact.parse_filename`
|
|
|
|
|
+ (src/lib/Repository/DcbaArtifact.vala:22-56) and rebuilds a signed
|
|
|
|
|
+ `ARTIFACTS.usml` (SHA-512 over entry lines, ed25519 signatures —
|
|
|
|
|
+ `DcbaListingWriter.write`, src/lib/Repository/DcbaArtifact.vala:82+).
|
|
|
|
|
+ The artifact itself is literally the package cache's `build.tar.xz`
|
|
|
|
|
+ renamed: `scripts/dcba-e2e.sh` copies
|
|
|
|
|
+ `<state>/packages/<n>-<v>/build.tar.xz` to `<n>-<v>.dcba` (its §3),
|
|
|
|
|
+ produced by `CachedPackage.archive_build`
|
|
|
|
|
+ (src/lib/State/CachedPackage.vala:101-131, digest-gated re-archive).
|
|
|
|
|
+ A consumer with the `.dcbar` configured gets the artifact as its build
|
|
|
|
|
+ cache ("Retrieved pre-built artifact"), records provenance in
|
|
|
|
|
+ `origin-info` (`"dcba"` field), and a tampered artifact is rejected with
|
|
|
|
|
+ a warning + fallback to source build — all asserted by the e2e script
|
|
|
|
|
+ (assertions a–f).
|
|
|
|
|
+6. **`usm deploy`.** `usm manifest deploy` (src/cli/Deploy.vala:223-473)
|
|
|
|
|
+ generates a Containerfile-based context and (unless `--no-build`) builds
|
|
|
|
|
+ with podman and saves an xz-compressed image. Default base is Fedora
|
|
|
|
|
+ (src/cli/Deploy.vala:16); `--base` overrides. This is the
|
|
|
|
|
+ "container from the chroot" path on hosts with podman; here we script
|
|
|
|
|
+ OCI assembly instead (Stage 6).
|
|
|
|
|
+7. **Scaffolds.** `usm scaffold <name> <template> [attrs]`
|
|
|
|
|
+ (src/cli/Scaffold.vala) generates `MANIFEST.usm` + `usm-scripts/` for
|
|
|
|
|
+ templates `basic`/`makefile`/`meson` with attributes `vala`/`c`/`acquire`
|
|
|
|
|
+ (the last adds a wget/tar acquire script + deps). Scaffolded manifests
|
|
|
|
|
+ express deps as resource refs (`bin:make`, `pc:glib-2.0.pc`, …), the
|
|
|
|
|
+ same vocabulary the LFS scaffolds will use — but LFS scaffolds will be
|
|
|
|
|
+ *generated*, not hand-scaffolded, and their scripts are bespoke per
|
|
|
|
|
+ package.
|
|
|
|
|
+
|
|
|
|
|
+## 4. The six brief items — analysis
|
|
|
|
|
+
|
|
|
|
|
+### 4.1 Item 1: a repository of USM scaffolds containing all packages needed to bootstrap LFS
|
|
|
|
|
+
|
|
|
|
|
+**Mapping.** One git repository ("the scaffold repo") holding one directory
|
|
|
|
|
+per LFS package, each a USM scaffold: `MANIFEST.usm` (name/version/licences/
|
|
|
|
|
+provides/depends/execs) + `usm-scripts/{acquire.sh,build.sh,install.sh}` +
|
|
|
|
|
+vendored patches. `usm manifest package` turns each into a `.usmc`; the
|
|
|
|
|
+collection is published with `usm repository init/add/build-list`. Nothing
|
|
|
|
|
+here needs new USM features — `Scaffold.vala` already writes this shape;
|
|
|
|
|
+the generator will write it directly (scaffold templates don't fit LFS's
|
|
|
|
|
+per-package configure flags anyway).
|
|
|
|
|
+
|
|
|
|
|
+**What's missing / must be built — the generator.** The book is DocBook
|
|
|
|
|
+XML; the machine-readable part is excellent (`packages.ent`: 718 entities —
|
|
|
|
|
+names, versions, URLs, md5s, SBU/disk estimates as text), but the ~640
|
|
|
|
|
+build steps are prose. Counts from this checkout:
|
|
|
|
|
+
|
|
|
|
|
+- `userinput remap="…"` command blocks across ch5–8: **558** — 105
|
|
|
|
|
+ `configure`, 118 `make`, 180 `install`, 76 `pre`, 76 `test`, 3 `adjust`;
|
|
|
|
|
+ plus **85** un-remapped `<userinput>` (chapter setup commands).
|
|
|
|
|
+- Package-ish pages: ch5 5, ch6 17, ch7 8, ch8 91 files (85 with build
|
|
|
|
|
+ commands) → **~94 source packages** distinct (some pages are
|
|
|
|
|
+ multi-pass: gcc appears in ch5 pass1/pass2 and ch8; binutils pass1/ch8).
|
|
|
|
|
+
|
|
|
|
|
+A naive `<userinput>`-to-shell-literal extraction produces *almost* right
|
|
|
|
|
+scripts — but not right: entities need expansion, `make -j1` /
|
|
|
|
|
+`make -k check` conventions need policy, ch5/6 scripts must hard-code
|
|
|
|
|
+`/tools` prefixes and env (`LFS`, `LFS_TGT`, `PATH=/tools/bin:...`), ch8
|
|
|
|
|
+scripts assume the chroot, and a few packages need hand-massaging (gcc's
|
|
|
|
|
+three passes and its `limitl`, glibc's `libc.picture`-era configure
|
|
|
|
|
+options, systemd's `meson -D*` policy, perl's `groff`-era hints). So: a
|
|
|
|
|
+generator (Python, ~1.5-2k lines) with a **curated overrides table** for
|
|
|
|
|
+the ~15 hairy packages, emitting both the scaffolds and a
|
|
|
|
|
+`stage-manifest.json` the orchestrator consumes. Generated scaffolds are
|
|
|
|
|
+checked into git, reviewed once, then regenerated only on book bumps.
|
|
|
|
|
+
|
|
|
|
|
+**Effort**: generator 2-3 engineer-weeks (including the overrides table and
|
|
|
|
|
+the ch5/6/ch7/ch8 script envs), scaffold review pass 2-4 days.
|
|
|
|
|
+
|
|
|
|
|
+### 4.2 Item 2: build a container into this bootstrap environment
|
|
|
|
|
+
|
|
|
|
|
+**Mapping.** The bootstrap container is the *build host* for stages 1-4: a
|
|
|
|
|
+root filesystem with a chapter-2-compliant toolchain, the `dnf`-installed
|
|
|
|
|
+gaps closed, `usm` + its runtime deps (curl, git, libsodium, glib, json-glib
|
|
|
|
|
+— via dnf on the Fedora base) preinstalled, and the LFS sources either
|
|
|
|
|
+pre-fetched or fetchable. Two ways to produce it:
|
|
|
|
|
+
|
|
|
|
|
+- **(a) dnf-based (recommended for stage 0).** Start from a Fedora
|
|
|
|
|
+ container base (or `dnf --installroot` into a directory on this host —
|
|
|
|
|
+ no runtime needed), `dnf install` the ch2 checklist + `gcc-c++ patch` +
|
|
|
|
|
+ usm's runtime closure, then tar it up. Fully reproducible-ish, fast.
|
|
|
|
|
+- **(b) host-clone (proven here).** Clone the sandbox host root, add the
|
|
|
|
|
+ missing pieces via dnf, prune to the ch2 list. Faster to stand up (the
|
|
|
|
|
+ bison proof used exactly this), less hermetic; fine for development,
|
|
|
|
|
+ not for the published artifact.
|
|
|
|
|
+
|
|
|
|
|
+**What's missing / must be built.** An OCI image *assembler* script: tar
|
|
|
|
|
+the rootfs into one (or a few) layers, write `oci-layout`, `index.json`,
|
|
|
|
|
+`blobs/sha256/…` config+manifest. ~200-300 lines of Python/shell; standard
|
|
|
|
|
+format, no runtime required to *produce*. (On podman hosts, `usm manifest
|
|
|
|
|
+deploy --no-build`'s context or a plain Containerfile is the alternative
|
|
|
|
|
+production path.)
|
|
|
|
|
+
|
|
|
|
|
+**Effort**: image build recipe + assembler 3-5 days, including a
|
|
|
|
|
+verify-by-chroot harness (unpack the image, chroot in, run the ch2
|
|
|
|
|
+version-check script from the book).
|
|
|
|
|
+
|
|
|
|
|
+### 4.3 Item 3: a USM repository of the LFS packages
|
|
|
|
|
+
|
|
|
|
|
+**Mapping.** Exactly `usm repository init lfs && usm repository add *.usmc
|
|
|
|
|
+&& usm repository build-list` (src/cli/Repository.vala): per-package
|
|
|
|
|
+`.usmc` from `usm manifest package`, a `PACKAGES.usml` signed listing
|
|
|
|
|
+(ed25519 keys under `keys/`, same mechanics as DCBA init), served as a
|
|
|
|
|
+`file://` (or later https) repository. Consumers `usm add-repo` it. The
|
|
|
|
|
+only structural question is *granularity of stages*: because the resolver
|
|
|
|
|
+is single-pass (§3.3), the repo should be **one repository with stage
|
|
|
|
|
+tags** — e.g. manifests carry a `stage` extra-property (or separate repos
|
|
|
|
|
+`lfs-tmp0`, `lfs-tmp1`, `lfs-final`; tags are lighter). The orchestrator
|
|
|
|
|
+installs stage by stage; each stage's install is one ordinary `usm install
|
|
|
|
|
+<pkg>...` transaction whose dependencies resolve against what earlier
|
|
|
|
|
+stages already left in the root (local presence, §3.2).
|
|
|
|
|
+
|
|
|
|
|
+**What's missing / must be built.** The stage tags (generator-emitted); a
|
|
|
|
|
+`provides` vocabulary decision for LFS (e.g. `lib:libc.so.6` from glibc,
|
|
|
|
|
+`bin:gcc` from gcc, `pc:…` where the book installs .pc files); discipline
|
|
|
|
|
+that ch5/6 temporary packages *self-contain* under `/tools` and do not
|
|
|
|
|
+`provides`-collide with ch8 finals (they must not — same package name,
|
|
|
|
|
+different version suffix, e.g. `gcc-tmp-16.2.0` vs `gcc-16.2.0`).
|
|
|
|
|
+
|
|
|
|
|
+**Effort**: 1 week, mostly the provides/depends modelling and validating
|
|
|
|
|
+resolution with the real resolver (the itemised `UNSATISFIABLE` reports
|
|
|
|
|
+make this pleasantly debuggable).
|
|
|
|
|
+
|
|
|
|
|
+### 4.4 Item 4: a DCBA repository of the LFS packages
|
|
|
|
|
+
|
|
|
|
|
+**Mapping.** Direct use of the DCBA flow (§3.5): after a build stage, each
|
|
|
|
|
+package cache's `build.tar.xz` becomes `<name>-<version>.dcba`
|
|
|
|
|
+(`cp`, exactly as `scripts/dcba-e2e.sh` §3 does), `usm dcba repository
|
|
|
|
|
+add` rebuilds the signed `ARTIFACTS.usml`, and consumers with the `.dcbar`
|
|
|
|
|
+configured install with pre-built artifacts ("Retrieved pre-built
|
|
|
|
|
+artifact") — turning later stages (and any third party) from compiles into
|
|
|
|
|
+cache-restores. This is precisely the artifact-distribution model the
|
|
|
|
|
+brief asks for, and it is what makes Stage 4 (final chroot build) cheap
|
|
|
|
|
+enough to iterate on.
|
|
|
|
|
+
|
|
|
|
|
+**What's missing / must be built.** A small publish driver (loop over the
|
|
|
|
|
+state dir, copy, add, verify — `dcba repository verify` exists). Real
|
|
|
|
|
+considerations: (i) **the artifact is only valid for one stage's
|
|
|
|
|
+environment** — a ch5 `build.tar.xz` is useless in ch8 — so artifacts must
|
|
|
|
|
+be namespaced (separate DCBA repos per stage, or name-embedded stage
|
|
|
|
|
+suffixes mirroring the `-tmp` package names); (ii) signing keys per repo
|
|
|
|
|
+with `keys/` 0600 and the public key distributed in the `.dcbar`; (iii)
|
|
|
|
|
+`.dcba` files are multi-hundred-MB for gcc — disk budgeting in the publish
|
|
|
|
|
+step (11 TB free here; a full set is ~15-25 GB xz).
|
|
|
|
|
+
|
|
|
|
|
+**Effort**: 2-4 days including the verify-by-fresh-root test (reuse the
|
|
|
|
|
+dcba-e2e.sh pattern).
|
|
|
|
|
+
|
|
|
|
|
+### 4.5 Item 5: build a chroot using USM from those repos
|
|
|
|
|
+
|
|
|
|
|
+**Mapping.** Stage 4 in the architecture below: enter the stage-0 container
|
|
|
|
|
+root (chroot on this sandbox; a container anywhere else), configure USM's
|
|
|
|
|
+`USM_CONFIGDIR`/state inside it, `usm add-repo` the file:// USM repo and
|
|
|
|
|
+`usm dcba add-repo` the DCBA repo(s), then run the orchestrator's stage
|
|
|
|
|
+installs — each `usm install <stage-packages>` resolves against the root's
|
|
|
|
|
+current contents, restores `.dcba` caches where available, builds where
|
|
|
|
|
+not, and stages outputs into the root. The result after the final stage is
|
|
|
|
|
+an LFS root *built by USM transactions* — the actual "USM distro" claim.
|
|
|
|
|
+
|
|
|
|
|
+**Constraints found.** (i) The chroot needs `/dev/null` (plain file is
|
|
|
|
|
+enough for most builds — proven) and ideally `/dev/zero`, `/dev/random`,
|
|
|
|
|
+`/dev/tty`; without `CAP_MKNOD` they stay plain files, so anything doing
|
|
|
|
|
+`ioctl` on them (systemd's build probes, some test suites) is at risk.
|
|
|
|
|
+(ii) `/proc` empty breaks packages that read `/proc/mounts` or
|
|
|
|
|
+`/proc/self/…` at build time; systemd again, plus `procps` tests. (iii)
|
|
|
|
|
+The ResourceFinder host-absolute locators (§3.2) are *correct inside* a
|
|
|
|
|
+chroot (destination `/`), so run `usm` inside the chroot for the build
|
|
|
|
|
+stages — which is also the honest end-to-end test of self-hosting. (iv)
|
|
|
|
|
+`USM never creates the state root itself` (dcba-e2e.sh header): the
|
|
|
|
|
+orchestrator pre-creates `<state>/{lists,packages,installed}` per root.
|
|
|
|
|
+
|
|
|
|
|
+**What's missing / must be built.** The stage orchestrator (~300-500
|
|
|
|
|
+lines): create root, populate /dev workaround files, run per-stage
|
|
|
|
|
+installs with the right env, checkpoint between stages (tar snapshot for
|
|
|
|
|
+fast iteration), and the sysvinit-vs-systemd switch (recommend building
|
|
|
|
|
+sysvinit revision first — no /proc dependency at build time — and
|
|
|
|
|
+systemd as a follow-up on a full host).
|
|
|
|
|
+
|
|
|
|
|
+**Effort**: 1-1.5 weeks including debugging the first full run.
|
|
|
|
|
+
|
|
|
|
|
+### 4.6 Item 6: a container based on that chroot with USM installed
|
|
|
|
|
+
|
|
|
|
|
+**Mapping.** The final root already *is* a USM-managed root (stages 4-5
|
|
|
|
|
+installed everything through USM transactions, and USM itself is installed
|
|
|
|
|
+by stage 5). Producing the container = the same OCI assembly as item 2
|
|
|
|
|
+(§4.2): tar the final root as one layer, write config (cmd `/bin/bash`,
|
|
|
|
|
+env `PATH=/usr/bin:/bin`, labels recording the repo commits + listing
|
|
|
|
|
+digests), manifest, `oci-layout`. On a podman host, `usm manifest deploy`
|
|
|
|
|
+of a meta-package (or just `podman import`) is the one-liner alternative.
|
|
|
|
|
+
|
|
|
|
|
+**What's missing / must be built.** Nothing structural. A "distro
|
|
|
|
|
+identity" decision (name, /etc/os-release — trivially a scaffold package),
|
|
|
|
|
+and the same assembler script with a different config blob. Verification:
|
|
|
|
|
+`podman run` on a capable host, or chroot-in-the-unpacked-image on this
|
|
|
|
|
+sandbox (bash + `usm --version` + `usm list`).
|
|
|
|
|
+
|
|
|
|
|
+**Effort**: 2-3 days, including the smoke-test matrix.
|
|
|
|
|
+
|
|
|
|
|
+## 5. The staged architecture
|
|
|
|
|
+
|
|
|
|
|
+One diagram, then per-stage detail. `H` is the build host (this sandbox or
|
|
|
|
|
+a full machine); every stage's output feeds the next.
|
|
|
|
|
+
|
|
|
|
|
+ Stage 0 Stage 1 Stage 2/3 Stage 4 Stage 5 Stage 6
|
|
|
|
|
+ bootstrap image ← scaffolds → USM repo + DCBA repo → final LFS root → +beyond-LFS → OCI image
|
|
|
|
|
+ (dnf + ch2 tool- (generated (built INSIDE a stage-0 (built by staged + USM itself (assembled by
|
|
|
|
|
+ chain + usm) from book) chroot, staged installs) usm transactions) (self-host) script)
|
|
|
|
|
+
|
|
|
|
|
+### Stage 0 — bootstrap container/image
|
|
|
|
|
+
|
|
|
|
|
+Purpose: a hermetic chapter-2-compliant build host. Recipe (dnf-based):
|
|
|
|
|
+
|
|
|
|
|
+1. `dnf --installroot=$ROOT --releasever=44 install @build-tools` or an
|
|
|
|
|
+ explicit list from the book's ch2 table, **plus `gcc-c++` and `patch`**
|
|
|
|
|
+ (the two gaps found on this sandbox — see §2.3), plus usm's runtime
|
|
|
|
|
+ closure: `curl git libsodium glib2 json-glib` (Fedora names).
|
|
|
|
|
+2. Install `usm` itself (any build that runs on the host glibc — the
|
|
|
|
|
+ Fedora dnf set is fine; this stage-0 copy never needs to be the
|
|
|
|
|
+ self-hosted one).
|
|
|
|
|
+3. Fetch/verify the ~94 LFS sources (URLs+md5 from `packages.ent`), or
|
|
|
|
|
+ vendor them into the image; pre-seed the file:// repo layout.
|
|
|
|
|
+4. Assemble the OCI image (script) — or just keep it as a directory tree
|
|
|
|
|
+ for chroot use on this sandbox.
|
|
|
|
|
+
|
|
|
|
|
+Outputs: `$ROOT` (chroot-able), `stage0.img` (OCI layout).
|
|
|
|
|
+
|
|
|
|
|
+### Stage 1 — the scaffold repository
|
|
|
|
|
+
|
|
|
|
|
+The generator (§4.1) reads `packages.ent` + chapter XML from the LFS
|
|
|
|
|
+checkout and emits one scaffold dir per package-pass:
|
|
|
|
|
+
|
|
|
|
|
+- `MANIFEST.usm` — name (with `-tmp` suffixes for ch5/6 passes),
|
|
|
|
|
+ version, licence (from the book's licence notes), `depends`
|
|
|
|
|
+ (manage: `bin:bash`, `bin:coreutils`…; build: the tools the book's
|
|
|
|
|
+ commands call; runtime: the libs they install), `provides`
|
|
|
|
|
+ (`bin:gcc`, `lib:libc.so.6`, …), `execs` (acquire/build/install/test),
|
|
|
|
|
+ extra-property `stage` ∈ {tmp0, tmp1, chroot, final}.
|
|
|
|
|
+- `usm-scripts/build.sh` — the book's `configure`/`make` commands with
|
|
|
|
|
+ `PATH`/`LFS_TGT` env set by the stage driver, `make -j$(nproc)`.
|
|
|
|
|
+- `usm-scripts/install.sh` — the book's install commands; DESTDIR-style
|
|
|
|
|
+ staging is NOT what LFS does (it installs into `/`/`/tools` inside the
|
|
|
|
|
+ chroot), so install scripts write directly and USM's per-package
|
|
|
|
|
+ install staging is bypassed via `install_path == destination` (or the
|
|
|
|
|
+ scripts `cp` from a destdir; per-scaffold choice, recorded in the
|
|
|
|
|
+ generator's overrides).
|
|
|
|
|
+- vendored patches from the book.
|
|
|
|
|
+
|
|
|
|
|
+Output: the scaffold git repo (reviewed, committed) → `*.usmc` via
|
|
|
|
|
+`usm manifest package` → the USM repo (§4.3).
|
|
|
|
|
+
|
|
|
|
|
+### Stage 2/3 — build the USM + DCBA repositories inside a stage-0 chroot
|
|
|
|
|
+
|
|
|
|
|
+Why inside: the packages must be built by the stage-0 toolchain in the
|
|
|
|
|
+stage-0 environment, and USM must be able to see its own staged outputs
|
|
|
|
|
+(local presence) — which is exactly "run usm inside the chroot". Driver:
|
|
|
|
|
+
|
|
|
|
|
+1. chroot into stage-0's `$ROOT` (plain-file /dev/null workaround; empty
|
|
|
|
|
+ /proc acknowledged).
|
|
|
|
|
+2. `usm add-repo` the stage-1 USM repo; for each stage s ∈
|
|
|
|
|
+ {tmp0, tmp1, chroot, final}: `usm install <stage-s packages>` — each
|
|
|
|
|
+ install resolves against the root as-left-by-stage-(s-1).
|
|
|
|
|
+3. After each stage: publish `.dcba` artifacts (copy `build.tar.xz` →
|
|
|
|
|
+ `<name>-<version>.dcba`, `usm dcba repository add`) into per-stage DCBA
|
|
|
|
|
+ repos.
|
|
|
|
|
+
|
|
|
|
|
+Outputs: `PACKAGES.usml`-signed USM repo; per-stage `ARTIFACTS.usml`-signed
|
|
|
|
|
+DCBA repos.
|
|
|
|
|
+
|
|
|
|
|
+### Stage 4 — USM-driven chroot build of the final root
|
|
|
|
|
+
|
|
|
|
|
+Fresh root `$LFS`; populate `/dev` workaround files + empty `/proc`;
|
|
|
|
|
+chroot; configure a *new* USM state inside it; add the stage-1 USM repo +
|
|
|
|
|
+the per-stage DCBA repos; run the same stage sequence. Because the DCBA
|
|
|
|
|
+caches restore, this stage is CPU-cheap (cache restores + final-system
|
|
|
|
|
+gcc/glibc builds are the only heavy items). End state: a complete LFS
|
|
|
|
|
+system whose every file was placed by a USM transaction.
|
|
|
|
|
+
|
|
|
|
|
+### Stage 5 — beyond-LFS self-hosting set + USM itself
|
|
|
|
|
+
|
|
|
|
|
+The book does not contain USM's build deps. The gap set (from the plan's
|
|
|
|
|
+evidence, re-confirmed against the book's package list): the book HAS
|
|
|
|
|
+pcre2 (chapter08/pcre.xml), libffi, python, meson, ninja, pkgconf,
|
|
|
|
|
+elfutils/libelf — so the beyond-LFS set is small: **glib, libgee, valac,
|
|
|
|
|
+json-glib, curl** (+ optional git). Order matters: glib → libgee →
|
|
|
|
|
+json-glib → valac → USM. Then `usm install usm` (from a `.usmc` of USM's
|
|
|
|
|
+own repo — USM's MANIFEST.usm exists at the repo root) is the
|
|
|
|
|
+self-hosting milestone: the distro's package manager installs itself.
|
|
|
|
|
+
|
|
|
|
|
+### Stage 6 — OCI image assembly
|
|
|
|
|
+
|
|
|
|
|
+Scripted (no runtime here): tar `$LFS` (+ stage-5 additions) as one
|
|
|
|
|
+gzip/xz layer, emit `oci-layout`, `index.json`, config JSON with
|
|
|
|
|
+`cmd ["/bin/bash"]`, env, and labels carrying the repo commit + listing
|
|
|
|
|
+digests. Optionally split `/usr` vs `/etc/var` into two layers for
|
|
|
|
|
+cache-friendlier future updates. On podman hosts: `podman load`/`run` for
|
|
|
|
|
+verification, or skip assembly entirely and use `usm manifest deploy`.
|
|
|
|
|
+
|
|
|
|
|
+## 6. Why staged (the resolver argument, precisely)
|
|
|
|
|
+
|
|
|
|
|
+`Resolver.resolve` (src/lib/Resolver.vala:232) computes ONE resolved set
|
|
|
|
|
+with ONE topological order over `manage`+`build`+`runtime` refs
|
|
|
|
|
+(src/lib/Resolver.vala:422-428). LFS needs three orders interleaved:
|
|
|
|
|
+binutils-p1 → gcc-p1 → (ch5 libs) → gcc-p2 must complete *before* ch7's
|
|
|
|
|
+chroot tools build against them, and ch8's final glibc/gcc must replace
|
|
|
|
|
+the temporary ones — i.e. the same *resource* (`lib:libc.so.6`,
|
|
|
|
|
+`bin:gcc`) is provided by different packages at different times, with
|
|
|
|
|
+deliberate replacement semantics. USM's resolver treats one name-version
|
|
|
|
|
+per package and picks providers once; replacement-across-stages is a
|
|
|
|
|
+different problem (it has `usm downgrade`/transaction machinery, but not
|
|
|
|
|
+"same provides, different package, by stage"). Hence: stage boundaries are
|
|
|
|
|
+orchestrator-level, stage interiors are USM resolutions. This is not a
|
|
|
|
|
+USM defect — it is the honest boundary between a package manager and a
|
|
|
|
|
+distro builder, and it keeps every stage debuggable with stock `usm`
|
|
|
|
|
+commands and stock resolver error reports.
|
|
|
|
|
+
|
|
|
|
|
+## 7. Effort estimates
|
|
|
|
|
+
|
|
|
|
|
+### 7.1 Wall-clock (build time)
|
|
|
|
|
+
|
|
|
|
|
+SBU data is machine-readable in `packages.ent` (entities
|
|
|
|
|
+`<pkg>-<tmp|fin>-sbu`). Summed this session (test-suite entities excluded,
|
|
|
|
|
+"less than X" counted as X/2):
|
|
|
|
|
+
|
|
|
|
|
+- ch5/6 temporary tools (`-tmp-sbu`, 33 entries): **~21 SBU**
|
|
|
|
|
+ (gcc-p2 5.2, gcc-p1 4.3, tcl 2.9, gettext 1.5, glibc-tmp 1.3, …).
|
|
|
|
|
+- ch8 final system (`-fin-sbu`, 82 entries): **~100 SBU**
|
|
|
|
|
+ (gcc 53, glibc 11, vim 3.3, python 2.7, e2fsprogs 2.4, …).
|
|
|
|
|
+- Linux kernel (~32 SBU) is *not* part of the bootstrap (no boot
|
|
|
|
|
+ requirement for a container/chroot distro); building it doubles the
|
|
|
|
|
+ gcc-dominated budget if wanted.
|
|
|
|
|
+
|
|
|
|
|
+One SBU = one core's binutils-p1 time, typically **~1-2 min** on modern
|
|
|
|
|
+hardware (the book: gcc ~5 min "on the fastest systems"). With ~120 SBU
|
|
|
|
|
+total and `make -j28`, the parallelisable `make` phases compress roughly
|
|
|
|
|
+10-20x, but configure phases, install phases and single-threaded steps
|
|
|
|
|
+(glibc's `make install`, gcc's early stages) do not. Honest estimate for
|
|
|
|
|
+one full stage-0→4 pass **on this 28-core sandbox**:
|
|
|
|
|
+
|
|
|
|
|
+- without test suites: **~3-5 hours** (gcc dominates: its 53 SBU ≈
|
|
|
|
|
+ 1.5-2 h alone at j28 with serial configure/install).
|
|
|
|
|
+- with the book's recommended test suites: **~8-14 hours** (tests are
|
|
|
|
|
+ mostly serial and several are multi-SBU).
|
|
|
|
|
+- Stage 5 (beyond-LFS + USM): ~30-45 min (glib ~0.5 SBU, valac ~2 SBU,
|
|
|
|
|
+ USM itself < 1 SBU at j28).
|
|
|
|
|
+- A DCBA-cached re-run of stage 4 (iteration loop): **~20-40 min** — this
|
|
|
|
|
+ is the number that matters for daily development.
|
|
|
|
|
+
|
|
|
|
|
+Disk per full run: sources ~1 GB, build trees ~40-80 GB, artifacts
|
|
|
|
|
+(build.tar.xz set) ~15-25 GB, final root ~4-8 GB. Budget ~200 GB per
|
|
|
|
|
+working copy; 11 TB free comfortably supports several parallel runs, and
|
|
|
|
|
+zero runs if any of it lands on `/` (17 GB — see §2.2).
|
|
|
|
|
+
|
|
|
|
|
+### 7.2 Engineering time
|
|
|
|
|
+
|
|
|
|
|
+Per-item figures from §4; consolidated (one engineer, familiar with both
|
|
|
|
|
+USM and LFS):
|
|
|
|
|
+
|
|
|
|
|
+- Stage 0 recipe + OCI assembler + verify harness: **3-5 days**
|
|
|
|
|
+- Generator (book → scaffolds, with overrides table): **2-3 weeks**
|
|
|
|
|
+- USM repo modelling (stages, provides vocab, resolution validation): **1 week**
|
|
|
|
|
+- DCBA publish driver + per-stage repos: **2-4 days**
|
|
|
|
|
+- Stage orchestrator (chroot mgmt, /dev workarounds, checkpoints):
|
|
|
|
|
+ **1-1.5 weeks**
|
|
|
|
|
+- Beyond-LFS set (6 packages, valac ordering) + USM self-host: **3-5 days**
|
|
|
|
|
+- OCI final image + smoke tests + documentation: **2-3 days**
|
|
|
|
|
+- Integration/debug of the first full run + book-drift tooling
|
|
|
|
|
+ (regeneration diff): **1 week**
|
|
|
|
|
+
|
|
|
|
|
+**Total: ~6-9 engineer-weeks** (the spread is almost entirely the
|
|
|
|
|
+generator: 94 packages × curated overrides vs mostly-mechanical).
|
|
|
|
|
+Uncertainty: a first full run will surface 5-15 per-package script bugs
|
|
|
|
|
+(extraction artefacts, env differences) — that's inside the integration
|
|
|
|
|
+line above; a systemd-revision build adds ~3-5 days if pursued (and needs
|
|
|
|
|
+a full host for /proc reasons, §8).
|
|
|
|
|
+
|
|
|
|
|
+## 8. Risk register
|
|
|
|
|
+
|
|
|
|
|
+- **Device nodes in /dev** (no CAP_MKNOD). Plain-file `/dev/null` works
|
|
|
|
|
+ for most builds (proven with bison); `/dev/zero`, `/dev/random`,
|
|
|
|
|
+ `/dev/tty` as plain files break anything doing ioctls. Mitigations:
|
|
|
|
|
+ stage-0 image carries *pre-created device nodes* (built where mknod is
|
|
|
|
|
+ allowed, e.g. the human's machine or a rootful builder — the image is
|
|
|
|
|
+ just a tarball, so nodes survive into it; only *creating* them here is
|
|
|
|
|
+ impossible); chroots unpacking that image inherit working /dev.
|
|
|
|
|
+- **Empty /proc and /sys** (no CAP_SYS_ADMIN to mount). Most packages
|
|
|
|
|
+ fine; systemd's build/configure and readers of `/proc/mounts` at risk.
|
|
|
|
|
+ Mitigations: sysvinit revision first (no /proc at build time);
|
|
|
|
|
+ systemd build on a full host; or bind-mount-free stub files where the
|
|
|
|
|
+ book's commands tolerate them.
|
|
|
|
|
+- **systemd vs sysvinit.** Both revisions are in the book (261.3 vs
|
|
|
|
|
+ 3.14). Recommend shipping sysvinit first, systemd as a follow-up goal.
|
|
|
|
|
+- **ResourceFinder host-absolute locators** (`ldconfig -p`, `/usr/include`,
|
|
|
|
|
+ `/usr/libexec`, gir/man/info/locale — §3.2). Outside a chroot, staged
|
|
|
|
|
+ roots can be "satisfied" by host files. Mitigation: run build-stage usm
|
|
|
|
|
+ inside the chroot (destination `/`, so absolute == correct); for
|
|
|
|
|
+ outside-chroot tooling, audit or later patch ResourceFinder with a
|
|
|
|
|
+ strict-destination mode (candidate small USM improvement, out of scope
|
|
|
|
|
+ here).
|
|
|
|
|
+- **Acquire phase needs curl/git/wget** — but the bootstrap's acquire
|
|
|
|
|
+ scripts can instead use *vendored* sources already inside the .usmc or
|
|
|
|
|
+ the stage-0 image, removing the network dependency from stages 2-5
|
|
|
|
|
+ entirely (recommended; the book's md5s gate integrity).
|
|
|
|
|
+- **Book drift.** LFS 13.1 → 13.2+ bumps package versions and commands.
|
|
|
|
|
+ Mitigation: the generator is the only thing that reads the book;
|
|
|
|
|
+ regeneration diff (git diff of scaffolds) makes drift reviewable.
|
|
|
|
|
+- **valac bootstrap ordering.** valac needs glib+libgee; USM needs valac
|
|
|
|
|
+ + glib + json-glib. Fixed linear order in stage 5 (glib → libgee →
|
|
|
|
|
+ json-glib → valac → USM); no cycle exists because valac's own
|
|
|
|
|
+ compilation can use a stage-0-hosted valac (dnf-installed) — the dnf
|
|
|
|
|
+ set already includes one, so USM's first self-host can even avoid
|
|
|
|
|
+ valac-from-source if desired.
|
|
|
|
|
+- **ed25519 key management.** `keys/` 0600, private keys never leave the
|
|
|
|
|
+ producer; `.dcbar`/`.usmr` carry the public key. Risk: losing the key
|
|
|
|
|
+ orphanes a repo — keep one offline master per distro release line, and
|
|
|
|
|
+ document rotation.
|
|
|
|
|
+- **Disk.** `/` here is 17 GB — *nothing* large may touch /tmp; all
|
|
|
|
|
+ scratch under /workspace (11 TB). Checkpoint tars double usage
|
|
|
|
|
+ transiently; clean between stages.
|
|
|
|
|
+- **gcc/binutils newer than the book** are explicitly unsupported by LFS
|
|
|
|
|
+ (ch2 warning). This host's gcc 16.2.1 == book's 16.2.0 line — fine;
|
|
|
|
|
+ any host bump past the book needs a book bump first. Relatedly, the
|
|
|
|
|
+ sandbox host itself lacks `g++`/`patch` (ch2 requirements) — with root
|
|
|
|
|
+ and dnf5 this is a non-event: stage 0 installs them, and a ch2-check
|
|
|
|
|
+ script fails fast on any host still missing items.
|
|
|
|
|
+
|
|
|
|
|
+## 9. Recommended task breakdown (for the implementation goal)
|
|
|
|
|
+
|
|
|
|
|
+Ordered; each is independently verifiable and lands value on its own:
|
|
|
|
|
+
|
|
|
|
|
+1. **stage0 recipe + OCI assembler + ch2 check** (3-5 d) — produce a
|
|
|
|
|
+ bootstrap image from dnf; verify by chroot + book ch2 script.
|
|
|
|
|
+2. **Generator v1: packages.ent → scaffold skeletons** (3-5 d) — manifest
|
|
|
|
|
+ fields, URLs/md5s, acquire scripts; build.sh placeholders; no commands
|
|
|
|
|
+ yet. Deliverable: 94 scaffolds packaging cleanly (`usm manifest
|
|
|
|
|
+ package` all green).
|
|
|
|
|
+3. **Generator v2: chapter XML → build/install scripts + overrides
|
|
|
|
|
+ table** (1.5-2 wk) — ch5/6 (tmp passes, /tools env), ch7, ch8;
|
|
|
|
|
+ curated handling for gcc/binutils/glibc/systemd|sysvinit/perl/python.
|
|
|
|
|
+4. **USM repo modelling + stage tags + resolution dry-runs** (1 wk) —
|
|
|
|
|
+ resolver-validated dependency graph per stage, zero UNSATISFIABLE.
|
|
|
|
|
+5. **Stage orchestrator + stage-2/3 build of tmp/chroot stages** (1 wk) —
|
|
|
|
|
+ first real chroot builds inside stage-0; publish tmp/chroot `.dcba`
|
|
|
|
|
+ repos.
|
|
|
|
|
+6. **Stage 4: final-system build via USM** (3-5 d) — full ch8 through usm
|
|
|
|
|
+ transactions with DCBA caches.
|
|
|
|
|
+7. **Stage 5: beyond-LFS set + USM self-host** (3-5 d) — the milestone:
|
|
|
|
|
+ `usm install usm` inside the final root.
|
|
|
|
|
+8. **Stage 6: final OCI image + smoke tests** (2-3 d) — assembled image,
|
|
|
|
|
+ chroot/podman verification matrix, docs.
|
|
|
|
|
+9. **(Follow-up, separate goal) systemd revision, kernel build, https
|
|
|
|
|
+ repo hosting, ResourceFinder strict-destination mode.**
|
|
|
|
|
+
|
|
|
|
|
+## 10. Conclusion
|
|
|
|
|
+
|
|
|
|
|
+The goal is feasible with current USM master: the staged architecture
|
|
|
|
|
+uses only documented, e2e-tested mechanics (staged installs, file://
|
|
|
|
|
+repos, DCBA artifact distribution), the LFS book is machine-minable
|
|
|
|
|
+enough for a generator with a modest overrides table, and the two hard
|
|
|
|
|
+environmental constraints of this sandbox (no device-node creation, no
|
|
|
|
|
+mounts) are worked around for every stage except a systemd build. The
|
|
|
|
|
+recommended first implementation milestone is tasks 1+2: a verified
|
|
|
|
|
+bootstrap image and 94 packaging-clean scaffolds — roughly a week of
|
|
|
|
|
+work that de-risks everything downstream.
|