|
@@ -16,7 +16,8 @@ be one USM resolution. It must be staged: a handful of driver invocations,
|
|
|
each a valid single-pass resolution, chained by an orchestrator. The dominant
|
|
each a valid single-pass resolution, chained by an orchestrator. The dominant
|
|
|
risks are environmental, not architectural: device nodes and `/proc` inside a
|
|
risks are environmental, not architectural: device nodes and `/proc` inside a
|
|
|
capability-restricted chroot, systemd's build requirements, and self-hosting
|
|
capability-restricted chroot, systemd's build requirements, and self-hosting
|
|
|
-USM itself (valac + glib are not in the LFS book).
|
|
|
|
|
|
|
+USM itself (valac + glib and several friends are not in the LFS
|
|
|
|
|
+book — see Stage 5).
|
|
|
|
|
|
|
|
Everything below was verified against the USM checkout at commit `b046afe`
|
|
Everything below was verified against the USM checkout at commit `b046afe`
|
|
|
("Merge branch 'goal/dcba' of Tilo15/usm into master", branch `master`) and
|
|
("Merge branch 'goal/dcba' of Tilo15/usm into master", branch `master`) and
|
|
@@ -34,7 +35,7 @@ checkouts.
|
|
|
USM scaffolds, stage drivers, and OCI assembly scripts) plus a small
|
|
USM scaffolds, stage drivers, and OCI assembly scripts) plus a small
|
|
|
beyond-LFS package set so USM can build itself.
|
|
beyond-LFS package set so USM can build itself.
|
|
|
- **The architecture must be staged.** `Resolver.resolve`
|
|
- **The architecture must be staged.** `Resolver.resolve`
|
|
|
- (src/lib/Resolver.vala:232) processes `manage`+`build`+`runtime` phases in
|
|
|
|
|
|
|
+ (src/lib/Resolver.vala:233) processes `manage`+`build`+`runtime` phases in
|
|
|
one pass and produces one topological order
|
|
one pass and produces one topological order
|
|
|
(src/lib/Resolver.vala:422-428) — a single resolution cannot express LFS's
|
|
(src/lib/Resolver.vala:422-428) — a single resolution cannot express LFS's
|
|
|
ch5/6 temporary tools → ch7 chroot tools → ch8 final system sequence, where
|
|
ch5/6 temporary tools → ch7 chroot tools → ch8 final system sequence, where
|
|
@@ -52,17 +53,18 @@ checkouts.
|
|
|
the pipeline itself.
|
|
the pipeline itself.
|
|
|
- **Disk layout matters**: `/` (and `/tmp`) is a 17 GB overlay while
|
|
- **Disk layout matters**: `/` (and `/tmp`) is a 17 GB overlay while
|
|
|
`/workspace` has ~11 TB free — all LFS scratch trees must live under
|
|
`/workspace` has ~11 TB free — all LFS scratch trees must live under
|
|
|
- `/workspace`. (The host also lacks `g++`/`patch` — two chapter-2
|
|
|
|
|
- requirements — but the sandbox has root and dnf5, so stage 0's image
|
|
|
|
|
|
|
+ `/workspace`. (The host also lacks `g++`, `patch` and `makeinfo` —
|
|
|
|
|
+ three chapter-2 requirements — but the sandbox has root and dnf5, so
|
|
|
|
|
+ stage 0's image
|
|
|
recipe closes that with one `dnf install` line; see §2.3.)
|
|
recipe closes that with one `dnf install` line; see §2.3.)
|
|
|
-- **Effort.** Roughly **6–9 engineer-weeks** for the pipeline (generator +
|
|
|
|
|
|
|
+- **Effort.** Roughly **7–10 engineer-weeks** for the pipeline (generator +
|
|
|
stage drivers + OCI scripts + key management + CI), of which the single
|
|
stage drivers + OCI scripts + key management + CI), of which the single
|
|
|
- largest item is the book-to-scaffold generator (94 packages, ~640 command
|
|
|
|
|
|
|
+ largest item is the book-to-scaffold generator (~90 packages, ~640 command
|
|
|
blocks, needing curated per-package handling). Wall-clock for one full
|
|
blocks, needing curated per-package handling). Wall-clock for one full
|
|
|
bootstrap run on this 28-core sandbox: **~3–5 hours without test suites,
|
|
bootstrap run on this 28-core sandbox: **~3–5 hours without test suites,
|
|
|
~8–14 hours with** (SBU-derived; see §7). Artifacts published per run:
|
|
~8–14 hours with** (SBU-derived; see §7). Artifacts published per run:
|
|
|
- ~94 `.usmc` source packages, ~94 `.dcba` build artifacts, two signed
|
|
|
|
|
- listings.
|
|
|
|
|
|
|
+ ~90 `.usmc` source packages (one per package-pass) and as many `.dcba`
|
|
|
|
|
+ build artifacts, plus the signed listings.
|
|
|
|
|
|
|
|
## 2. Verified platform facts
|
|
## 2. Verified platform facts
|
|
|
|
|
|
|
@@ -116,14 +118,22 @@ one-liner.
|
|
|
(`chapter02/hostreqs.xml`) that *newer* gcc/binutils than the book's are
|
|
(`chapter02/hostreqs.xml`) that *newer* gcc/binutils than the book's are
|
|
|
untested and likely to break the build; equal is the intended case;
|
|
untested and likely to break the build; equal is the intended case;
|
|
|
- kernel 7.0 > minimum → glibc headers configure cleanly.
|
|
- kernel 7.0 > minimum → glibc headers configure cleanly.
|
|
|
-- **Host tool gaps (new this session):** `/usr/bin/g++` and `cc1plus` are
|
|
|
|
|
- **absent** (`gcc -x c++` fails: "cannot execute 'cc1plus'"), and `patch`
|
|
|
|
|
- is absent. `find`/`makeinfo` exist where needed. The book's ch2 software
|
|
|
|
|
- list requires both a C++ compiler and patch. `dnf5 5.4.3.0` is present and
|
|
|
|
|
- both `gcc-c++.x86_64 16.2.1-2.fc44` (updates) and `patch.x86_64
|
|
|
|
|
- 2.8-4.fc44` (fedora) are available → the Stage-0 image build must
|
|
|
|
|
- `dnf install gcc-c++ patch` (plus any other missing ch2 items re-checked
|
|
|
|
|
- at image-build time).
|
|
|
|
|
|
|
+- **Host tool gaps (audited this session against the book's ch2 list):**
|
|
|
|
|
+ `g++`/`cc1plus` **absent** (`gcc -x c++` fails: "cannot execute
|
|
|
|
|
+ 'cc1plus'"), `patch` **absent**, and `makeinfo`/`texi2any` (Texinfo)
|
|
|
|
|
+ **absent** — all three are ch2 requirements (Texinfo-5.0+ is on the
|
|
|
|
|
+ list; the glibc/gcc/perl builds invoke `makeinfo` if present and skip
|
|
|
|
|
+ documentation otherwise, but the book expects a compliant host).
|
|
|
|
|
+ `/usr/bin/yacc` is also absent (the book wants yacc as a bison link —
|
|
|
|
|
+ bison 3.8.2 itself is present, and the ch5 bison rebuild provides
|
|
|
|
|
+ `/usr/bin/yacc` inside the toolchain, so this only matters for the
|
|
|
|
|
+ stage-0 host surface). Everything else on the ch2 list is present:
|
|
|
|
|
+ bash 5.3.9, as/ld (binutils 2.46.1), bison, coreutils (incl. `nproc`),
|
|
|
|
|
+ diffutils (`diff`/`cmp`), findutils, gawk, gcc, grep, gzip, m4, make,
|
|
|
|
|
+ perl, python3, sed, tar, xz. `dnf5 5.4.3.0` is present and the missing
|
|
|
|
|
+ items are plain dnf installs (`gcc-c++`, `patch`, `texinfo`) → the
|
|
|
|
|
+ Stage-0 image build must `dnf install gcc-c++ patch texinfo` (plus a
|
|
|
|
|
+ ch2 re-check script at image-build time).
|
|
|
|
|
|
|
|
### 2.4 Container runtimes
|
|
### 2.4 Container runtimes
|
|
|
|
|
|
|
@@ -152,7 +162,7 @@ in this session.
|
|
|
pre-created, `usm.config` with `is_managed=true` +
|
|
pre-created, `usm.config` with `is_managed=true` +
|
|
|
`managed.state_path`).
|
|
`managed.state_path`).
|
|
|
2. **ResourceFinder local presence.** `ResourceFinder.locate_resource`
|
|
2. **ResourceFinder local presence.** `ResourceFinder.locate_resource`
|
|
|
- (src/lib/ResourceFinder.vala:13-58) resolves refs mostly **relative to
|
|
|
|
|
|
|
+ (src/lib/ResourceFinder.vala:13-64) resolves refs mostly **relative to
|
|
|
`paths.destination`** (`locate_path`, `locate_res`, `locate_cfg`,
|
|
`paths.destination`** (`locate_path`, `locate_res`, `locate_cfg`,
|
|
|
`locate_bin` walks `$PATH` under destination, `locate_libres`,
|
|
`locate_bin` walks `$PATH` under destination, `locate_libres`,
|
|
|
`locate_pc` including `PKG_CONFIG_PATH` entries, `locate_vapi`,
|
|
`locate_pc` including `PKG_CONFIG_PATH` entries, `locate_vapi`,
|
|
@@ -160,14 +170,14 @@ in this session.
|
|
|
sees only the staged root. **Caveat (a real constraint, not a blocker):
|
|
sees only the staged root. **Caveat (a real constraint, not a blocker):
|
|
|
several locators are host-absolute** — `locate_lib` consults
|
|
several locators are host-absolute** — `locate_lib` consults
|
|
|
`ldconfig -p` and falls back to absolute `/lib /usr/lib /lib64
|
|
`ldconfig -p` and falls back to absolute `/lib /usr/lib /lib64
|
|
|
- /usr/lib64` (src/lib/ResourceFinder.vala:88-126), and `locate_inc`,
|
|
|
|
|
|
|
+ /usr/lib64` (src/lib/ResourceFinder.vala:75-104), and `locate_inc`,
|
|
|
`locate_libexec`, `locate_gir`, `locate_info/man/locale`,
|
|
`locate_libexec`, `locate_gir`, `locate_info/man/locale`,
|
|
|
`locate_app` use absolute `/usr/...` paths. Inside a chroot that is
|
|
`locate_app` use absolute `/usr/...` paths. Inside a chroot that is
|
|
|
correct (destination is `/`); *outside* a chroot with USM_DESTDIR set,
|
|
correct (destination is `/`); *outside* a chroot with USM_DESTDIR set,
|
|
|
`lib:`/`include:` refs can be satisfied by the host — the stage drivers
|
|
`lib:`/`include:` refs can be satisfied by the host — the stage drivers
|
|
|
must run usm inside the target chroot, or audited for it.
|
|
must run usm inside the target chroot, or audited for it.
|
|
|
3. **Resolver is single-pass.** `Resolver.resolve`
|
|
3. **Resolver is single-pass.** `Resolver.resolve`
|
|
|
- (src/lib/Resolver.vala:232-287) picks roots, does ONE batched SPM query,
|
|
|
|
|
|
|
+ (src/lib/Resolver.vala:233-287) picks roots, does ONE batched SPM query,
|
|
|
processes all packages, and `resolution_phases` folds
|
|
processes all packages, and `resolution_phases` folds
|
|
|
`manage`+`build`+`runtime` (src/lib/Resolver.vala:422-428) into one
|
|
`manage`+`build`+`runtime` (src/lib/Resolver.vala:422-428) into one
|
|
|
dependency graph with one topological install order. Missing resources
|
|
dependency graph with one topological install order. Missing resources
|
|
@@ -230,20 +240,28 @@ XML; the machine-readable part is excellent (`packages.ent`: 718 entities —
|
|
|
names, versions, URLs, md5s, SBU/disk estimates as text), but the ~640
|
|
names, versions, URLs, md5s, SBU/disk estimates as text), but the ~640
|
|
|
build steps are prose. Counts from this checkout:
|
|
build steps are prose. Counts from this checkout:
|
|
|
|
|
|
|
|
-- `userinput remap="…"` command blocks across ch5–8: **558** — 105
|
|
|
|
|
- `configure`, 118 `make`, 180 `install`, 76 `pre`, 76 `test`, 3 `adjust`;
|
|
|
|
|
- plus **85** un-remapped `<userinput>` (chapter setup commands).
|
|
|
|
|
-- Package-ish pages: ch5 5, ch6 17, ch7 8, ch8 91 files (85 with build
|
|
|
|
|
- commands) → **~94 source packages** distinct (some pages are
|
|
|
|
|
- multi-pass: gcc appears in ch5 pass1/pass2 and ch8; binutils pass1/ch8).
|
|
|
|
|
|
|
+- `userinput remap="…"` command blocks across ch5–8: **563** — 105
|
|
|
|
|
+ `configure`, 120 `make`, 180 `install`, 76 `pre`, 76 `test`, 3 `adjust`,
|
|
|
|
|
+ plus 3 one-offs (`clean`, `locale-test`, `locale-full`); plus **83**
|
|
|
|
|
+ un-remapped `<userinput>` (chapter setup commands) → 646 `<userinput>`
|
|
|
|
|
+ elements in total across ch5–8.
|
|
|
|
|
+- Package pages (procedural pages like `kernfs/chroot/createfiles`
|
|
|
|
|
+ excluded): ch5 5, ch6 17, ch7 8, ch8 85 → 115 pages, 91 distinct
|
|
|
|
|
+ packages (some are multi-pass: gcc appears as ch5 pass1/pass2 plus
|
|
|
|
|
+ `libstdc++` and ch8; binutils as ch5/ch6 pass1/pass2 and ch8).
|
|
|
|
|
+ Chapter 3 lists **93 downloadable URLs** (~85 of them source tarballs
|
|
|
|
|
+ with md5s, the rest docs/bootscripts/udev-lfs/kernel); the ~90
|
|
|
|
|
+ package-pass scaffolds in the executive summary exclude docs/extras
|
|
|
|
|
+ and count multi-pass packages once.
|
|
|
|
|
|
|
|
A naive `<userinput>`-to-shell-literal extraction produces *almost* right
|
|
A naive `<userinput>`-to-shell-literal extraction produces *almost* right
|
|
|
scripts — but not right: entities need expansion, `make -j1` /
|
|
scripts — but not right: entities need expansion, `make -j1` /
|
|
|
`make -k check` conventions need policy, ch5/6 scripts must hard-code
|
|
`make -k check` conventions need policy, ch5/6 scripts must hard-code
|
|
|
`/tools` prefixes and env (`LFS`, `LFS_TGT`, `PATH=/tools/bin:...`), ch8
|
|
`/tools` prefixes and env (`LFS`, `LFS_TGT`, `PATH=/tools/bin:...`), ch8
|
|
|
scripts assume the chroot, and a few packages need hand-massaging (gcc's
|
|
scripts assume the chroot, and a few packages need hand-massaging (gcc's
|
|
|
-three passes and its `limitl`, glibc's `libc.picture`-era configure
|
|
|
|
|
-options, systemd's `meson -D*` policy, perl's `groff`-era hints). So: a
|
|
|
|
|
|
|
+three passes plus its `ulimit -s -H unlimited` test preamble, glibc's
|
|
|
|
|
+long multi-line configure block, systemd's `meson -D*` policy, perl's
|
|
|
|
|
+and python's non-`./configure` build commands). So: a
|
|
|
generator (Python, ~1.5-2k lines) with a **curated overrides table** for
|
|
generator (Python, ~1.5-2k lines) with a **curated overrides table** for
|
|
|
the ~15 hairy packages, emitting both the scaffolds and a
|
|
the ~15 hairy packages, emitting both the scaffolds and a
|
|
|
`stage-manifest.json` the orchestrator consumes. Generated scaffolds are
|
|
`stage-manifest.json` the orchestrator consumes. Generated scaffolds are
|
|
@@ -256,14 +274,16 @@ the ch5/6/ch7/ch8 script envs), scaffold review pass 2-4 days.
|
|
|
|
|
|
|
|
**Mapping.** The bootstrap container is the *build host* for stages 1-4: a
|
|
**Mapping.** The bootstrap container is the *build host* for stages 1-4: a
|
|
|
root filesystem with a chapter-2-compliant toolchain, the `dnf`-installed
|
|
root filesystem with a chapter-2-compliant toolchain, the `dnf`-installed
|
|
|
-gaps closed, `usm` + its runtime deps (curl, git, libsodium, glib, json-glib
|
|
|
|
|
-— via dnf on the Fedora base) preinstalled, and the LFS sources either
|
|
|
|
|
-pre-fetched or fetchable. Two ways to produce it:
|
|
|
|
|
|
|
+gaps closed, `usm` + its runtime/build closure (curl, git, libsodium, glib,
|
|
|
|
|
+libgee, json-glib, vala, gobject-introspection, and the two invercargill
|
|
|
|
|
+libraries — all via dnf on the Fedora base; see §5 Stage 5) preinstalled,
|
|
|
|
|
+and the LFS sources either pre-fetched or fetchable. Two ways to produce it:
|
|
|
|
|
|
|
|
- **(a) dnf-based (recommended for stage 0).** Start from a Fedora
|
|
- **(a) dnf-based (recommended for stage 0).** Start from a Fedora
|
|
|
container base (or `dnf --installroot` into a directory on this host —
|
|
container base (or `dnf --installroot` into a directory on this host —
|
|
|
- no runtime needed), `dnf install` the ch2 checklist + `gcc-c++ patch` +
|
|
|
|
|
- usm's runtime closure, then tar it up. Fully reproducible-ish, fast.
|
|
|
|
|
|
|
+ no runtime needed), `dnf install` the ch2 checklist +
|
|
|
|
|
+ `gcc-c++ patch texinfo` + usm's runtime closure, then tar it up.
|
|
|
|
|
+ Fully reproducible-ish, fast.
|
|
|
- **(b) host-clone (proven here).** Clone the sandbox host root, add the
|
|
- **(b) host-clone (proven here).** Clone the sandbox host root, add the
|
|
|
missing pieces via dnf, prune to the ch2 list. Faster to stand up (the
|
|
missing pieces via dnf, prune to the ch2 list. Faster to stand up (the
|
|
|
bison proof used exactly this), less hermetic; fine for development,
|
|
bison proof used exactly this), less hermetic; fine for development,
|
|
@@ -396,13 +416,16 @@ a full machine); every stage's output feeds the next.
|
|
|
Purpose: a hermetic chapter-2-compliant build host. Recipe (dnf-based):
|
|
Purpose: a hermetic chapter-2-compliant build host. Recipe (dnf-based):
|
|
|
|
|
|
|
|
1. `dnf --installroot=$ROOT --releasever=44 install @build-tools` or an
|
|
1. `dnf --installroot=$ROOT --releasever=44 install @build-tools` or an
|
|
|
- explicit list from the book's ch2 table, **plus `gcc-c++` and `patch`**
|
|
|
|
|
- (the two gaps found on this sandbox — see §2.3), plus usm's runtime
|
|
|
|
|
- closure: `curl git libsodium glib2 json-glib` (Fedora names).
|
|
|
|
|
|
|
+ explicit list from the book's ch2 table, **plus `gcc-c++`, `patch` and
|
|
|
|
|
+ `texinfo`** (the ch2 gaps found on this sandbox — see §2.3; `bison`
|
|
|
|
|
+ provides `yacc` only via a separate symlink the image recipe must
|
|
|
|
|
+ create), plus usm's runtime closure: `curl git libsodium glib2
|
|
|
|
|
+ json-glib libgee vala gobject-introspection` (Fedora names).
|
|
|
2. Install `usm` itself (any build that runs on the host glibc — the
|
|
2. Install `usm` itself (any build that runs on the host glibc — the
|
|
|
Fedora dnf set is fine; this stage-0 copy never needs to be the
|
|
Fedora dnf set is fine; this stage-0 copy never needs to be the
|
|
|
self-hosted one).
|
|
self-hosted one).
|
|
|
-3. Fetch/verify the ~94 LFS sources (URLs+md5 from `packages.ent`), or
|
|
|
|
|
|
|
+3. Fetch/verify the ~93 LFS downloads listed in ch3 (URLs+md5 from
|
|
|
|
|
+ `packages.ent`; ~85 are source tarballs, the rest docs/extras), or
|
|
|
vendor them into the image; pre-seed the file:// repo layout.
|
|
vendor them into the image; pre-seed the file:// repo layout.
|
|
|
4. Assemble the OCI image (script) — or just keep it as a directory tree
|
|
4. Assemble the OCI image (script) — or just keep it as a directory tree
|
|
|
for chroot use on this sandbox.
|
|
for chroot use on this sandbox.
|
|
@@ -424,10 +447,13 @@ checkout and emits one scaffold dir per package-pass:
|
|
|
`PATH`/`LFS_TGT` env set by the stage driver, `make -j$(nproc)`.
|
|
`PATH`/`LFS_TGT` env set by the stage driver, `make -j$(nproc)`.
|
|
|
- `usm-scripts/install.sh` — the book's install commands; DESTDIR-style
|
|
- `usm-scripts/install.sh` — the book's install commands; DESTDIR-style
|
|
|
staging is NOT what LFS does (it installs into `/`/`/tools` inside the
|
|
staging is NOT what LFS does (it installs into `/`/`/tools` inside the
|
|
|
- chroot), so install scripts write directly and USM's per-package
|
|
|
|
|
- install staging is bypassed via `install_path == destination` (or the
|
|
|
|
|
- scripts `cp` from a destdir; per-scaffold choice, recorded in the
|
|
|
|
|
- generator's overrides).
|
|
|
|
|
|
|
+ chroot). USM runs install scripts with a per-package install dir as
|
|
|
|
|
+ `DESTDIR` (Transaction.vala passes `create_install_directory()` into
|
|
|
|
|
+ `run_install`, which overrides `destination` — Manifest.vala:441-455);
|
|
|
|
|
+ LFS scaffolds must therefore either install into that DESTDIR and let
|
|
|
|
|
+ the stage driver move its contents into place, or use the book's direct
|
|
|
|
|
+ paths (`/`/`/tools`) from inside the chroot. Per-scaffold choice,
|
|
|
|
|
+ recorded in the generator's overrides.
|
|
|
- vendored patches from the book.
|
|
- vendored patches from the book.
|
|
|
|
|
|
|
|
Output: the scaffold git repo (reviewed, committed) → `*.usmc` via
|
|
Output: the scaffold git repo (reviewed, committed) → `*.usmc` via
|
|
@@ -462,12 +488,40 @@ system whose every file was placed by a USM transaction.
|
|
|
|
|
|
|
|
### Stage 5 — beyond-LFS self-hosting set + USM itself
|
|
### Stage 5 — beyond-LFS self-hosting set + USM itself
|
|
|
|
|
|
|
|
-The book does not contain USM's build deps. The gap set (from the plan's
|
|
|
|
|
-evidence, re-confirmed against the book's package list): the book HAS
|
|
|
|
|
-pcre2 (chapter08/pcre.xml), libffi, python, meson, ninja, pkgconf,
|
|
|
|
|
-elfutils/libelf — so the beyond-LFS set is small: **glib, libgee, valac,
|
|
|
|
|
-json-glib, curl** (+ optional git). Order matters: glib → libgee →
|
|
|
|
|
-json-glib → valac → USM. Then `usm install usm` (from a `.usmc` of USM's
|
|
|
|
|
|
|
+The book does not contain USM's build deps. The gap set, re-derived this
|
|
|
|
|
+session from USM's own `MANIFEST.usm` (which is the authoritative
|
|
|
|
|
+dependency list) plus its direct library deps: the book HAS pcre2
|
|
|
|
|
+(chapter08/pcre.xml), libffi, python, meson, ninja, pkgconf, zlib, xz,
|
|
|
|
|
+util-linux (libmount/libblkid), elfutils/libelf. **libselinux is NOT
|
|
|
|
|
+in the book**, yet USM's manifest names `lib:libselinux.so.1` as a
|
|
|
|
|
+runtime ref (it arrives via glib's linkage on a typical distro).
|
|
|
|
|
+Beyond-LFS set, from the manifests:
|
|
|
|
|
+
|
|
|
|
|
+- **glib** (glib/gobject/gio/gmodule; pulls libffi+pcre2+util-linux from
|
|
|
|
|
+ the book, plus **libselinux** which must be added beyond-LFS — or glib
|
|
|
|
|
+ rebuilt `-Dselinux=disabled`, in which case USM's `lib:libselinux.so.1`
|
|
|
|
|
+ runtime ref needs relaxing),
|
|
|
|
|
+- **libgee**, **json-glib**, **valac**,
|
|
|
|
|
+- **invercargill** and **invercargill-json** (USM's own base libraries —
|
|
|
|
|
+ `lib:libinvercargill-1.so`, `pc:invercargill-1.pc`,
|
|
|
|
|
+ `pc:invercargill-json.pc` are hard runtime/build refs of USM; both are
|
|
|
|
|
+ vala/meson packages needing only glib+gobject, so they slot after
|
|
|
|
|
+ json-glib),
|
|
|
|
|
+- **curl** (http(s) repositories at runtime; `usm dcba add-repo` for
|
|
|
|
|
+ remote repos too) (+ optional git),
|
|
|
|
|
+- **libsodium** (USM's ed25519 signing links `libsodium`; build dep via
|
|
|
|
|
+ vapi, runtime `lib:libsodium.so.26`),
|
|
|
|
|
+- **gobject-introspection** — USM's build deps include
|
|
|
|
|
+ `pc:gobject-introspection-1.0.pc` (optional in principle but currently
|
|
|
|
|
+ required by the manifest); it needs glib and Python, both present by
|
|
|
|
|
+ this point in stage 5.
|
|
|
|
|
+
|
|
|
|
|
+So the honest count is **~10 beyond-LFS packages** (glib, libselinux or a
|
|
|
|
|
+glib flag change, libgee, json-glib, gobject-introspection, valac,
|
|
|
|
|
+invercargill, invercargill-json, libsodium, curl), not the ~6 the plan's
|
|
|
|
|
+evidence suggested. Order: glib (+libselinux) → libgee → json-glib →
|
|
|
|
|
+gobject-introspection → valac → invercargill → invercargill-json →
|
|
|
|
|
+libsodium → curl → USM. Then `usm install usm` (from a `.usmc` of USM's
|
|
|
own repo — USM's MANIFEST.usm exists at the repo root) is the
|
|
own repo — USM's MANIFEST.usm exists at the repo root) is the
|
|
|
self-hosting milestone: the distro's package manager installs itself.
|
|
self-hosting milestone: the distro's package manager installs itself.
|
|
|
|
|
|
|
@@ -482,7 +536,7 @@ verification, or skip assembly entirely and use `usm manifest deploy`.
|
|
|
|
|
|
|
|
## 6. Why staged (the resolver argument, precisely)
|
|
## 6. Why staged (the resolver argument, precisely)
|
|
|
|
|
|
|
|
-`Resolver.resolve` (src/lib/Resolver.vala:232) computes ONE resolved set
|
|
|
|
|
|
|
+`Resolver.resolve` (src/lib/Resolver.vala:233) computes ONE resolved set
|
|
|
with ONE topological order over `manage`+`build`+`runtime` refs
|
|
with ONE topological order over `manage`+`build`+`runtime` refs
|
|
|
(src/lib/Resolver.vala:422-428). LFS needs three orders interleaved:
|
|
(src/lib/Resolver.vala:422-428). LFS needs three orders interleaved:
|
|
|
binutils-p1 → gcc-p1 → (ch5 libs) → gcc-p2 must complete *before* ch7's
|
|
binutils-p1 → gcc-p1 → (ch5 libs) → gcc-p2 must complete *before* ch7's
|
|
@@ -502,17 +556,23 @@ commands and stock resolver error reports.
|
|
|
|
|
|
|
|
### 7.1 Wall-clock (build time)
|
|
### 7.1 Wall-clock (build time)
|
|
|
|
|
|
|
|
-SBU data is machine-readable in `packages.ent` (entities
|
|
|
|
|
-`<pkg>-<tmp|fin>-sbu`). Summed this session (test-suite entities excluded,
|
|
|
|
|
-"less than X" counted as X/2):
|
|
|
|
|
-
|
|
|
|
|
-- ch5/6 temporary tools (`-tmp-sbu`, 33 entries): **~21 SBU**
|
|
|
|
|
- (gcc-p2 5.2, gcc-p1 4.3, tcl 2.9, gettext 1.5, glibc-tmp 1.3, …).
|
|
|
|
|
-- ch8 final system (`-fin-sbu`, 82 entries): **~100 SBU**
|
|
|
|
|
- (gcc 53, glibc 11, vim 3.3, python 2.7, e2fsprogs 2.4, …).
|
|
|
|
|
-- Linux kernel (~32 SBU) is *not* part of the bootstrap (no boot
|
|
|
|
|
- requirement for a container/chroot distro); building it doubles the
|
|
|
|
|
- gcc-dominated budget if wanted.
|
|
|
|
|
|
|
+SBU data is machine-readable in `packages.ent` (entities named
|
|
|
|
|
+`<pkg>-<tmp|tmpp1|tmpp2|fin>-sbu`). Summed this session (separate
|
|
|
|
|
+test-suite entities excluded, "less than X" counted as X/2, e2fsprogs'
|
|
|
|
|
+spinning-disk figure taken):
|
|
|
|
|
+
|
|
|
|
|
+- ch5/6 temporary tools (`-tmp*`-sbu entities: 33 of them, counting
|
|
|
|
|
+ binutils/gcc pass1+pass2): **~21 SBU** (gcc-p2 5.2, gcc-p1 4.3, tcl 2.9,
|
|
|
|
|
+ gettext 1.5, glibc-tmp 1.3, …).
|
|
|
|
|
+- ch8 final system (`-fin-sbu`, 82 entries): **~100 SBU** (gcc 53,
|
|
|
|
|
+ glibc 11, vim 3.3, python 2.7, e2fsprogs 2.4 on spinning disk / 0.4 on
|
|
|
|
|
+ SSD, …). **The gcc entity is explicitly "53 SBU (with tests)"** — the
|
|
|
|
|
+ book publishes no tests-free gcc figure, so the sum double-counts gcc's
|
|
|
|
|
+ tests as if they were build time; treat ~100 as an upper bound for
|
|
|
|
|
+ build-only; the no-tests estimate below keeps that headroom.
|
|
|
|
|
+- Linux kernel ("0.4 - 32 SBU, typically about 2.5 SBU") is *not* part of
|
|
|
|
|
+ the bootstrap (no boot requirement for a container/chroot distro);
|
|
|
|
|
+ adding it costs ~2.5 SBU typical / up to ~32 SBU worst-case.
|
|
|
|
|
|
|
|
One SBU = one core's binutils-p1 time, typically **~1-2 min** on modern
|
|
One SBU = one core's binutils-p1 time, typically **~1-2 min** on modern
|
|
|
hardware (the book: gcc ~5 min "on the fastest systems"). With ~120 SBU
|
|
hardware (the book: gcc ~5 min "on the fastest systems"). With ~120 SBU
|
|
@@ -521,12 +581,14 @@ total and `make -j28`, the parallelisable `make` phases compress roughly
|
|
|
(glibc's `make install`, gcc's early stages) do not. Honest estimate for
|
|
(glibc's `make install`, gcc's early stages) do not. Honest estimate for
|
|
|
one full stage-0→4 pass **on this 28-core sandbox**:
|
|
one full stage-0→4 pass **on this 28-core sandbox**:
|
|
|
|
|
|
|
|
-- without test suites: **~3-5 hours** (gcc dominates: its 53 SBU ≈
|
|
|
|
|
- 1.5-2 h alone at j28 with serial configure/install).
|
|
|
|
|
|
|
+- without test suites: **~3-5 hours** (gcc dominates; discounting a
|
|
|
|
|
+ third to a half of its 53 SBU as test time still leaves ≈ 1-2 h alone
|
|
|
|
|
+ at j28 with serial configure/install — inside this range).
|
|
|
- with the book's recommended test suites: **~8-14 hours** (tests are
|
|
- with the book's recommended test suites: **~8-14 hours** (tests are
|
|
|
mostly serial and several are multi-SBU).
|
|
mostly serial and several are multi-SBU).
|
|
|
-- Stage 5 (beyond-LFS + USM): ~30-45 min (glib ~0.5 SBU, valac ~2 SBU,
|
|
|
|
|
- USM itself < 1 SBU at j28).
|
|
|
|
|
|
|
+- Stage 5 (beyond-LFS, ~10 packages + USM): ~45-75 min (glib ~0.5 SBU,
|
|
|
|
|
+ valac ~2 SBU, gobject-introspection ~1-2 SBU, curl ~1 SBU, the two
|
|
|
|
|
+ invercargills and USM itself well under 1 SBU each at j28).
|
|
|
- A DCBA-cached re-run of stage 4 (iteration loop): **~20-40 min** — this
|
|
- A DCBA-cached re-run of stage 4 (iteration loop): **~20-40 min** — this
|
|
|
is the number that matters for daily development.
|
|
is the number that matters for daily development.
|
|
|
|
|
|
|
@@ -546,13 +608,15 @@ USM and LFS):
|
|
|
- DCBA publish driver + per-stage repos: **2-4 days**
|
|
- DCBA publish driver + per-stage repos: **2-4 days**
|
|
|
- Stage orchestrator (chroot mgmt, /dev workarounds, checkpoints):
|
|
- Stage orchestrator (chroot mgmt, /dev workarounds, checkpoints):
|
|
|
**1-1.5 weeks**
|
|
**1-1.5 weeks**
|
|
|
-- Beyond-LFS set (6 packages, valac ordering) + USM self-host: **3-5 days**
|
|
|
|
|
|
|
+- Beyond-LFS set (~10 packages, valac + gobject-introspection
|
|
|
|
|
+ ordering) + USM self-host: **4-6 days**
|
|
|
- OCI final image + smoke tests + documentation: **2-3 days**
|
|
- OCI final image + smoke tests + documentation: **2-3 days**
|
|
|
- Integration/debug of the first full run + book-drift tooling
|
|
- Integration/debug of the first full run + book-drift tooling
|
|
|
(regeneration diff): **1 week**
|
|
(regeneration diff): **1 week**
|
|
|
|
|
|
|
|
-**Total: ~6-9 engineer-weeks** (the spread is almost entirely the
|
|
|
|
|
-generator: 94 packages × curated overrides vs mostly-mechanical).
|
|
|
|
|
|
|
+**Total: ~7-10 engineer-weeks** (summing the lines above at 5-day weeks:
|
|
|
|
|
+36-50 working days; the spread is almost entirely the generator: ~90
|
|
|
|
|
+packages × curated overrides vs mostly-mechanical).
|
|
|
Uncertainty: a first full run will surface 5-15 per-package script bugs
|
|
Uncertainty: a first full run will surface 5-15 per-package script bugs
|
|
|
(extraction artefacts, env differences) — that's inside the integration
|
|
(extraction artefacts, env differences) — that's inside the integration
|
|
|
line above; a systemd-revision build adds ~3-5 days if pursued (and needs
|
|
line above; a systemd-revision build adds ~3-5 days if pursued (and needs
|
|
@@ -588,12 +652,16 @@ a full host for /proc reasons, §8).
|
|
|
- **Book drift.** LFS 13.1 → 13.2+ bumps package versions and commands.
|
|
- **Book drift.** LFS 13.1 → 13.2+ bumps package versions and commands.
|
|
|
Mitigation: the generator is the only thing that reads the book;
|
|
Mitigation: the generator is the only thing that reads the book;
|
|
|
regeneration diff (git diff of scaffolds) makes drift reviewable.
|
|
regeneration diff (git diff of scaffolds) makes drift reviewable.
|
|
|
-- **valac bootstrap ordering.** valac needs glib+libgee; USM needs valac
|
|
|
|
|
- + glib + json-glib. Fixed linear order in stage 5 (glib → libgee →
|
|
|
|
|
- json-glib → valac → USM); no cycle exists because valac's own
|
|
|
|
|
- compilation can use a stage-0-hosted valac (dnf-installed) — the dnf
|
|
|
|
|
- set already includes one, so USM's first self-host can even avoid
|
|
|
|
|
- valac-from-source if desired.
|
|
|
|
|
|
|
+- **valac bootstrap ordering.** USM needs valac + glib + libgee +
|
|
|
|
|
+ json-glib + invercargill ×2 (+ libsodium, gobject-introspection at
|
|
|
|
|
+ build time). valac's own build (per its LFS-blfs recipe and the dnf
|
|
|
|
|
+ package) needs glib (libgee is a common *consumer*, not a valac
|
|
|
|
|
+ prerequisite). Fixed linear order in stage 5 (glib → libgee →
|
|
|
|
|
+ json-glib → gobject-introspection → valac → invercargill →
|
|
|
|
|
+ invercargill-json → libsodium → curl → USM); no cycle exists because
|
|
|
|
|
+ valac's compilation can use a stage-0-hosted valac (dnf-installed) —
|
|
|
|
|
+ the dnf set already includes one, so USM's first self-host can even
|
|
|
|
|
+ avoid valac-from-source if desired.
|
|
|
- **ed25519 key management.** `keys/` 0600, private keys never leave the
|
|
- **ed25519 key management.** `keys/` 0600, private keys never leave the
|
|
|
producer; `.dcbar`/`.usmr` carry the public key. Risk: losing the key
|
|
producer; `.dcbar`/`.usmr` carry the public key. Risk: losing the key
|
|
|
orphanes a repo — keep one offline master per distro release line, and
|
|
orphanes a repo — keep one offline master per distro release line, and
|
|
@@ -602,11 +670,12 @@ a full host for /proc reasons, §8).
|
|
|
scratch under /workspace (11 TB). Checkpoint tars double usage
|
|
scratch under /workspace (11 TB). Checkpoint tars double usage
|
|
|
transiently; clean between stages.
|
|
transiently; clean between stages.
|
|
|
- **gcc/binutils newer than the book** are explicitly unsupported by LFS
|
|
- **gcc/binutils newer than the book** are explicitly unsupported by LFS
|
|
|
- (ch2 warning). This host's gcc 16.2.1 == book's 16.2.0 line — fine;
|
|
|
|
|
- any host bump past the book needs a book bump first. Relatedly, the
|
|
|
|
|
- sandbox host itself lacks `g++`/`patch` (ch2 requirements) — with root
|
|
|
|
|
- and dnf5 this is a non-event: stage 0 installs them, and a ch2-check
|
|
|
|
|
- script fails fast on any host still missing items.
|
|
|
|
|
|
|
+ (ch2 warning). This host's gcc 16.2.1 sits on the book's 16.2.0 line
|
|
|
|
|
+ and its binutils 2.46.1 is *older* than the book's 2.47 — fine; any
|
|
|
|
|
+ host bump past the book needs a book bump first. Relatedly, the
|
|
|
|
|
+ sandbox host lacks `g++`/`patch`/`makeinfo` (ch2 requirements) — with
|
|
|
|
|
+ root and dnf5 this is a non-event: stage 0 installs them, and a
|
|
|
|
|
+ ch2-check script fails fast on any host still missing items.
|
|
|
|
|
|
|
|
## 9. Recommended task breakdown (for the implementation goal)
|
|
## 9. Recommended task breakdown (for the implementation goal)
|
|
|
|
|
|
|
@@ -616,7 +685,7 @@ Ordered; each is independently verifiable and lands value on its own:
|
|
|
bootstrap image from dnf; verify by chroot + book ch2 script.
|
|
bootstrap image from dnf; verify by chroot + book ch2 script.
|
|
|
2. **Generator v1: packages.ent → scaffold skeletons** (3-5 d) — manifest
|
|
2. **Generator v1: packages.ent → scaffold skeletons** (3-5 d) — manifest
|
|
|
fields, URLs/md5s, acquire scripts; build.sh placeholders; no commands
|
|
fields, URLs/md5s, acquire scripts; build.sh placeholders; no commands
|
|
|
- yet. Deliverable: 94 scaffolds packaging cleanly (`usm manifest
|
|
|
|
|
|
|
+ yet. Deliverable: ~90 scaffolds packaging cleanly (`usm manifest
|
|
|
package` all green).
|
|
package` all green).
|
|
|
3. **Generator v2: chapter XML → build/install scripts + overrides
|
|
3. **Generator v2: chapter XML → build/install scripts + overrides
|
|
|
table** (1.5-2 wk) — ch5/6 (tmp passes, /tools env), ch7, ch8;
|
|
table** (1.5-2 wk) — ch5/6 (tmp passes, /tools env), ch7, ch8;
|
|
@@ -628,8 +697,8 @@ Ordered; each is independently verifiable and lands value on its own:
|
|
|
repos.
|
|
repos.
|
|
|
6. **Stage 4: final-system build via USM** (3-5 d) — full ch8 through usm
|
|
6. **Stage 4: final-system build via USM** (3-5 d) — full ch8 through usm
|
|
|
transactions with DCBA caches.
|
|
transactions with DCBA caches.
|
|
|
-7. **Stage 5: beyond-LFS set + USM self-host** (3-5 d) — the milestone:
|
|
|
|
|
- `usm install usm` inside the final root.
|
|
|
|
|
|
|
+7. **Stage 5: beyond-LFS set (~10 packages) + USM self-host** (4-6 d) —
|
|
|
|
|
+ the milestone: `usm install usm` inside the final root.
|
|
|
8. **Stage 6: final OCI image + smoke tests** (2-3 d) — assembled image,
|
|
8. **Stage 6: final OCI image + smoke tests** (2-3 d) — assembled image,
|
|
|
chroot/podman verification matrix, docs.
|
|
chroot/podman verification matrix, docs.
|
|
|
9. **(Follow-up, separate goal) systemd revision, kernel build, https
|
|
9. **(Follow-up, separate goal) systemd revision, kernel build, https
|
|
@@ -644,5 +713,5 @@ enough for a generator with a modest overrides table, and the two hard
|
|
|
environmental constraints of this sandbox (no device-node creation, no
|
|
environmental constraints of this sandbox (no device-node creation, no
|
|
|
mounts) are worked around for every stage except a systemd build. The
|
|
mounts) are worked around for every stage except a systemd build. The
|
|
|
recommended first implementation milestone is tasks 1+2: a verified
|
|
recommended first implementation milestone is tasks 1+2: a verified
|
|
|
-bootstrap image and 94 packaging-clean scaffolds — roughly a week of
|
|
|
|
|
|
|
+bootstrap image and ~90 packaging-clean scaffolds — roughly a week of
|
|
|
work that de-risks everything downstream.
|
|
work that de-risks everything downstream.
|