Add PACKAGING_GUIDE.md — best practices for MANIFEST.usm, dependency declaration, provides, scripts, .usmignore, common pitfalls

PACKAGING_GUIDE.md

@@ -0,0 +1,311 @@
+# USM Packaging Guide
+
+Best practices for writing `MANIFEST.usm` files and creating packages that install correctly across platforms.
+
+---
+
+## The manifest
+
+```json
+{
+    "name": "mylib",
+    "version": "1.2.3",
+    "summary": "A concise one-line description",
+    "licences": ["MIT"],
+    "flags": ["ninjaStyleProgress"],
+    "provides": { "lib:libmylib.so": "as-expected" },
+    "depends": {
+        "runtime": ["lib:libglib-2.0.so.0"],
+        "build": ["bin:valac", "pc:glib-2.0.pc"],
+        "manage": ["bin:bash"]
+    },
+    "execs": {
+        "build": "usm-scripts/build.sh",
+        "install": "usm-scripts/install.sh"
+    }
+}
+```
+
+### Naming
+
+- Package names are lowercase kebab-case (`invercargill-sql`, not `InvercargillSql`)
+- Match the meson project name
+- The version comes from the meson project version — keep them in sync
+
+### Summary
+
+One line, lowercase, no trailing period. This is what `usm search` shows.
+
+---
+
+## Declaring dependencies
+
+### The most common mistake: trusting `ldd`
+
+`ldd` shows what the *linker* resolved at load time on *this specific machine*. It is **not** a complete picture of what your program needs, and it often shows things you don't actually depend on:
+
+| `ldd` says | Reality | Declare it? |
+|---|---|---|
+| `libselinux.so.1` | Indirect via glib — only needed on SELinux systems | **No** — it's a transitive dep of glibc/glib, not yours |
+| `libgcrypt.so.20` | Indirect via glib (gcrypt backend) | **No** — same reason |
+| `liblzma.so.5` | Indirect via libarchive | **No** — if you depend on `lib:libarchive.so`, its own manifest declares lzma |
+| `libglib-2.0.so.0` | You linked against it directly | **Yes** — `lib:libglib-2.0.so.0` |
+| `libjson-glib-1.0.so.0` | You linked against it directly | **Yes** — `lib:libjson-glib-1.0.so.0` |
+| *(missing from ldd)* `bin:valac` | Compile-time tool, not a library | **Yes** in `build` — ldd can't see build tools |
+| *(missing from ldd)* `pc:glib-2.0.pc` | Pkg-config discovery for meson | **Yes** in `build` — ldd can't see header files |
+| *(missing from ldd)* `gio:libgiognutls.so` | GIO TLS plugin loaded at runtime | **Yes** in `runtime` — ldd can't see dlopen'd modules |
+
+**Rule of thumb**: declare what *you* link against or *invoke*, not what your dependencies pull in transitively. Each package's manifest handles its own dependency chain.
+
+### Runtime dependencies (`depends.runtime`)
+
+Things the installed program needs at *run* time:
+
+- **Directly linked libraries**: `lib:libglib-2.0.so.0` (check with `ldd ./myprogram | grep -v linux-vdso`)
+- **GIO plugin modules**: `gio:libgiognutls.so` if the program uses HTTPS via GLib's GIO
+- **Data files**: `res:` refs for share-time resources the program reads
+
+**What NOT to put in runtime:**
+
+- `lib:libc.so.6` — the C library is implicit on every Linux system; musl doesn't even have it
+- `lib:libm.so.6` — same, the math library is always present
+- Transitive dependencies (glib pulls in pcre, ffi, selinux — don't declare those)
+- Build tools (valac, meson — those go in `build`)
+
+### Build dependencies (`depends.build`)
+
+Things needed to *compile* the package from source:
+
+- **Compilers**: `bin:valac`, `bin:gcc` (gcc is not always transitive — alpine's apk doesn't pull it)
+- **Build systems**: `bin:meson`, `bin:ninja`
+- **Pkg-config files**: `pc:glib-2.0.pc`, `pc:json-glib-1.0.pc` (these map to `-dev`/`-devel` system packages)
+- **Code generators**: `bin:g-ir-compiler` (if GIR/typelib are generated), `bin:statum-mkpstm` (for Statum pages)
+- **Scripting**: `bin:python3` if the build uses Python tools
+- **Stack-internal libraries**: `pc:invercargill-1.pc` (resolved from USM repositories, not the system)
+
+**What NOT to put in build:**
+
+- `bin:bash` — that goes in `manage` (build scripts run under the install environment's shell)
+- Runtime-only libraries the build doesn't link against
+
+### Manage dependencies (`depends.manage`)
+
+Things needed to *run the package's management scripts*:
+
+- `bin:bash` — almost always needed (usm-scripts are bash)
+- `bin:curl` — if the scripts download anything
+
+---
+
+## Declaring provides
+
+`provides` tells USM what your package installs and where to find it.
+
+### `"as-expected"` paths
+
+The default for most resources — USM knows the standard install location:
+
+```json
+"lib:libmylib.so": "as-expected",
+"inc:mylib.h": "as-expected",
+"pc:mylib.pc": "as-expected",
+"vapi:mylib.vapi": "as-expected"
+```
+
+USM derives the path from `Paths.get_suggested_base_path_for_type()` + the resource name.
+
+### Explicit paths
+
+For non-standard locations:
+
+```json
+"bin:myhelper": "usr/lib/myapp/myhelper"
+```
+
+### What to provide
+
+Provide what downstream packages might want to depend on:
+
+- `lib:<soname>` — the shared library (use the soname, not the full path)
+- `bin:<name>` — installed executables
+- `pc:<name>.pc` — the pkg-config file
+- `inc:<name>.h` or `inc:<dirname>` — headers (use the include directory name for multi-file headers)
+- `vapi:<name>.vapi` — Vala API definitions
+- `gir:<name>.gir` — GObject introspection XML
+- `typelib:<name>.typelib` — compiled introspection data
+
+**Don't provide** internal implementation details that nothing else should link against.
+
+---
+
+## Dependency groups (alternatives)
+
+When a package can be satisfied by *either* of multiple dependency sets, use the nested array form:
+
+```json
+"depends": {
+    "build": [
+        "bin:valac",
+        ["pc:gtk4.pc", "pc:libadwaita-1.pc"],
+        ["pc:gtk3.pc", "pc:libhandy-1.pc"]
+    ]
+}
+```
+
+USM picks whichever group is fully satisfiable, preferring the one that installs the fewest packages.
+
+**When to use groups:**
+- A library supports multiple backends (e.g., OpenSSL OR GnuTLS)
+- A newer monolithic package replaces two older split ones
+
+**When NOT to use groups:**
+- Optional features (use separate packages or flags instead)
+- Platform-specific deps (declare both; the resolver picks what's available)
+
+---
+
+## Scripts (`usm-scripts/`)
+
+### `build.sh`
+
+```bash
+#!/bin/bash
+set -e
+
+SRC_DIR="$(cd "$(dirname "$0")/.." && pwd)"
+BUILD_DIR="$1"
+PREFIX="${PREFIX:-/usr}"
+LIBDIR="${LIBDIR:-lib}"
+
+cd "$SRC_DIR"
+meson setup "$BUILD_DIR" --prefix="$PREFIX" --libdir="$LIBDIR"
+ninja -C "$BUILD_DIR"
+```
+
+**Rules:**
+- Accept the build directory as `$1` — USM provides it
+- Respect `PREFIX`, `LIBDIR`, `BINDIR`, `INCLUDEDIR` environment variables
+- Be idempotent — ninja handles incremental builds; don't `rm -rf` the build dir
+- Don't install — that's `install.sh`'s job
+
+### `install.sh`
+
+```bash
+#!/bin/bash
+set -e
+
+SRC_DIR="$(cd "$(dirname "$0")/.." && pwd)"
+BUILD_DIR="$1"
+INSTALL_DIR="$2"
+TYPE="$3"
+
+cd "$SRC_DIR"
+DESTDIR="$INSTALL_DIR" meson install -C "$BUILD_DIR"
+```
+
+**Rules:**
+- `$1` = build dir, `$2` = install staging dir (DESTDIR), `$3` = install type (`FRESH`, `UPDATE`, `DOWNGRADE`)
+- Install into the staging dir, not the real system — USM handles the final placement
+- Match the `provides` entries exactly — if you provide `lib:libfoo.so`, the library must actually land where USM expects it
+
+---
+
+## `.usmignore`
+
+Exclude files that shouldn't be in the `.usmc` archive:
+
+```
+builddir*/
+*.sqlite*
+db.sqlite
+web-config.json
+.kilo/
+.git/
+*.tar.xz
+```
+
+**Always exclude:**
+- Build directories (`builddir*/`, `build/`)
+- Databases (`*.sqlite*`)
+- Secrets and keys (`web-config.json`, `*-key*`, `*.env`)
+- VCS (`.git/`, `.hg/`)
+- Editor artifacts (`.kilo/`, `.vscode/`, `*.swp`)
+
+---
+
+## Common pitfalls
+
+### 1. Over-declaring dependencies
+
+**Symptom**: The resolver pulls in packages you don't need; installs are slow.
+
+**Fix**: Remove transitive dependencies. If you depend on `lib:libglib-2.0.so.0`, you don't also need `lib:libpcre2-8.so.0` — glib's own manifest declares that.
+
+### 2. Under-declaring build deps
+
+**Symptom**: Build fails in a container but works on the dev machine.
+
+**Fix**: The dev machine has tools installed that the container doesn't. Check: `bin:gcc` (alpine doesn't pull it), `bin:g-ir-compiler` (GIR generation), `bin:python3` (build tooling).
+
+### 3. Wrong soname in provides
+
+**Symptom**: Downstream packages can't resolve `lib:libfoo.so.1` because you provided `lib:libfoo.so`.
+
+**Fix**: Use the actual soname (what `readelf -d yourlib.so | grep SONAME` reports). The soname includes the version suffix.
+
+### 4. Platform-specific deps on the wrong platform
+
+**Symptom**: `usm install` fails on alpine because `lib:libc.so.6` doesn't exist on musl.
+
+**Fix**: Don't declare the C library. If you must (rare), use dependency groups with platform alternatives.
+
+### 5. GIR/typelib not generated on musl
+
+**Symptom**: Install fails on alpine with "Expected to find file at gir-1.0/..."
+
+**Fix**: Only declare `gir:` and `typelib:` in provides if gobject-introspection is available on all target platforms. If it's not, omit them — they're only needed for Python/JS bindings.
+
+---
+
+## Version bumping
+
+```bash
+# Bump patch version (1.2.3 → 1.2.4)
+usm manifest bump patch
+
+# Bump + git commit + tag
+usm manifest bump minor --git
+
+# Bump + commit + tag + push
+usm manifest bump major --git --push
+
+# Set explicit version
+usm manifest bump 2.0.0
+
+# Manage the rebuildDependants flag
+usm manifest bump patch --rebuild-dependants
+usm manifest bump patch --no-rebuild-dependants
+```
+
+Only set `rebuildDependants` when the update includes C API changes that break downstream compilation. Don't set it for bug fixes or internal changes.
+
+---
+
+## Testing your package
+
+```bash
+# Validate the manifest
+usm manifest validate
+
+# Build + install to a scratch DESTDIR
+USM_DESTDIR=/tmp/kilo/test-stage usm manifest install
+
+# Package it
+usm manifest package
+
+# Add to a scratch repository and test resolution
+cd /tmp/kilo/test-repo && usm repository init test && usm repository add ../mylib-1.0.0.usmc
+USM_CONFIGDIR=/tmp/kilo/test-config usm install mylib
+```
+
+Verify that every `provides` entry actually exists at the expected location in the DESTDIR. If it doesn't, either the install script or the manifest is wrong.