usm-web

usm-web

A Statum + Spry application that browses and serves any number of USM repositories: human-facing pages for exploring packages (an all-repository overview plus per-repository browse views), plus the exact HTTP endpoints a real USM client (usm repository list / verify / install) expects to fetch from a repository host. Alongside USM source repositories it also serves DCBA artifact repositories — repositories of pre-built build artifacts (.dcba) that usm install fetches instead of building a package from source (usm dcba add-repo / usm dcba repository list).

What it serves

Endpoint	Purpose
GET /	Overview: every repository of both kinds as a card (name, summary, chip, kind-appropriate counts and sudo usm add-repo / sudo usm dcba add-repo instructions), the global install-USM block, and a source-package-only combined table (usmc rows from USM repositories, repo badges) with server-side search across them — DCBA repositories appear as cards, never as package rows
GET /repo/{name}	Single-repo browse: that repository's overview card and instructions. A USM repository renders its package table with search scoped to it; a DCBA repository renders an artifacts table (name, version, size, download) instead — search stays source-package-only
GET /repo/{name}/package/{file}	Package detail: manifest fields read out of the .usmc (licences, flags, provides/depends tables), sha512, on-disk size/mtime, download button, plus a "Pre-built artifacts" section listing every DCBA repository carrying an artifact for this exact name + version
GET /repo/{name}/artifact/{file}	Artifact detail (DCBA repositories): listing-derived name/version, sha512, on-disk size/mtime, download button, the usm dcba add-repo instruction and — on an exact name+version match — a one-directional link to the source package's detail page
GET /repo/{name}/PACKAGES.usml	That USM repository's listing streamed verbatim (application/json); a DCBA repository has none and answers 404
GET /repo/{name}/ARTIFACTS.usml	That DCBA repository's artifact listing streamed verbatim (application/json) so signatures and checksums stay valid; a USM repository answers 404
GET /repo/{name}/repo.usmr	A rewritten copy of the .usmr with its embedded url replaced by the derived base + repo/{name}/ — the URL sudo usm add-repo consumes
GET /repo/{name}/repo.dcbar	The rewritten DCBA counterpart: a copy of the .dcbar with its url replaced by the derived base + repo/{name}/ — the URL sudo usm dcba add-repo consumes (the route name is canonical regardless of the on-disk descriptor's filename)
GET /repo/{name}/{file}.usmc / GET /repo/{name}/{file}.dcba	Package archives / pre-built artifacts streamed from that repository's public/ (application/octet-stream, exact content-length); the suffix selects the repository kind, so the wrong kind (or a listing the other kind owns) answers 404
GET /install-usm.sh	The configured installer script (installer.path mode only), served by the root-level catch-all

The derived base is base_url from the config when set, else X-Forwarded-Proto/X-Forwarded-Host, else the request's own scheme/host — so clients behind a reverse proxy resolve against the public name. Each repository is addressed by the name in its .usmr or .dcbar, which is also its routing segment.

Configuration

web-config.json (in the app's working directory, or ASTRALIS_CONFIG_PATH) carries the static Statum keys plus an optional "usm-web" section:

{
  "statum": { "…": "static keys from spry keys / statum-genkeys" },
  "usm-web": {
    "repo": "/repo",
    "repositories": ["web-stack", "extra"],
    "installer": { "path": "install-usm.sh" },
    "base_url": "https://repo.example.com/",
    "name": "My repositories"
  }
}

Key	Meaning
repo	The parent directory holding the repositories (each a subdirectory with a .usmr or a .dcbar plus public/); relative paths resolve against the config file's directory
repositories	Optional explicit list of repository subdirectories to serve, of either kind, in that order; when absent or empty every subdirectory containing a .usmr or a .dcbar is discovered by scanning
installer.url	External installer location; the homepage links out and shows curl -fsSL <url> \| sh
installer.path	Installer script this app serves at /install-usm.sh; relative paths resolve against the config file's directory
base_url	Optional override of the request-derived repository base (normalised to a trailing slash)
name	Optional overview-title override

installer takes exactly one of url/path; providing both (or neither, when the block should appear) is a configuration error. Omit the section entirely to hide the install-USM card.

The repository parent is selected by (in order) the --repo <dir> argument, the USM_WEB_REPO_DIR environment variable, the config's repo key, or the /repo default, and is expected to look like:

/repo
├── web-stack/
│   ├── web-stack.usmr
│   ├── keys/            (not used by usm-web; signing is upstream)
│   └── public/
│       ├── PACKAGES.usml
│       └── <name>-<version>.usmc
├── prebuilt/
│   ├── prebuilt.dcbar   (same JSON shape as a .usmr)
│   ├── keys/
│   └── public/
│       ├── ARTIFACTS.usml
│       └── <name>-<version>.dcba
└── extra/
    └── …

USM source repositories and DCBA artifact repositories live side by side under the same parent — there is no separate configuration for the second kind. A subdirectory is classified by its descriptor: .usmr makes it a USM source repository (.usmc packages in public/PACKAGES.usml), .dcbar a DCBA artifact repository (.dcba artifacts in public/ARTIFACTS.usml). A directory holding both descriptors is a configuration error; split it into two subdirectories instead.

A legacy single-repository layout with the .usmr directly inside the configured directory is still served as one repository (this fallback stays .usmr-only; a bare .dcbar in the parent is not discovered). Discovery re-runs on every request, so repositories added or removed on disk are picked up without a restart, and each repository's descriptor (.usmr/.dcbar) and listing (PACKAGES.usml/ARTIFACTS.usml) re-read when their own mtimes change — one repository can be rebuilt without disturbing the others. There is no database and no authentication; all Statum state is stateless PAGE slots.

DCBA repositories

A DCBA repository distributes pre-built build artifacts: one .dcba archive per package name + version (built and published upstream with usm dcba repository init / add / publish), described by a .dcbar (the same JSON shape as a .usmr) and listed in a signed public/ARTIFACTS.usml. With such a repository configured, usm install fetches the artifact instead of building the package from source.

Two things differ from the usmc flow when pointing a client at one:

• the verb carries the dcba subcommand: sudo usm dcba add-repo <url> (not sudo usm add-repo), where <url> is the served .../repo/<name>/repo.dcbar rewrite;
• clients match artifacts by plain name + version string comparison, so the package detail pages surface pre-built artifacts only when a DCBA repository carries that exact version — a newer or older artifact version never shows up as "for" that package.

The views keep the two kinds apart: the home page lists DCBA repositories as cards (with artifact counts and their own add-repo instruction) but the package table, the Packages/Total-size hero counts and search stay usmc-only; a DCBA repository's browse page shows its artifacts table, and each artifact has a detail page with its sha512 and a one-directional link to the matching source package (a package page then lists every DCBA repository carrying its pre-built artifacts, so the pair is discoverable from both sides).

Development

spry dev

Builds, runs the app on port 8080 (--port N to change) and watches src/, meson.build and web-config.json: saving a file rebuilds incrementally — page HTML and resources recompile via the meson codegen targets — and restarts the app when the build succeeds.

The environment below is required for the underlying meson builds (the spry dev child processes set it for you; export it in shells where you build or run manually):

export WS_PREFIX="$HOME/.local"
export PKG_CONFIG_PATH="$WS_PREFIX/lib64/pkgconfig${PKG_CONFIG_PATH:+:$PKG_CONFIG_PATH}"
export XDG_DATA_DIRS="$WS_PREFIX/share${XDG_DATA_DIRS:+:$XDG_DATA_DIRS}"
export LD_LIBRARY_PATH="$WS_PREFIX/lib64${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"

meson setup builddir
ninja -C builddir
./builddir/usm-web 8080 --repo ../repos

--repo points at the parent directory holding the repositories (see the layout above). Repo parsing uses libusm (Usm.Repository, Usm.RepositoryListing, Usm.DcbaListing, Usm.Manifest.from_package) — no hand-rolled usml/usmr/dcbar mappers.

Layout

Path	Purpose
src/main.vala	application wiring: modules, config, pages, action, endpoints
src/UsmWebConfig.vala	the "usm-web" config section (repo parent, repositories, installer url|path, base_url, name)
src/RepositoryService.vala	multi-repository discovery (.usmr and .dcbar subdirectories, kind-guarded members) + libusm-backed read side with per-repo mtime reload
src/DerivedBase.vala	base-URI derivation from config/forwarded headers
src/PackagesState.vala	shared packages slot-state builder (overview + repo-scoped, both repository kinds, DCBA artifact rows) and size/fingerprint formatting
src/entrypoints/	HomeEntrypoint (/), RepoBrowseEntrypoint (/repo/{name}), PackageDetailEntrypoint (/repo/{name}/package/{file}) and ArtifactDetailEntrypoint (/repo/{name}/artifact/{file})
src/actions/	SearchPackagesAction — server-side table filtering (all repositories or one)
src/endpoints/	the USM/DCBA-facing endpoints listed above (PackagesListingEndpoint, RepositoryDefinitionEndpoint, ArtifactsListingEndpoint, DcbaDefinitionEndpoint, PackageDownloadEndpoint, InstallerScriptEndpoint)
src/pages/*.html	Statum pages (home, repo, package, artifact), compiled by statum-mkpstm at build time
src/Static/main.css	stylesheet embedded by statum-mkres
MANIFEST.usm	USM package manifest: provides bin:usm-web, build/runtime/manage deps
usm-scripts/	USM build + install scripts (meson staging)
web-config.json	static keys and the usm-web section (gitignored, 0600)

Registrations live between // spry:*-begin/end markers in src/main.vala, # spry:*-begin/end markers in meson.build and <!-- spry:nav-begin/end --> markers in src/pages/main.html; the spry add commands edit only those blocks and are idempotent.

Deployment (USM)

USM itself installs from the canonical installer URL (https://packages.astrologue.nz/install-usm.sh); pass --installer-url file://… to carry a locally built installer into the image instead.

spry deploy --verbose \
  --repository ../web-stack-repo/web-stack.usmr

--system {fedora,debian,ubuntu,alpine,gentoo} is shorthand for the matching base image plus system package manager (mutually exclusive with --spm/--base); with no flags the fedora equivalent applies, so the command above is unchanged. Alpine images build the stack natively on musl; gentoo stage3 compiles the platform from source — allow an hour or more for the image build.

This packages the application and builds a single-stage container image through USM (usm manifest deploy): the MANIFEST.usm + usm-scripts/ drive the in-container build and install, the Web-Stack resolves from the given repository (including the usm package that provides the libusm pc:/vapi: refs this app builds against), and the system package manager provides the platform libraries. The result is usm-web-<version>.image.tar.xz, loadable with podman load -i.

web-config.json is never packaged (.usmignore), so keys are never baked into the image. Bind-mount the repository read-only at /repo and, when you want stable keys or an installer section, mount a config:

podman load -i usm-web-0.1.image.tar.xz
podman run -d -p 8080:8080 \
    -v /path/to/repos-parent:/repo:ro,Z \
    -v $PWD/web-config.json:/run/web-config.json:ro \
    -e ASTRALIS_CONFIG_PATH=/run/web-config.json \
    localhost/usm-web:0.1

/repo inside the container is the repositories parent directory: every subdirectory holding a .usmr or a .dcbar is served at /repo/<name>/….