cc-ci/machine-docs/JOURNAL-canon.md

JOURNAL — phase canon (canonical sweep, make it real)

Builder reasoning log. WHY lives here; WHAT/HOW/EXPECTED/WHERE live in STATUS-canon.md.

2026-06-17 — bootstrap / code survey

Read the phase canon (plan-phase-canon-canonical-sweep.md) + plan.md §6.1/§7/§9. Surveyed the existing canonical/sweep machinery before designing. Key findings:

Clone identity

/srv/cc-ci is a symlink → /srv/cc-ci-orch; the env's two "working dirs" are the same directory. This IS the Builder clone (reflog shows the claim(M2)/status(samever) ## DONE commits). The Adversary cold-verifies from its own fresh clones. No collision.

What already works (phase doc is partly stale)

• The phase doc says "ZERO canonical.json exist". Not true any more: a real canonical for custom-html exists on the host at /var/lib/ci-warm/custom-html/canonical.json (version 1.13.0+1.31.1, commit 2b82eba…, status idle, ts 20260617T050314Z) with its retained data volume warm-custom-html_..._content. It was produced by a manual cold run during the samever phase, NOT by the timer. So the promote primitive (seed_canonical → write_registry + warmsnap) demonstrably works; the sweep that should drive it is what's hollow.

The real "hollow sweep" defect (root cause, confirmed live)

The deployed nightly-sweep.timer fired 2026-06-17 03:09 and logged: ===== nightly cold sweep: enrolled canonicals = [] ===== → a true no-op. Cause: nightly_sweep.py does REPO = os.environ.get("CCCI_REPO", "/root/cc-ci") then sys.path.insert(0, REPO/runner); from harness import canonical. The systemd unit (nix/modules/nightly-sweep.nix) sets no CCCI_REPO, and /root/cc-ci does not exist on the host. So the import falls through to the harness packaged in the nix store (runnerSrc=../../runner — runner/ only, NO tests/). meta.TESTS_DIR = ROOT/tests then points at a nonexistent dir → enrolled_recipes() swallows the OSError → []. Even though custom-html is enrolled in the repo, the deployed timer never sees it. This is the machinery that was "specified but never doing anything." Fix: point the sweep at a real, current checkout that has tests/.

How current code stays live on the host

• Normal recipe CI: Drone exec pipeline auto-clones cc-ci per build into its workspace, then runs cc-ci-run runner/run_recipe_ci.py from that fresh clone → tests/runner always current.
• /etc/cc-ci is a git clone (the nixos flake source: nixos-rebuild --flake /etc/cc-ci#…). It is currently STALE (e60415d, far behind main) because recent phases only touched runner/ (picked up by Drone's fresh clone) and needed no nixos-rebuild. The sweep is the first thing that needs /etc/cc-ci current.
• Plan: sweep service sets CCCI_REPO=/etc/cc-ci and runs nightly_sweep.py FROM the checkout (change the nix to exec $CCCI_REPO/runner/nightly_sweep.py, not the store copy) → after a deploy that does git -C /etc/cc-ci pull && nixos-rebuild, the sweep reads current tests/ + runner. This reuses the flake-source checkout (declarative, reproducible) rather than inventing a new clone.

Promote path (the core, §2.A)

• should_promote_canonical(recipe, ref, overall, quick) = enrolled & green & cold(not quick) & not-ref (no PR head). promote_canonical deploys latest_version(recipe_tags(recipe)) (the latest git tag) fresh/in-place, waits healthy, undeploys, seed_canonical (snapshot + write_registry).
• Tagged-promote addition needed: the green gate currently tests whatever fetch_recipe checked out (catalogue main HEAD for a cold run), which can be untagged-ahead of the latest tag, while promote always writes the latest TAG. Per operator: a canonical must only ever be a real release. Add a tagged requirement: the tested head version (abra.head_compose_version, the compose version label) must equal a published release tag (recipe_tags). When main HEAD == latest release (the common just-cut case) head_version == latest tag → promote; when main is untagged-ahead → no promote.

Trigger on a NEW RELEASE TAG (§2.D) + test the tag (not main)

• Version ordering is centralized in warm_reconcile.version_key / latest_version / newest_older_version (already used by samever step-back). Reuse them.
• Trigger (pure, in the sweep, per recipe): after mirror-sync, latest = latest_version(recipe_tags); canon = read_registry(recipe).version. No tag → SKIP (never released). latest <= canon (by version_key) → SKIP no-new-version (even if main has untagged commits — we compare tags not commits). latest > canon → run cold on the tag.
• Test the TAG cold: to honour "run CI cold on that tagged version" (and so a green gate proves the exact thing that gets promoted), check out the latest tag in ~/.abra/recipes/<recipe> and run with CCCI_SKIP_FETCH=1 (the existing staging mechanism) → head_version = tag, head_ref = tag commit, REF empty (so not ref still holds → promote allowed). The upgrade-base resolver then sees canonical(older) < head(new tag) → real delta (samever step-back never fires: tag>canon by construction).

samever orthogonality (operator-required)

The release-tag trigger guarantees, in the sweep, version-under-test > canonical, so the upgrade base is strictly older → samever's same-version step-back never fires. (a) no new tag → SKIP, no upgrade-tier run; (b) new tag → canonical(older)→new, real delta, promote. samever's same-version behaviour stays owned by the samever phase on the PR path. Will demonstrate both in M2.

Enroll-all set (§2.B)

Authoritative inventory = cc-ci-plan/used-recipes.md (21 rows: 20 weekly + uptime-kuma external). NOT the test fixtures (custom-html-bkp-bad / -rst-bad, concurrency, regression, _generic). custom-html-tiny IS in used-recipes (weekly) → enroll it too.

Disk budget (§2.B watch-item)

Host /: 150G total, 104G used, 40G free (73%). du of /var/lib/ci-warm today: custom-html 32K, keycloak 159M. Retaining ~21 fresh-install data volumes should be a few GB; immich/matrix/mailu are the ones to watch. Will measure during the M2 full sweep and record the real budget; raise the VM disk (orchestrator) rather than silently drop recipes if it binds.

§2.G UPGRADE_BASE_VERSION retirement — gated on M2

plausible pins UPGRADE_BASE_VERSION="3.0.1+v2.0.0"; bluesky-pds only references it in a comment. Retirement requires plausible's canonical to actually land at its latest green release so the dynamic resolver picks the right base — so this is sequenced AFTER M2 promotes plausible. Keep the pin if plausible can't go green dynamically (record why).