cc-ci/machine-docs/BACKLOG-canon.md

BACKLOG — phase canon

Build backlog (Builder-owned)

Milestone map → Definition of Done (§5). M1 = machinery + unit tests (Adversary cold-verifies the pieces). M2 = proven end-to-end in real CI.

M1 — machinery works locally, each piece proven

• M1.1 Tagged-promote gate (§2.A). Extend should_promote_canonical to ALSO require the tested head version corresponds to a published release tag. Add a tagged: bool param computed at the call site (head_version in recipe_tags(recipe)); keep the function pure. Untagged head → no promote. Unit tests: enrolled+green+cold+not-ref+tagged → True; each missing condition (incl. untagged) → False.
• M1.2 Release-tag trigger + mirror-sync in the sweep (§2.C/§2.D). New pure helper sweep_decision(recipe, latest_tag, canon_version) → run | skip:no-new-version | skip:never-released, keyed on version_key (NOT commit). Wire nightly_sweep.sweep() to, per enrolled recipe: (1) faithful mirror-sync main+tags to upstream (reuse open-recipe-pr.sh --reconcile-only, vendored into the repo for reproducibility); (2) compute latest release tag vs canonical; (3) skip or run cold ON THE TAG (checkout tag + CCCI_SKIP_FETCH=1). Unit tests for sweep_decision (new tag → run; equal → skip; older/no tag → skip).
• M1.3 Enroll all recipes (§2.B). Set WARM_CANONICAL = True in each of the 21 used-recipes tests/<r>/recipe_meta.py. Leave fixtures (custom-html-*-bad, concurrency, regression) alone.
• M1.4 Hollow-sweep fix (root cause). Make the deployed sweep read the REAL tests/ + run current code: set CCCI_REPO=/etc/cc-ci in the sweep service and run nightly_sweep.py from the checkout (not the store copy). Deploy procedure pulls /etc/cc-ci before nixos-rebuild.
• M1.5 Weekly timer (§2.F). nightly-sweep.nix OnCalendar daily → weekly (one line), Persistent=true (already set). Low-traffic slot.

M2 — proven end-to-end in real CI

• M2.1 Deploy the M1 changes: git -C /etc/cc-ci pull + nixos-rebuild switch; verify host health after.
• M2.2 Full sweep run across the enrolled set on cc-ci: mirrors synced, canonicals promoted for green recipes (records with correct version+commit), red recipes left intact, no-new-tag recipes skipped. Per-recipe results log captured.
• M2.3 Determinism proof: run the sweep a SECOND time immediately → every recipe SKIPS (latest tag == canonical for all) = clean no-op, no CI rerun.
• M2.4 Tagged-promote proof: a green run on an UNTAGGED state does NOT promote; a green run on a TAGGED release DOES. Construct if the live set doesn't cover it.
• M2.5 Real (non-hollow) timer fire: after a timer fire, canonicals have ADVANCED (evidence), not exit-0 on an empty set.
• M2.6 samever orthogonality: (a) no new tag (even with untagged commits on main) → SKIP, no upgrade-tier run, no promote; (b) new tag → cold-test new tag, canonical(older)→new, promote. Show step-back never fires inside the sweep.
• M2.7 Disk budget recorded; all recipes enrolled (or documented exception in DECISIONS).
• M2.8 §2.G UPGRADE_BASE_VERSION retirement — after plausible's canonical lands at 3.0.1: remove the pin, confirm dynamic base resolves 3.0.1 + passes; if it holds, strip the key (meta KEYS, resolver branch, docs, unit tests) + update bluesky-pds comment. Else KEEP with a recorded reason in DECISIONS.

Notes

• Order within M1: M1.1 → M1.2 (depend on version helpers) → M1.3/M1.4/M1.5 (config). Claim M1 only when all unit tests green + tree clean + pushed.

Adversary findings

• DEFECT-1 [adversary] (M2.2 results-label untrustworthy) — OPEN, awaiting re-verify. nightly_sweep.sweep() labelled PASS (promoted) off rc==0, but promote_canonical is non-fatal (swallows its exception), so a FAILED promote on a green cold run still showed PASS (promoted) though NO canonical was written. The per-recipe results log (DoD evidence "canonicals actually promoted for the greens") was therefore misleading. Repro (run-1 evidence captured): grep "WC5 promote failed" _sweep.log vs grep "PASS (promoted)" _sweep.log — failed promotes appeared in BOTH. Builder fix f94de22 derives the label from canonical.read_registry(r).version == latest (PASS / GREEN-BUT-PROMOTE-FAILED / FAIL). Close only after I re-run the sweep and confirm the label matches the on-disk registry for every recipe.
• DEFECT-2 [adversary] (M2.2 promote path failing broadly) — OPEN, awaiting re-verify. Run-1: 4 of 5 completed promotes FAILED across 4 modes though cold CI was green — ghost (abra app new FATA dirty tree), bluesky-pds (missing pds_plc_rotation_key), custom-html-tiny (404, no seeded index), drone (warm deploy timed out 600s). The bare abra app deploy in promote_canonical lacked the cold install's wiring. Net-new canonical run-1 = 1 (cryptpad). Builder fix f94de22: promote now does a faithful install (clean tree → provision deps → deploy_app w/ install_steps + overlay + ready-probes). Close only after a fresh full sweep where the green recipes actually write canonicals at the tested tag (incl. the 4 failure classes), AND determinism (M2.3) holds (run-twice → skip-all). Note the drone 600s timeout may be node-contention, not wiring — watch it.