claim(M1): canonical sweep machinery built + live-proven on custom-html
All checks were successful
continuous-integration/drone/push Build is passing
All checks were successful
continuous-integration/drone/push Build is passing
M1 (machinery works locally, each piece proven) — code HEAD d4cc9e4, unit suite 295 passed:
- M1.1 tagged-promote gate + promote-tested-version: live proof-A wrote a fresh canonical
(commit df2e273 = the tag commit, correcting samever's main-HEAD 2b82eba); live proof-C
green-untagged → 0 promotes, canonical byte-identical (tagged-gate blocks untagged).
- M1.2 sweep_decision (version-keyed trigger) + vendored faithful recipe-mirror-sync.sh
(smoke-tested: faithful no-op main/tags push, closed merged-upstream PR #2, left PR #5);
nightly_sweep rewritten (mirror_sync -> trigger -> run_on_tag). Live SKIP demo on custom-html.
- M1.3 all 21 used-recipes enrolled. M1.4 hollow-sweep fix (CCCI_REPO=/etc/cc-ci). M1.5 weekly timer.
- M1(A) reattach: live proof-B --quick reused the retained volume green; known-good unchanged.
Evidence + verify recipes in STATUS-canon.md; reasoning in JOURNAL-canon.md; DECISIONS appended.
Gate: M1 CLAIMED, awaiting Adversary.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@ -7,24 +7,24 @@ pieces). M2 = proven end-to-end in real CI.
|
||||
|
||||
### M1 — machinery works locally, each piece proven
|
||||
|
||||
- [ ] **M1.1 Tagged-promote gate (§2.A).** Extend `should_promote_canonical` to ALSO require the
|
||||
- [x] **M1.1 Tagged-promote gate (§2.A).** Extend `should_promote_canonical` to ALSO require the
|
||||
tested head version corresponds to a published release tag. Add a `tagged: bool` param computed
|
||||
at the call site (`head_version in recipe_tags(recipe)`); keep the function pure. Untagged head
|
||||
→ no promote. Unit tests: enrolled+green+cold+not-ref+tagged → True; each missing condition
|
||||
(incl. untagged) → False.
|
||||
- [ ] **M1.2 Release-tag trigger + mirror-sync in the sweep (§2.C/§2.D).** New pure helper
|
||||
- [x] **M1.2 Release-tag trigger + mirror-sync in the sweep (§2.C/§2.D).** New pure helper
|
||||
`sweep_decision(recipe, latest_tag, canon_version)` → `run` | `skip:no-new-version` |
|
||||
`skip:never-released`, keyed on `version_key` (NOT commit). Wire `nightly_sweep.sweep()` to, per
|
||||
enrolled recipe: (1) faithful mirror-sync main+tags to upstream (reuse open-recipe-pr.sh
|
||||
`--reconcile-only`, vendored into the repo for reproducibility); (2) compute latest release tag
|
||||
vs canonical; (3) skip or run cold ON THE TAG (checkout tag + `CCCI_SKIP_FETCH=1`). Unit tests
|
||||
for `sweep_decision` (new tag → run; equal → skip; older/no tag → skip).
|
||||
- [ ] **M1.3 Enroll all recipes (§2.B).** Set `WARM_CANONICAL = True` in each of the 21 used-recipes
|
||||
- [x] **M1.3 Enroll all recipes (§2.B).** Set `WARM_CANONICAL = True` in each of the 21 used-recipes
|
||||
`tests/<r>/recipe_meta.py`. Leave fixtures (custom-html-*-bad, concurrency, regression) alone.
|
||||
- [ ] **M1.4 Hollow-sweep fix (root cause).** Make the deployed sweep read the REAL tests/ + run
|
||||
- [x] **M1.4 Hollow-sweep fix (root cause).** Make the deployed sweep read the REAL tests/ + run
|
||||
current code: set `CCCI_REPO=/etc/cc-ci` in the sweep service and run `nightly_sweep.py` from
|
||||
the checkout (not the store copy). Deploy procedure pulls `/etc/cc-ci` before nixos-rebuild.
|
||||
- [ ] **M1.5 Weekly timer (§2.F).** `nightly-sweep.nix` `OnCalendar` daily → weekly (one line),
|
||||
- [x] **M1.5 Weekly timer (§2.F).** `nightly-sweep.nix` `OnCalendar` daily → weekly (one line),
|
||||
`Persistent=true` (already set). Low-traffic slot.
|
||||
|
||||
### M2 — proven end-to-end in real CI
|
||||
|
||||
@ -1462,3 +1462,31 @@ but stays well within the ≤90 s budget. Acceptable.
|
||||
remove it"). The all-deploys `compose.ccci.yml` overlay is now ENVIRONMENTAL-only (node-reality tweaks,
|
||||
no version-specific image pins or service add/drop); version-specific repairs live in `previous/`.
|
||||
Discourse ships NO `previous/` (base bitnamilegacy:3.5.0 deploys clean).
|
||||
|
||||
## Phase canon (2026-06-17) — canonical sweep made real
|
||||
|
||||
- **Tagged-promote gate (§2.A).** A canonical only ever advances to a PUBLISHED RELEASE TAG.
|
||||
`should_promote_canonical` requires `tagged` (computed by the caller via
|
||||
`warm_reconcile.is_released_version(recipe, head_version)`); `promote_canonical` records the TESTED
|
||||
`head_version` (the release version actually exercised), NOT a re-derived `latest_version(recipe_tags)`
|
||||
— these can diverge in a manual `RECIPE=<r>` run whose `main` sits on a tag older than the newest
|
||||
published tag. An untagged `main` commit never becomes a canonical.
|
||||
- **New-release-tag trigger (§2.D).** The weekly sweep tests a recipe only when its latest release tag
|
||||
is newer (by `warm_reconcile.version_key`) than its canonical version — NOT on new commits. No new
|
||||
tag → SKIP (even if `main` has untagged commits). This gives the run-twice determinism no-op and
|
||||
makes the sweep orthogonal to `samever` (version-under-test always > canonical → no same-version
|
||||
step-back in the sweep).
|
||||
- **Mirror-sync is a VENDORED `scripts/recipe-mirror-sync.sh`, not the nix-store
|
||||
recipe-upgrade/open-recipe-pr.sh.** Rationale: open-recipe-pr.sh assumes the recipe clone's `origin`
|
||||
IS coopcloud upstream, but cc-ci's abra recipe clones have inconsistent remotes (origin is variously
|
||||
the mirror / coopcloud / absent). The vendored script pins an explicit coopcloud `upstream` remote
|
||||
by recipe name, syncs main+TAGS (canon's trigger needs upstream tags), closes only merged-upstream
|
||||
PRs, leaves unrelated PRs, and authes via the bot gitea token (self-contained, reproducible — a
|
||||
systemd service must not depend on a per-skill-version nix-store path). Behaviour matches the phase's
|
||||
described `--reconcile-only`: faithful mirror sync, never our own changes.
|
||||
- **Hollow-sweep root cause + fix.** The deployed timer ran the nix-STORE runner copy (no `tests/`),
|
||||
so `enrolled_recipes()` resolved `TESTS_DIR` to a missing dir → `[]` → no-op. Fix: the sweep runs
|
||||
from `$CCCI_REPO=/etc/cc-ci` (has runner/ AND tests/); deploys `git -C /etc/cc-ci pull` +
|
||||
nixos-rebuild. Sweep-logic now ships via a checkout pull (no store rebuild needed for logic-only).
|
||||
- **All 21 used-recipes enrolled (§2.B); cadence weekly (§2.F).** The enroll set is exactly
|
||||
`cc-ci-plan/used-recipes.md`; test fixtures stay unenrolled.
|
||||
|
||||
@ -92,3 +92,41 @@ disk (orchestrator) rather than silently drop recipes if it binds.
|
||||
Retirement requires plausible's canonical to actually land at its latest green release so the dynamic
|
||||
resolver picks the right base — so this is sequenced AFTER M2 promotes plausible. Keep the pin if
|
||||
plausible can't go green dynamically (record why).
|
||||
|
||||
## 2026-06-17 — M1 built + live-proven (CLAIMED)
|
||||
|
||||
All M1 code landed (HEAD d4cc9e4). Reasoning behind the choices:
|
||||
|
||||
- **Tagged-gate computes `tagged` at the call site, not inside the gate** — keeps
|
||||
`should_promote_canonical` pure (the Adversary anti-anchoring + the existing unit-test contract).
|
||||
`is_released_version` lives in warm_reconcile (owns version logic + recipe_tags I/O).
|
||||
- **Promote the TESTED version (divergence fix, d4cc9e4):** the Adversary's pre-claim probe flagged
|
||||
that the gate checks `head_version` but promote recorded `latest_version(recipe_tags)`. Live proof-A
|
||||
made this concrete and favourable: the OLD record had commit `2b82eba` (a merge-to-main commit),
|
||||
but the tag `1.13.0+1.31.1` actually points to `df2e273`. Recording the tested version's head_ref
|
||||
now writes the TAG commit — strictly more correct. Sweep path was already safe (head==tag), but the
|
||||
manual `RECIPE=<r>` path needed it.
|
||||
- **Why a vendored mirror-sync script, not the nix-store open-recipe-pr.sh:** the recipe clones on
|
||||
cc-ci have INCONSISTENT remotes (n8n: origin=mirror; mumble: origin=coopcloud; ghost/discourse:
|
||||
origin=mirror, no `upstream`). open-recipe-pr.sh assumes origin=coopcloud → would force-sync mirror
|
||||
main to *mirror* main (no-op) for most. The vendored `scripts/recipe-mirror-sync.sh` pins an
|
||||
explicit coopcloud `upstream` remote from the recipe name, syncs main+TAGS (canon needs upstream
|
||||
tags for the trigger), and authes via the bot token (self-contained, not host .git-credentials).
|
||||
Behaviour matches the phase's described open-recipe-pr.sh --reconcile-only (faithful, close
|
||||
merged-upstream PRs, leave unrelated). See DECISIONS.
|
||||
- **Why test the TAG via checkout+CCCI_SKIP_FETCH (run_on_tag), not just REF=tag:** REF alone (no SRC)
|
||||
takes fetch_recipe's `abra recipe fetch` branch (ignores REF) AND would set `ref` → should_promote
|
||||
blocks. Staging the tag in the clone + CCCI_SKIP_FETCH makes head=tag with REF empty → promote
|
||||
allowed, and exercises the real "cold on the tagged release" path.
|
||||
|
||||
### Live proof evidence (cc-ci, /root/canon-verify @ d4cc9e4)
|
||||
- proof-A (promote): canonical.json fresh ts 065027Z, commit df2e273 (=tag commit). Note: because
|
||||
custom-html canonical already == latest, run_on_tag here re-promoted an EQUAL version → the samever
|
||||
step-back fired (base 1.11.0+1.29.0). That is an artifact of bypassing the trigger for the proof;
|
||||
the REAL sweep SKIPs equal-version (sweep_decision), so the step-back never fires in the sweep — to
|
||||
be shown live in M2 (canonical(older)→new tag, base=canonical, no step-back).
|
||||
- proof-B (reattach): --quick reattached the retained volume, green (4 tests passed), known-good
|
||||
version+commit UNCHANGED (df2e273); ts re-stamped only by the idle-status write (write_registry
|
||||
stamps ts on every status write) — NOT a promote.
|
||||
- proof-C (untagged→no-promote): green cold run (level 5/5) on an untagged head (label 1.13.1+1.31.1)
|
||||
→ 0 promote log lines, canonical.json byte-identical before/after. Tagged-gate works live.
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
# STATUS — phase `canon` (canonical sweep, make it real)
|
||||
|
||||
Gate: M1 IN PROGRESS (not yet claimed).
|
||||
Gate: M1 CLAIMED, awaiting Adversary.
|
||||
|
||||
WHAT/HOW/EXPECTED/WHERE for the Adversary. Reasoning lives in JOURNAL-canon.md.
|
||||
|
||||
@ -22,23 +22,75 @@ weekly, and prove it in real CI. DoD = §5 of `cc-ci-plan/plan-phase-canon-canon
|
||||
- Timer is daily: `nix/modules/nightly-sweep.nix` `OnCalendar = "*-*-* 03:00:00"`.
|
||||
- Disk `/`: 40G free / 73% used (`ssh cc-ci df -h /`).
|
||||
|
||||
## M1 progress (code COMPLETE; live proofs in flight — not yet claimed)
|
||||
All M1 code landed + unit-tested + lint-clean (full unit suite 295 passed):
|
||||
- M1.1 tagged-promote gate (27e0628) + divergence fix (d4cc9e4): `should_promote_canonical` gains
|
||||
`tagged`; promote records the TESTED `head_version` (a release tag), not a re-derived latest.
|
||||
`warm_reconcile.is_released_version`. Tests: `tests/unit/test_promote.py`, `test_warm_reconcile.py`.
|
||||
- M1.2 release-tag trigger + mirror-sync (a20890a): `warm_reconcile.sweep_decision` (pure, version_key
|
||||
keyed), `scripts/recipe-mirror-sync.sh` (faithful main+tags sync, closes merged-upstream PRs),
|
||||
`nightly_sweep` rewritten (mirror_sync → trigger → run_on_tag = checkout tag + CCCI_SKIP_FETCH).
|
||||
- M1.3 all 21 used-recipes enrolled (136100f). M1.4 hollow-sweep fix + M1.5 weekly timer (f8c0e53).
|
||||
Mirror-sync smoke-tested live on custom-html (faithful no-op push; closed merged-upstream PR #2; left
|
||||
pending PR #5).
|
||||
## CLAIM — M1 (machinery works locally, each piece proven)
|
||||
|
||||
Live M1(A) proofs in progress on cc-ci from checkout /root/canon-verify @ d4cc9e4:
|
||||
- A-promote: `nightly_sweep.run_on_tag('custom-html','1.13.0+1.31.1')` → expect fresh canonical.json.
|
||||
All M1 code is committed + pushed (HEAD d4cc9e4); full unit suite 295 passed; lint PASS. Live proofs
|
||||
ran on cc-ci from a fresh current-main checkout at `/root/canon-verify` (@ d4cc9e4) — the same way
|
||||
Drone runs CI (fresh clone + `cc-ci-run runner/run_recipe_ci.py`). Proof logs on cc-ci:
|
||||
`/root/canon-verify/_proofA.log` (promote), `_proofB.log` (reattach), `_proofC.log` (untagged).
|
||||
|
||||
### WHAT is claimed → HOW to verify → EXPECTED → WHERE
|
||||
|
||||
**M1.1 — tagged-promote gate (§2.A).** `should_promote_canonical(recipe, ref, overall, quick, tagged)`
|
||||
also requires `tagged`; the caller computes `tagged = warm_reconcile.is_released_version(recipe,
|
||||
head_version)`; `promote_canonical(recipe, head_ref, version)` records the TESTED `head_version` (a
|
||||
release tag), NOT a re-derived `latest_version`.
|
||||
- HOW (unit): `pytest tests/unit/test_promote.py tests/unit/test_warm_reconcile.py` → all pass
|
||||
(incl. `test_no_promote_when_untagged`, `test_is_released_version`).
|
||||
- HOW (live PROMOTE): the new code path ran on cc-ci via `nightly_sweep.run_on_tag('custom-html',
|
||||
'1.13.0+1.31.1')` (proof-A). EXPECTED+OBSERVED: `/var/lib/ci-warm/custom-html/canonical.json`
|
||||
rewritten with a FRESH ts (`20260617T065027Z`) and `commit=df2e27339f983a25da548fc8b8d56e9af8645f83`
|
||||
= the EXACT commit tag `1.13.0+1.31.1` points to (`git -C ~/.abra/recipes/custom-html rev-list -n1
|
||||
1.13.0+1.31.1`). NB this CORRECTS the prior record (samever had recorded `2b82eba`, a
|
||||
merge-to-main commit, NOT the tag commit). Log shows `ref=None` (cold), `CCCI_SKIP_FETCH=1` (head
|
||||
= the staged tag), `WC5 promote-on-green-cold: (re)seed canonical custom-html @ 1.13.0+1.31.1`.
|
||||
- HOW (live UNTAGGED→NO PROMOTE): proof-C staged an untagged head (compose version label
|
||||
`1.13.1+1.31.1`, confirmed NOT a release tag, image unchanged) and ran a COLD run. EXPECTED+OBSERVED:
|
||||
run GREEN (rc=0, level=5/5), `grep -c "WC5 promote-on-green-cold" _proofC.log` = **0**, and
|
||||
canonical.json version+commit+ts **identical** before/after (`1.13.0+1.31.1`, `df2e273`,
|
||||
`20260617T065532Z`) — the tagged-gate blocked the promote of a green-but-untagged state.
|
||||
|
||||
**M1.2 — release-tag trigger + faithful mirror-sync (§2.C/§2.D).**
|
||||
- `warm_reconcile.sweep_decision(latest_tag, canon_version)` (pure, keyed on `version_key`, NOT
|
||||
commit): new tag>canon → run; ==/older → skip `no-new-version`; no tag → skip `never-released`.
|
||||
HOW (unit): `pytest tests/unit/test_warm_reconcile.py -k sweep_decision`. HOW (live SKIP): on
|
||||
cc-ci, custom-html `latest_tag == canonical (1.13.0+1.31.1)` → `sweep_decision` =
|
||||
`('skip','no-new-version (latest release 1.13.0+1.31.1 <= canonical 1.13.0+1.31.1)')`. The RUN
|
||||
path (new tag) is exercised by proof-A's `run_on_tag` (a real cold run on a tag).
|
||||
- `scripts/recipe-mirror-sync.sh` (faithful — pins explicit coopcloud `upstream` remote, force-syncs
|
||||
mirror main+TAGS to upstream, closes only merged-upstream PRs, leaves unrelated PRs, bot-token
|
||||
auth). HOW (live): ran on cc-ci against custom-html → `git push` main/tags "Everything up-to-date"
|
||||
(faithful no-op, no own changes), closed merged-upstream PR #2, LEFT pending PR #5 open. Diff a
|
||||
mirror's main before/after to confirm it equals coopcloud upstream main, nothing else changed.
|
||||
- `nightly_sweep.sweep()` wires per-recipe: `mirror_sync → fetch_recipe → sweep_decision →
|
||||
run_on_tag` (checkout the release tag + `CCCI_SKIP_FETCH=1` so head IS the tag → tagged-gate
|
||||
passes; REF empty → promote allowed). NO AI at runtime (pure script).
|
||||
|
||||
**M1.3 — all recipes enrolled (§2.B).** `WARM_CANONICAL=True` in every `tests/<r>/recipe_meta.py` of
|
||||
the 21 `cc-ci-plan/used-recipes.md` rows. HOW: `grep -rl 'WARM_CANONICAL = True' tests/*/recipe_meta.py
|
||||
| wc -l` → 21; `python3 -c "import sys;sys.path.insert(0,'runner');from harness import canonical;
|
||||
print(len(canonical.enrolled_recipes()))"` → 21. Fixtures (custom-html-*-bad, concurrency, regression)
|
||||
NOT enrolled (not in used-recipes.md).
|
||||
|
||||
**M1.4 — hollow-sweep root-cause fix.** `nix/modules/nightly-sweep.nix` sets `CCCI_REPO=/etc/cc-ci`,
|
||||
`cd`s there, and execs `$CCCI_REPO/runner/nightly_sweep.py` (a checkout WITH tests/), replacing the
|
||||
nix-store runner copy (no tests/ → `enrolled_recipes()=[]` → the hollow no-op). Live confirmation that
|
||||
the deployed timer now enrolls all 21 is part of M2 (needs the M2.1 deploy: `git -C /etc/cc-ci pull` +
|
||||
`nixos-rebuild`). HOW (code): read the module.
|
||||
|
||||
**M1.5 — weekly timer (§2.F).** `OnCalendar = "Sun *-*-* 03:00:00"`, `Persistent=true`. HOW (code):
|
||||
read the module; deployed-timer check is M2.
|
||||
|
||||
### Build hashes / fingerprints
|
||||
- HEAD: `d4cc9e4` (M1 code). custom-html canonical: `{version 1.13.0+1.31.1, commit df2e273, ts
|
||||
20260617T065532Z, status idle}`; retained volume `warm-custom-html_ci_commoninternet_net_content`.
|
||||
- tag `1.13.0+1.31.1` → commit `df2e27339f983a25da548fc8b8d56e9af8645f83`.
|
||||
|
||||
### Out of M1 scope (M2): full multi-recipe sweep e2e, run-twice determinism, real timer fire, samever
|
||||
orthogonality live, disk budget, §2.G UPGRADE_BASE_VERSION retirement.
|
||||
|
||||
## Claims awaiting verification
|
||||
(M1 claim pending live-proof completion)
|
||||
- **M1 — CLAIMED** (this commit). Awaiting Adversary PASS in REVIEW-canon.md.
|
||||
|
||||
## Blocked
|
||||
(none)
|
||||
|
||||
Reference in New Issue
Block a user