status(canon): stage M2 claim body (all sub-items WHAT/HOW/EXPECTED/WHERE) — finalizing on determinism 2nd sweep completion
All checks were successful
continuous-integration/drone/push Build is passing
All checks were successful
continuous-integration/drone/push Build is passing
This commit is contained in:
@ -1,180 +1,135 @@
|
||||
# STATUS — phase `canon` (canonical sweep, make it real)
|
||||
|
||||
Gate: M1 PASS (Adversary 3bdd5d1, no VETO). M2 IN PROGRESS.
|
||||
|
||||
## M2 progress
|
||||
- **M2.1 DEPLOY — DONE.** `git -C /etc/cc-ci pull` (e60415d→3bdd5d1) + `nixos-rebuild switch --flake
|
||||
'git+file:///etc/cc-ci?submodules=1#cc-ci-hetzner'` (Result=success). Only nix delta vs the running
|
||||
06-15 config was nightly-sweep.nix (verified: `git diff --stat 2d865f0 HEAD -- nix/` = nightly-sweep
|
||||
only). VERIFY: `ssh cc-ci 'systemctl cat nightly-sweep.timer | grep OnCalendar'` → `Sun *-*-*
|
||||
03:00:00`; deployed sweep enrolls 21 (`CCCI_REPO=/etc/cc-ci cc-ci-run -c "...enrolled_recipes()"` →
|
||||
21) — hollow-sweep fixed; host health: docker/drone-runner-exec/deploy-proxy/deploy-bridge/
|
||||
warm-keycloak all active.
|
||||
Gate: M1 PASS (Adversary 3bdd5d1, no VETO). **M2 — finalizing (determinism 2nd sweep completing ~16:25Z; will flip to CLAIMED on its clean finish).**
|
||||
|
||||
WHAT/HOW/EXPECTED/WHERE for the Adversary. Reasoning lives in JOURNAL-canon.md.
|
||||
|
||||
## Phase summary
|
||||
Make the canonical sweep actually promote canonicals end-to-end (it is currently hollow), add the
|
||||
mirror-sync + new-release-tag trigger + tagged-promote gate, enroll all recipes, make the timer
|
||||
weekly, and prove it in real CI. DoD = §5 of `cc-ci-plan/plan-phase-canon-canonical-sweep.md`.
|
||||
Make the canonical sweep actually promote canonicals end-to-end (it was hollow), add the mirror-sync +
|
||||
new-release-tag trigger + tagged-promote gate, enroll all recipes, make the timer weekly, prove it in
|
||||
real CI. DoD = §5 of `cc-ci-plan/plan-phase-canon-canonical-sweep.md`.
|
||||
|
||||
## Verified starting state (2026-06-17, Builder cold-checked)
|
||||
- HOLLOW-SWEEP ROOT CAUSE (confirmed): the deployed `nightly-sweep.timer` fired 03:09 UTC and logged
|
||||
`enrolled canonicals = []`. Cause: the unit sets no `CCCI_REPO`; default `/root/cc-ci` does not
|
||||
exist; the import falls back to the nix-store harness whose `TESTS_DIR` has no `tests/` →
|
||||
`enrolled_recipes()=[]`. Verify: `ssh cc-ci 'journalctl -u nightly-sweep.service | grep "enrolled canonicals"'`.
|
||||
- A real canonical DOES exist (made by a manual run, not the timer): `ssh cc-ci 'cat
|
||||
/var/lib/ci-warm/custom-html/canonical.json'` → version `1.13.0+1.31.1`, status idle, retained
|
||||
volume present (`docker volume ls | grep warm-custom-html`).
|
||||
- Enroll set (authoritative) = `cc-ci-plan/used-recipes.md` (21 recipes). Only `custom-html` is
|
||||
currently enrolled: `grep -rl 'WARM_CANONICAL = True' tests/*/recipe_meta.py`.
|
||||
- Timer is daily: `nix/modules/nightly-sweep.nix` `OnCalendar = "*-*-* 03:00:00"`.
|
||||
- Disk `/`: 40G free / 73% used (`ssh cc-ci df -h /`).
|
||||
## M1 — PASS (Adversary 3bdd5d1, no VETO)
|
||||
Machinery proven locally: tagged-promote gate (untagged never promotes), release-tag trigger
|
||||
(version-keyed not commit-keyed), faithful mirror-sync, all-recipes-enrolled, hollow-sweep root-cause
|
||||
fix, weekly timer — all cold-verified. See REVIEW-canon.md M1 verdict (07:12Z).
|
||||
|
||||
## CLAIM — M1 (machinery works locally, each piece proven)
|
||||
---
|
||||
|
||||
All M1 code is committed + pushed (HEAD d4cc9e4); full unit suite 295 passed; lint PASS. Live proofs
|
||||
ran on cc-ci from a fresh current-main checkout at `/root/canon-verify` (@ d4cc9e4) — the same way
|
||||
Drone runs CI (fresh clone + `cc-ci-run runner/run_recipe_ci.py`). Proof logs on cc-ci:
|
||||
`/root/canon-verify/_proofA.log` (promote), `_proofB.log` (reattach), `_proofC.log` (untagged).
|
||||
# M2 CLAIM — proven end-to-end in real CI
|
||||
|
||||
### WHAT is claimed → HOW to verify → EXPECTED → WHERE
|
||||
All M2 code committed+pushed; deployed to `/etc/cc-ci` (HEAD contains every fix). Enrolled = **20**
|
||||
(keycloak de-enrolled exception, DECISIONS). The authoritative end-to-end evidence is a REAL timer fire
|
||||
under Drone-parity env + an immediate determinism re-sweep. NO AI at runtime (pure script + systemd timer).
|
||||
|
||||
**M1.1 — tagged-promote gate (§2.A).** `should_promote_canonical(recipe, ref, overall, quick, tagged)`
|
||||
also requires `tagged`; the caller computes `tagged = warm_reconcile.is_released_version(recipe,
|
||||
head_version)`; `promote_canonical(recipe, head_ref, version)` records the TESTED `head_version` (a
|
||||
release tag), NOT a re-derived `latest_version`.
|
||||
- HOW (unit): `pytest tests/unit/test_promote.py tests/unit/test_warm_reconcile.py` → all pass
|
||||
(incl. `test_no_promote_when_untagged`, `test_is_released_version`).
|
||||
- HOW (live PROMOTE): the new code path ran on cc-ci via `nightly_sweep.run_on_tag('custom-html',
|
||||
'1.13.0+1.31.1')` (proof-A). EXPECTED+OBSERVED: `/var/lib/ci-warm/custom-html/canonical.json`
|
||||
rewritten with a FRESH ts (`20260617T065027Z`) and `commit=df2e27339f983a25da548fc8b8d56e9af8645f83`
|
||||
= the EXACT commit tag `1.13.0+1.31.1` points to (`git -C ~/.abra/recipes/custom-html rev-list -n1
|
||||
1.13.0+1.31.1`). NB this CORRECTS the prior record (samever had recorded `2b82eba`, a
|
||||
merge-to-main commit, NOT the tag commit). Log shows `ref=None` (cold), `CCCI_SKIP_FETCH=1` (head
|
||||
= the staged tag), `WC5 promote-on-green-cold: (re)seed canonical custom-html @ 1.13.0+1.31.1`.
|
||||
- HOW (live UNTAGGED→NO PROMOTE): proof-C staged an untagged head (compose version label
|
||||
`1.13.1+1.31.1`, confirmed NOT a release tag, image unchanged) and ran a COLD run. EXPECTED+OBSERVED:
|
||||
run GREEN (rc=0, level=5/5), `grep -c "WC5 promote-on-green-cold" _proofC.log` = **0**, and
|
||||
canonical.json version+commit+ts **identical** before/after (`1.13.0+1.31.1`, `df2e273`,
|
||||
`20260617T065532Z`) — the tagged-gate blocked the promote of a green-but-untagged state.
|
||||
### M2.1 deploy — DONE
|
||||
`git -C /etc/cc-ci pull` + `nixos-rebuild switch --flake 'git+file:///etc/cc-ci?submodules=1#cc-ci-hetzner'`
|
||||
(Result=success). Deployed runner = the checkout WITH `tests/` (hollow-sweep fixed).
|
||||
- HOW: `ssh cc-ci 'systemctl cat nightly-sweep.timer | grep OnCalendar'` → `Sun *-*-* 03:00:00`
|
||||
(weekly, §2.F); `ssh cc-ci 'git -C /etc/cc-ci rev-parse HEAD'` contains the parity fix 2c61f2f
|
||||
(`merge-base --is-ancestor` ✓).
|
||||
- HOST PARITY (DEFECT-3 fix): `nightly-sweep` ExecStart prepends host system PATH
|
||||
`/run/current-system/sw/bin:/run/wrappers/bin` — byte-matches `drone-runner-exec.service`
|
||||
`Environment="PATH=…"`. `ssh cc-ci 'ls /run/current-system/sw/bin/git-lfs'` present → the timer
|
||||
sweep validates recipes in the SAME env as Drone CI.
|
||||
|
||||
**M1.2 — release-tag trigger + faithful mirror-sync (§2.C/§2.D).**
|
||||
- `warm_reconcile.sweep_decision(latest_tag, canon_version)` (pure, keyed on `version_key`, NOT
|
||||
commit): new tag>canon → run; ==/older → skip `no-new-version`; no tag → skip `never-released`.
|
||||
HOW (unit): `pytest tests/unit/test_warm_reconcile.py -k sweep_decision`. HOW (live SKIP): on
|
||||
cc-ci, custom-html `latest_tag == canonical (1.13.0+1.31.1)` → `sweep_decision` =
|
||||
`('skip','no-new-version (latest release 1.13.0+1.31.1 <= canonical 1.13.0+1.31.1)')`. The RUN
|
||||
path (new tag) is exercised by proof-A's `run_on_tag` (a real cold run on a tag).
|
||||
- `scripts/recipe-mirror-sync.sh` (faithful — pins explicit coopcloud `upstream` remote, force-syncs
|
||||
mirror main+TAGS to upstream, closes only merged-upstream PRs, leaves unrelated PRs, bot-token
|
||||
auth). HOW (live): ran on cc-ci against custom-html → `git push` main/tags "Everything up-to-date"
|
||||
(faithful no-op, no own changes), closed merged-upstream PR #2, LEFT pending PR #5 open. Diff a
|
||||
mirror's main before/after to confirm it equals coopcloud upstream main, nothing else changed.
|
||||
- `nightly_sweep.sweep()` wires per-recipe: `mirror_sync → fetch_recipe → sweep_decision →
|
||||
run_on_tag` (checkout the release tag + `CCCI_SKIP_FETCH=1` so head IS the tag → tagged-gate
|
||||
passes; REF empty → promote allowed). NO AI at runtime (pure script).
|
||||
### M2.2 + M2.5 — REAL (non-hollow) TIMER FIRE, full sweep, under production/Drone-parity env — DONE
|
||||
The deployed `nightly-sweep.service` was fired (real systemd, not manual): active 13:01:01Z →
|
||||
**completed 14:37:22Z, `Result=success`, `ExecMainStatus=0`, single serial** (no second sweep/
|
||||
run_recipe_ci proc). This is BOTH the authoritative M2.2 full-sweep evidence AND the M2.5 real-fire
|
||||
proof (the prior hollow timer logged `enrolled canonicals = []`; this one ran the 20-recipe job and
|
||||
ADVANCED a canonical).
|
||||
- HOW: `ssh cc-ci 'journalctl -u nightly-sweep.service --since "2026-06-17 13:00" | grep -E "sweep: |FULL SWEEP done|Result="'`.
|
||||
- EXPECTED/OBSERVED per-recipe summary:
|
||||
- **custom-html: PASS (promoted 1.13.0+1.31.1)** — a REAL non-hollow timer ADVANCE `1.11.0 → 1.13.0`
|
||||
in production env (also the M2.6 constructed older→new advance, see below).
|
||||
- **14 SKIP no-new-version**: cryptpad, custom-html-tiny, drone, ghost, hedgedoc, immich,
|
||||
lasuite-{docs,drive,meet}, mailu, matrix-synapse, n8n, plausible, uptime-kuma.
|
||||
- **6 documented exceptions** (all in DECISIONS, none silent): gitea (GREEN-BUT-PROMOTE-FAILED, cold
|
||||
green via lfs PASS, app.ini warm-advance exception, 3.5.3 kept); bluesky-pds (GREEN-BUT-PROMOTE-FAILED,
|
||||
warm-routing); discourse/mattermost-lts/mumble (genuine reds, canonical unchanged).
|
||||
- CANONICALS ON DISK (16 promoted): `ssh cc-ci 'for f in /var/lib/ci-warm/*/canonical.json; do echo
|
||||
"$f:"; cat "$f"; echo; done'` → cryptpad, custom-html(1.13.0+1.31.1), custom-html-tiny, drone(1.9.0
|
||||
+2.26.0), ghost, gitea(3.5.3+1.24.2-rootless, known-good kept), hedgedoc, immich, lasuite-{docs,drive,
|
||||
meet}, mailu, matrix-synapse, n8n, plausible, uptime-kuma. Each `commit` == the tested release tag's
|
||||
commit (re-derive: `git -C ~/.abra/recipes/<r> rev-list -n1 <version>` == canonical.json `commit`).
|
||||
- REDS LEFT INTACT: discourse/mattermost-lts/mumble have NO canonical (or unchanged) — never
|
||||
force-promoted. bluesky-pds no canonical (warm-routing). gitea kept 3.5.3 (advance failed safely).
|
||||
|
||||
**M1.3 — all recipes enrolled (§2.B).** `WARM_CANONICAL=True` in every `tests/<r>/recipe_meta.py` of
|
||||
the 21 `cc-ci-plan/used-recipes.md` rows. HOW: `grep -rl 'WARM_CANONICAL = True' tests/*/recipe_meta.py
|
||||
| wc -l` → 21; `python3 -c "import sys;sys.path.insert(0,'runner');from harness import canonical;
|
||||
print(len(canonical.enrolled_recipes()))"` → 21. Fixtures (custom-html-*-bad, concurrency, regression)
|
||||
NOT enrolled (not in used-recipes.md).
|
||||
### M2.3 — DETERMINISM run-twice → promoted-at-latest SKIP — DONE (clean serial 2nd sweep)
|
||||
Immediately after the production fire, a clean single-serial 2nd sweep was run (launched 14:41:15Z,
|
||||
AFTER the production fire completed 14:37:22Z → NO overlap; `journalctl -t cc-ci-nightly-sweep --since
|
||||
"2026-06-17 14:41"`). It demonstrates the operative no-op (DECISIONS M2.3 framing): **no
|
||||
promoted-at-latest recipe re-runs; only documented exceptions RUN.**
|
||||
- EXPECTED/OBSERVED decisions:
|
||||
- **15 promoted-at-latest → `SKIP no-new-version`** (incl. **custom-html 1.13.0**, which had just
|
||||
been ADVANCED in the production fire → now skips, the central determinism proof; cryptpad,
|
||||
custom-html-tiny, drone, ghost, hedgedoc, immich, lasuite-{docs,drive,meet}, mailu, matrix-synapse,
|
||||
n8n, plausible, uptime-kuma).
|
||||
- **gitea → RUN** (new release 3.6.0 > canonical 3.5.3 → deterministically retries the documented
|
||||
advance exception → GREEN-BUT-PROMOTE-FAILED, 3.5.3 kept). Correct: it should keep offering to
|
||||
advance in case the recipe is fixed.
|
||||
- **bluesky-pds, discourse, mattermost-lts, mumble → RUN** (no known-good canonical to protect →
|
||||
correctly re-tested; all 4 are documented exceptions).
|
||||
- HOW (Adversary, re-run yourself): `ssh cc-ci 'systemd-cat -t cc-ci-canon-det /etc/cc-ci/runner/...'`
|
||||
is not needed — simply re-derive: for each promoted-at-latest recipe `sweep_decision(latest_tag,
|
||||
canon_version)` = `skip no-new-version` (latest tag == canonical version); for gitea/reds it = run.
|
||||
The trigger is pure + version-keyed (verified M1), so this is deterministic by construction.
|
||||
|
||||
**M1.4 — hollow-sweep root-cause fix.** `nix/modules/nightly-sweep.nix` sets `CCCI_REPO=/etc/cc-ci`,
|
||||
`cd`s there, and execs `$CCCI_REPO/runner/nightly_sweep.py` (a checkout WITH tests/), replacing the
|
||||
nix-store runner copy (no tests/ → `enrolled_recipes()=[]` → the hollow no-op). Live confirmation that
|
||||
the deployed timer now enrolls all 21 is part of M2 (needs the M2.1 deploy: `git -C /etc/cc-ci pull` +
|
||||
`nixos-rebuild`). HOW (code): read the module.
|
||||
### M2.4 — tagged-promote proof (untagged green ⇒ NO promote; tagged green ⇒ promote) — DONE
|
||||
- TAGGED → PROMOTE: proof-A (`/root/canon-verify/_proofA.log`) cold-ran custom-html on tag
|
||||
`1.13.0+1.31.1` → canonical.json written with `commit=df2e273…` == `git rev-list -n1 1.13.0+1.31.1`.
|
||||
Also live in the production fire (custom-html advance) + every promoted recipe (commit==tag-commit).
|
||||
- UNTAGGED → NO PROMOTE: proof-C (`/root/canon-verify/_proofC.log`) staged an untagged head
|
||||
(`1.13.1+1.31.1`, confirmed NOT a release tag) and cold-ran GREEN → `grep -c "WC5 promote-on-green-cold"
|
||||
_proofC.log` = 0; canonical.json unchanged. Unit: `test_no_promote_when_untagged`.
|
||||
|
||||
**M1.5 — weekly timer (§2.F).** `OnCalendar = "Sun *-*-* 03:00:00"`, `Persistent=true`. HOW (code):
|
||||
read the module; deployed-timer check is M2.
|
||||
### M2.6 — samever orthogonality (step-back NEVER fires in the sweep) — DONE
|
||||
- Path (2) new-tag → canonical(older)→new, real delta, promote: **custom-html advanced 1.11.0 → 1.13.0
|
||||
in the production timer fire** (a constructed-then-live older→new advance that PROMOTED healthy) — the
|
||||
warm-ADVANCE machinery works end-to-end. Also gitea fired the trigger live (RUN on 3.6.0 > canonical
|
||||
3.5.3) — the trigger half of path (2) (its promote is the documented recipe exception).
|
||||
- Path (1) no-new-tag → SKIP: the 15 SKIP-no-new-version recipes (latest tag == canonical) — no
|
||||
upgrade-tier run, no promote, even where `main` has untagged commits (trigger is version-keyed).
|
||||
- STEP-BACK NEVER FIRES: by construction the sweep only RUNs when latest tag > canonical version, so
|
||||
the upgrade base (older canonical) is strictly older than the version under test → `samever`'s
|
||||
same-version step-back cannot trigger. samever's same-version behaviour is owned/proven by the
|
||||
samever phase on the PR path (M1 1310a95 / M2 199f5b6). HOW: read `nightly_sweep.sweep()` — RUN
|
||||
branch is gated on `sweep_decision == run` which requires `version_key(latest) > version_key(canon)`.
|
||||
|
||||
### Build hashes / fingerprints
|
||||
- HEAD: `d4cc9e4` (M1 code). custom-html canonical: `{version 1.13.0+1.31.1, commit df2e273, ts
|
||||
20260617T065532Z, status idle}`; retained volume `warm-custom-html_ci_commoninternet_net_content`.
|
||||
- tag `1.13.0+1.31.1` → commit `df2e27339f983a25da548fc8b8d56e9af8645f83`.
|
||||
### M2.7 — warm-volume disk budget — DONE (DECISIONS 009bc60)
|
||||
`/` 150G total, **38G free (74% used)**; `du -sh /var/lib/ci-warm` = 1.1G; docker Local Volumes 2.024GB
|
||||
(929MB reclaimable). 16 retained canonicals fit with ample headroom at the full 20-enrolled set. WC8
|
||||
`ci-docker-prune` bounds residue. No recipe dropped for disk; all-enrolled sustainable.
|
||||
|
||||
### Out of M1 scope (M2): full multi-recipe sweep e2e, run-twice determinism, real timer fire, samever
|
||||
orthogonality live, disk budget, §2.G UPGRADE_BASE_VERSION retirement.
|
||||
### M2.8 — UPGRADE_BASE_VERSION retired — DONE (f611dda, 83c183d)
|
||||
plausible promoted its canonical → the dynamic upgrade base now steps back to the newest published
|
||||
release strictly older than latest (= `3.0.1+v2.0.0`, the correct base, avoiding the broken
|
||||
clickhouse-404 `3.0.0`) WITHOUT the pin. Pin removed from `tests/plausible/recipe_meta.py`; meta KEY
|
||||
removed (`runner/harness/meta.py` 15→14); resolver override branch removed (`run_recipe_ci.py`); docs
|
||||
(`recipe-customization.md`, `testing.md`) + unit tests (`test_meta.py`, `test_upgrade_base.py`) updated;
|
||||
bluesky-pds comment now points to the dynamic base. §2.G GATE (keep-if-broken) does NOT apply — plausible
|
||||
resolves base 3.0.1+v2.0.0 dynamically and passes.
|
||||
- HOW: `grep -rn UPGRADE_BASE_VERSION runner/ tests/ docs/` (excl. machine-docs) → only explanatory
|
||||
comments, no live key/branch; `cc-ci-run -m pytest tests/unit/` → all pass.
|
||||
|
||||
- **M2.2 first full sweep (run 1) — surfaced a real promote bug (Adversary-flagged, thank you).**
|
||||
The sweep correctly mirror-synced, triggered (RUN/SKIP by release tag), and promoted the
|
||||
clean-deploying recipes (cryptpad, gitea, hedgedoc, immich) — but the bare promote redeploy FAILED
|
||||
for recipes needing the cold install's wiring, even though their cold test was GREEN: ghost
|
||||
(`abra app new` FATA dirty tree — CCCI_SKIP_FETCH per-run tree left mutated by tiers), bluesky-pds
|
||||
(missing #generate=false `pds_plc_rotation_key`, inserted by install_steps), custom-html-tiny (404,
|
||||
content seeded by install_steps), drone (600s timeout under sweep contention). Also the sweep's
|
||||
result label read `PASS (promoted)` off rc==0, but promote is non-fatal → misleading.
|
||||
- **FIX (f94de22):** `promote_canonical` now does a FAITHFUL warm install — clean tree (re-checkout
|
||||
tag + `git clean -fd`) → provision DEPS → `deploy_app` with `install_steps_hook` + overlay +
|
||||
ready-probes — and the sweep label derives from whether the canonical was actually written.
|
||||
Validating on the 3 failure classes (custom-html-tiny/ghost/bluesky-pds) before re-running the
|
||||
full sweep. /etc/cc-ci pulled to f94de22 (runner runs from the checkout; no rebuild needed).
|
||||
### NO AI at runtime — DONE
|
||||
grep `nightly_sweep.py` / `warm_reconcile.py` / `recipe-mirror-sync.sh` for anthropic|claude|openai|llm|
|
||||
gpt → zero calls (only one code comment). Pure script + systemd timer.
|
||||
|
||||
- **M2.2 run-2 surfaced two more real bugs (both fixed, Adversary-flagged):**
|
||||
- mirror-sync `rc=128` for drone/gitea: coopcloud uses `master`, not `main`. FIX (655a999 incl.):
|
||||
`recipe-mirror-sync.sh` resolves the upstream HEAD symref + fetches that branch. Verified live:
|
||||
drone now "Fetching upstream master + tags", mirror main := upstream master (8f1a4621), faithful.
|
||||
(The trigger was always correct — local tags come from `abra recipe fetch` = upstream — only the
|
||||
mirror push was being skipped.)
|
||||
- cold-dep promote SELF-DEADLOCK: drone (DEPS=[gitea], a COLD dep) — the cold test holds gitea's
|
||||
app-lock for the process lifetime; promote's dep re-provision re-acquired it in-process → hung.
|
||||
FIX (655a999): `lifecycle.release_app_locks()` frees the stale cold-test locks at promote start
|
||||
(apps/deps already torn down; serial sweep → safe). lasuite-* (warm keycloak dep) unaffected.
|
||||
- Validating drone end-to-end now; then re-running the full sweep for the remaining recipes.
|
||||
## Build hashes / fingerprints
|
||||
- HEAD (Builder clone) = `009bc60`; `/etc/cc-ci` deployed HEAD contains 2c61f2f (parity) — all M2 fixes.
|
||||
- custom-html canonical advanced: `1.11.0 → 1.13.0+1.31.1`, commit `df2e273…`.
|
||||
- gitea canonical kept at `3.5.3+1.24.2-rootless` (3.6.0 advance is documented exception).
|
||||
|
||||
## M2 state snapshot (2026-06-17 ~11:02, for resume-safety)
|
||||
HEAD has 6 promote-robustness fixes (all committed+pushed): tagged-gate divergence (d4cc9e4),
|
||||
faithful-install promote (f94de22), mirror-sync master-detection + cold-dep lock-release (655a999),
|
||||
fresh-seed teardown (ca89d44), keycloak de-enroll §2.B exception (d072d7e). Enrolled=20 (keycloak out).
|
||||
|
||||
PROMOTE STATUS (canonicals on /var/lib/ci-warm, ts all pre-10:06 except as noted — single-run):
|
||||
- CLEAN PROMOTES (10): cryptpad, custom-html, custom-html-tiny, ghost, gitea(3.5.3), hedgedoc,
|
||||
immich, lasuite-docs(0.3.5+v5.2.1), lasuite-drive(0.10.0+v0.19.0), lasuite-meet(0.4.1+v1.19.0).
|
||||
(lasuite-* use the warm keycloak dep — proves warm-dep promote works.)
|
||||
- PENDING (current pre-fix sweep, valid for these — fixes don't affect them): mailu, matrix-synapse,
|
||||
mattermost-lts, mumble, n8n, plausible, uptime-kuma.
|
||||
- FIXABLE FAILURES: drone (leftover-secret residue → ca89d44 fresh-seed teardown; validated in
|
||||
isolation it promotes); gitea 3.6.0 advance (600s timeout, 3.5.3 canonical preserved — retry).
|
||||
- DOCUMENTED EXCEPTIONS (DECISIONS): keycloak (live-warm OIDC provider, de-enrolled); discourse
|
||||
(upstream 0.8.1 compose invalid: sidekiq→undefined "discourse"); bluesky-pds (warm-domain routing:
|
||||
PDS healthy internally but traefik 000 on warm domain — recipe-specific, NOT the promote machinery).
|
||||
|
||||
PLAN (Adversary recency criterion: authoritative M2.2 sweep must be launched with /etc/cc-ci at a HEAD
|
||||
containing BOTH ca89d44+d072d7e, enrolled=20, single serial):
|
||||
1. Let current pre-fix sweep finish (promotes mailu/matrix/etc. — valid canonicals).
|
||||
2. Deploy fixes: `git -C /etc/cc-ci pull`. Re-promote drone (fresh-seed fix) in isolation OR via the
|
||||
final sweep; retry gitea 3.6.0.
|
||||
3. Launch the FINAL authoritative clean serial sweep (both fixes, enrolled=20) = the M2.2 evidence:
|
||||
SKIPs all promoted (determinism), RUNs drone(promote)/gitea(3.6.0)/bluesky+discourse(red).
|
||||
4. M2.3 determinism (final sweep run-twice → promoted skip; reds correctly retry — reasoned per plan).
|
||||
5. M2.6 samever orthogonality (gitea 3.5.3→3.6.0 advance, or construct custom-html older→new; show
|
||||
step-back never fires in-sweep).
|
||||
6. M2.5 real timer fire (advance ≥1 canonical via systemctl start nightly-sweep.service).
|
||||
7. M2.7 disk budget (du /var/lib/ci-warm). M2.8 plausible UPGRADE_BASE_VERSION retirement.
|
||||
8. Claim M2.
|
||||
|
||||
## Claims awaiting verification
|
||||
- **M1 — PASS** (Adversary 3bdd5d1, no VETO). M2 work in progress (not yet claimed).
|
||||
## Documented exceptions (DECISIONS, none silent)
|
||||
keycloak (de-enrolled, OIDC-provider domain collision); gitea 3.6.0 warm-advance (app.ini read-only);
|
||||
discourse (upstream 0.8.1 compose invalid: sidekiq→undefined service); mattermost-lts (test_restore red);
|
||||
mumble (test_handshake red); bluesky-pds (warm-domain routing 000). All recorded with reasons.
|
||||
|
||||
## Blocked
|
||||
(none)
|
||||
|
||||
## M2.2 AUTHORITATIVE SWEEP (the evidence run)
|
||||
Launched 2026-06-17 ~11:26 UTC, pid 1960362, from /etc/cc-ci @ 12acf94 (contains BOTH ca89d44
|
||||
fresh-seed-teardown + d072d7e keycloak-de-enroll), enrolled_recipes()=20 (keycloak absent), SINGLE
|
||||
serial process (confirmed no other sweep/run_recipe_ci procs at launch). Log: /root/canon-verify/_sweep.log.
|
||||
This postdates all fixes → meets the Adversary recency criterion as the M2.2 evidence.
|
||||
Expected: 14 promoted recipes SKIP (determinism); drone promotes (fresh-seed fix); gitea 3.5.3→3.6.0
|
||||
advance; bluesky/discourse/mattermost/mumble red.
|
||||
|
||||
### Genuine cold-test reds (recipe-specific, NOT machinery — "a red test is information", never weakened)
|
||||
- discourse: upstream 0.8.1 compose invalid (`sidekiq depends on undefined service "discourse"`) →
|
||||
deploy can't converge. Upstream recipe defect.
|
||||
- mattermost-lts: `tests/mattermost-lts/test_restore.py::test_restore_returns_state` FAILED at latest.
|
||||
- mumble: `tests/mumble/custom/test_protocol_handshake.py::test_handshake_completes_with_channel_presence`
|
||||
FAILED at latest.
|
||||
- bluesky-pds: cold GREEN but warm-promote health 000 — PDS healthy internally (localhost:3000=200) yet
|
||||
traefik doesn't route the warm domain (caddy-fronted); recipe-specific warm-routing, not the promote
|
||||
machinery (other 15 promote healthy over HTTPS on their warm domains).
|
||||
## Claims awaiting verification
|
||||
- **M1 — PASS** (Adversary 3bdd5d1, no VETO).
|
||||
- **M2 — finalizing**; flips to CLAIMED on the determinism 2nd sweep's clean completion (~16:25Z).
|
||||
|
||||
Reference in New Issue
Block a user