From 4bef9a359b452e861d2c28e1cc01e7bcb208c2af Mon Sep 17 00:00:00 2001 From: autonomic-bot Date: Thu, 9 Jul 2026 00:19:39 +0000 Subject: [PATCH] =?UTF-8?q?review(m2-redfix):=20M2=20PASS=20=E2=80=94=20VE?= =?UTF-8?q?TO=20CLEARED,=20F-redfix-4=20CLOSED=20(remedy=20verified=20cold?= =?UTF-8?q?=20at=20b5f2b10)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cold-verified redfix-m2-harness@b5f2b10 (parent 07fc6d4 = the sha the earlier M2 PASS was given against, so the other five fixes are provably untouched by this commit). Clearing condition MET verbatim: live_slot=keycloak vs canonical_slot=canon-keycloak are disjoint; restore(canon) and restore(live) each return their OWN stack's volumes; reconciler last_good survives; foreign snapshot AND foreign restore both refused (probed both directions). canonical_domain unchanged, so M2's original keycloak evidence still stands. Checks the Builder did not run: - integrity: canon mariadb byte-identical across the destructive restore round-trip (cksum 4271926745 166164480, 386 files, before and after). - mutation testing (are the +10 tests vacuous?): reverting canonical_ns() to `return recipe` reds 4 of them; removing both _assert_slot_not_foreign() call sites reds 2. Not vacuous. Suites reproduce exactly: 325 passed @b5f2b10, 315 passed @07fc6d4. - caller audit: every snapshot/restore/app_dir/snap_dir call site passes an explicit slot; no bare recipe survives. - blast radius (the risk this refactor most plausibly created): all 21 enrolled recipes still resolve to their EXISTING on-disk dirs; registry_path("bluesky-pds") is character-identical at parent and fix. The 3 without a registry have no dir at all and never did. No migration. - prune_stale invariant now structural: / never gains a canonical.json, so a de-enrolled provider can no longer rmtree the reconciler's last_good (scratch-root sim, fake ns only — prune_stale calls `docker volume rm`). Enrollment retained (WARM_CANONICAL=True), no silent de-enrollment. Both false "can never touch each other" comments removed. B-redfix-5 (reconciler rollback restore() outside the upgrade try/except) accepted as NON-BLOCKING: verified verbatim present at parent 07fc6d4, so it predates the enrollment. F-redfix-4 made it reachable; that path is now closed. Correctly filed, not silently fixed. All six recipes now hold a fresh Adversary PASS; F-redfix-1/2/3/4 CLOSED; no open blocking finding. Builder may re-assert ## DONE. Node clean: throwaway volume + scratch removed, real warm root untouched (keycloak/ = last_good only, no canon-keycloak/ created), canon volumes intact, live keycloak /realms/master 200 throughout. --- machine-docs/BACKLOG-redfix.md | 31 +++++++++++++- machine-docs/REVIEW-redfix.md | 74 ++++++++++++++++++++++++++++++++++ 2 files changed, 104 insertions(+), 1 deletion(-) diff --git a/machine-docs/BACKLOG-redfix.md b/machine-docs/BACKLOG-redfix.md index 6915d0b..786d54d 100644 --- a/machine-docs/BACKLOG-redfix.md +++ b/machine-docs/BACKLOG-redfix.md @@ -74,7 +74,34 @@ hold). Concrete fix designs from M1 evidence: (Adversary-owned — do not edit.) -### [adversary] F-redfix-4 — keycloak enrollment is collision-free in DOMAIN but NOT in warm-state: the live-warm reconciler and the new data-warm canonical share one per-recipe snapshot slot — **OPEN, BLOCKING (VETO)** @2026-07-08T23:56Z +### [adversary] F-redfix-4 — keycloak enrollment is collision-free in DOMAIN but NOT in warm-state: the live-warm reconciler and the new data-warm canonical share one per-recipe snapshot slot — **CLOSED @2026-07-09T00:18Z (VETO CLEARED)** + +**CLOSED by Adversary re-test.** Fixed at `redfix-m2-harness`@`b5f2b10` (parent `07fc6d4`): `canonical_ns()` +is now the single namespace behind BOTH the canonical's domain and its warm-state slot, so a live-warm +provider gets slot `canon-/`, disjoint from the reconciler's `/`. Plus a +naming-independent `_assert_slot_not_foreign()` guard on snapshot AND restore. + +My cold re-test (full evidence in REVIEW-redfix.md @2026-07-09T00:18Z): the published clearing condition is +met verbatim — slots disjoint, `restore(canon)` and `restore(live)` each return their OWN stack's volumes, +reconciler `last_good` survives, foreign snapshot *and* foreign restore both refused. Beyond the Builder's +own checks I added four: (a) the canon mariadb volume is byte-identical across the destructive restore +round-trip (`4271926745 166164480`, 386 files); (b) **mutation testing** — reverting `canonical_ns()` reds 4 +of the new tests, removing the guard reds 2, so the 315→325 test delta is not vacuous; (c) every +`snapshot`/`restore`/`app_dir` caller now passes an explicit slot, no bare recipe survives; (d) all 21 +enrolled recipes still resolve to their existing on-disk dirs (`registry_path("bluesky-pds")` is +character-identical at parent and fix) — zero blast radius, no migration needed. + +Consequences 1–3 resolved. Consequence 4 (`prune_stale`) is now structural: `/` never gains a +`canonical.json`, verified in a scratch root. Enrollment retained (`WARM_CANONICAL = True`) — no silent +de-enrollment. The two false "can never touch each other" comments are gone. + +Residual **B-redfix-5** (reconciler rollback `restore()` outside the upgrade's `try/except`) is NOT part of +this finding's clearing condition and is **not** blocking: I confirmed it is verbatim present at parent +`07fc6d4`, so it predates the enrollment. F-redfix-4 made it *reachable*; that path is now closed. + +
Original report (as filed 2026-07-08T23:56Z) + +### [adversary] F-redfix-4 — original text — **OPEN, BLOCKING (VETO)** **Severity:** BLOCKS the phase's keycloak DoD item and must be fixed before the operator merges `redfix-m2-harness`. Worst case is an outage of the live shared OIDC provider that `lasuite-*`/`drone` @@ -173,6 +200,8 @@ de-enroll keycloak and re-open the DoD item, so it is not sufficient. survives a live-reconciler stateful upgrade (and vice versa) — demonstrated by re-running the repro above and seeing each `restore()` return its OWN stack's volumes. +
+ --- ### [adversary] F-redfix-1 — discourse migration INCOMPLETE: dangling image-less `sidekiq` in compose.smtpauth.yml (R011 lint regression + breaks SMTP-auth deploys) — **CLOSED @2026-06-18T07:06Z** diff --git a/machine-docs/REVIEW-redfix.md b/machine-docs/REVIEW-redfix.md index 877d742..2bcb3ab 100644 --- a/machine-docs/REVIEW-redfix.md +++ b/machine-docs/REVIEW-redfix.md @@ -575,3 +575,77 @@ JOURNAL-redfix.md for context. It contains **zero** mentions of `snap_dir` / `ap `seed_canonical` / warm-state. The keycloak entry (@2026-06-18T01:05Z) reasons only about the *domain* namespace. So F-redfix-4 is a genuine blind spot in the fix's design, not a hazard the Builder identified and knowingly deferred. This does not change the verdict. + +### M2 (F-redfix-4 remedy) — **PASS @2026-07-09T00:18Z**. VETO CLEARED. F-redfix-4 CLOSED. + +Cold-verified the Builder's re-claim of `redfix-m2-harness` @`b5f2b10` (parent `07fc6d4` — exactly the sha +the earlier M2 PASS was given against, so the other five fixes are provably untouched by this commit: +`git diff --stat` shows only `warmsnap.py`, `canonical.py`, `warm_reconcile.py`, `run_recipe_ci.py`, +`tests/keycloak/recipe_meta.py` + two unit-test files). Fresh clone on cc-ci, my own runs throughout. + +**1. The clearing condition I published — MET, verbatim.** Scratch `CCCI_WARM_ROOT`, real idle +`warm-canon-keycloak` stack, throwaway `warm-advlive…` volume as the live stand-in: + +``` +live_slot = keycloak -> /tmp/advw/keycloak/snapshot +canonical_slot = canon-keycloak -> /tmp/advw/canon-keycloak/snapshot SLOTS DISJOINT = True +registry_path = /tmp/advw/canon-keycloak/canonical.json (NOT the reconciler's dir) +canonical_domain UNCHANGED = warm-canon-keycloak.ci.commoninternet.net +restore(canon) -> ['warm-canon-keycloak_..._mariadb', 'warm-canon-keycloak_..._providers'] +restore(live) -> ['warm-advlive_ci_commoninternet_net_data'] +reconciler last_good SURVIVES: 10.7.1+26.6.2 +``` +Each `restore()` returns its OWN stack's volumes — the exact condition. Pre-fix this same script printed one +shared slot and `restore(canon)` raised `SnapshotError`. I additionally probed the guard in **both** +directions (the Builder's script only showed one): foreign *snapshot* REFUSED, foreign *restore* REFUSED. + +**2. Integrity of the round-trip (my addition).** `restore()` is destructive (clears volumes, untars back). +Canon mariadb tar-checksum **byte-identical** before and after: `4271926745 166164480`, 386 files both times. + +**3. Non-vacuity of the 10 new tests — mutation testing (my addition; the Builder asserted 315→325 but not +that the tests bite).** Suites reproduce exactly: `325 passed` @`b5f2b10`, `315 passed` @`07fc6d4`. Then: + * Mutation A — `canonical_ns()` reverted to `return recipe`: **4 failed** (`…_for_live_warm_provider`, + `test_live_and_canonical_slots_are_disjoint`, `test_registry_path_of_live_warm_provider_is_not_the_reconciler_dir`, + `test_prune_stale_keeps_enrolled_provider_canonical_and_reconciler`). + * Mutation B — both `_assert_slot_not_foreign()` call sites removed: **2 failed** + (`test_snapshot_refuses_to_clobber_another_domains_slot`, `test_restore_refuses_a_foreign_slot`). + Tests genuinely exercise the fix; not vacuous. + +**4. All callers migrated (my addition).** Audited every `snapshot(`/`restore(`/`app_dir(`/`snap_dir(` call +site: no bare recipe survives. `warm_reconcile` passes `warmsnap.live_slot(recipe)` (×3, incl. `last_good_path`), +`run_recipe_ci.py:896` passes `canonical.canonical_slot(recipe)`, `seed_canonical` passes `canonical_slot`. + +**5. Zero blast radius on the other canonicals (my addition — the risk this refactor most plausibly created).** +Ran `canonical_ns`/`registry_path` for all 21 enrolled recipes against the REAL warm root, read-only: every +non-provider keeps `ns == recipe` and resolves to its EXISTING `/var/lib/ci-warm//canonical.json` + +`snapshot/`. `registry_path("bluesky-pds")` is character-identical at parent and at the fix. The three with no +registry (bluesky-pds, discourse, mattermost-lts) have no dir on disk at all and never did — unseeded, not +regressed. **No migration needed**, as claimed: `/var/lib/ci-warm/keycloak/` = `last_good` only. + +**6. `prune_stale` invariant now structural (consequence 4).** Scratch root, fake ns only (never real stack +names — `prune_stale` calls `docker volume rm`): `keycloak/` has no `canonical.json` and structurally cannot +gain one, so it is skipped; `pruned: ['canon-fakerecipe']`, reconciler `last_good` survives. + +**7. Enrollment retained, no silent de-enrollment.** `WARM_CANONICAL = True`; `enrolled_recipes()` includes +keycloak at ns `canon-keycloak`. `canonical_domain` is unchanged, so M2's original keycloak evidence (promote +at `warm-canon-keycloak`, live 200 throughout) still stands without a re-run. The two comments asserting the +deployments "can never touch each other" are gone (`grep -c` → 0 in both files) and replaced with accurate text. + +**Consequences 1–3 of F-redfix-4 are resolved:** slots disjoint (1,2), and the outage race (3) is removed with +the shared slot — a promote can no longer replace the reconciler's known-good. + +**Residual B-redfix-5 — accepted as NON-BLOCKING, independently confirmed to PREDATE this work.** The +reconciler's rollback `warmsnap.restore()` sits outside the `try/except` guarding the upgrade, so any restore +failure leaves live keycloak undeployed after `abra.undeploy()`. I verified this is verbatim true at parent +`07fc6d4` (and thus predates the enrollment): F-redfix-4 supplied a *reachable* way to make that restore raise, +and that way is now closed; the structural gap is older and is not part of the clearing condition I published. +Correctly filed rather than silently fixed — the safe-behaviour choice (redeploy last_good without data restore +vs. die loudly) is a real trade-off for a DB-backed app after a forward migration. Not a VETO. + +Node left clean: throwaway volume removed, all scratch removed, real warm root untouched +(`/var/lib/ci-warm/keycloak/` = `last_good` only; no `canon-keycloak/` created), both canon volumes intact, +live `warm-keycloak…/realms/master` → **200** throughout. + +**## VETO CLEARED** (F-redfix-4). M2 keycloak item: **PASS**. All six recipes now have a fresh Adversary PASS; +F-redfix-1/2/3/4 all CLOSED; no open blocking finding. The Builder may re-assert `## DONE`. +JOURNAL not consulted before this verdict.