claim(m2-redfix): M2 RE-CLAIMED — F-redfix-4 remedied at redfix-m2-harness@b5f2b10 (warm state keyed by stack namespace); clearing condition re-run green on cc-ci, unit suite 315->325; awaiting Adversary

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FS8p1esg57UAC69riNvuBX

machine-docs/STATUS-redfix.md

@@ -7,20 +7,101 @@ gitea, keycloak) → isolate → root-cause → classify (flake vs genuine; reci
 warm-machinery vs load) → FIX each (recipe PR or harness improvement) → verify green. No standing
 exceptions. Nothing merged.
 
-## NOT DONE — claim withdrawn 2026-07-09 (standing VETO: F-redfix-4, keycloak)
+## Gate: M2 RE-CLAIMED (F-redfix-4 remedy), awaiting Adversary — 2026-07-09
 
-The Adversary raised **F-redfix-4** (REVIEW-redfix.md, re-confirmation #8 @2026-07-08T23:56Z): the keycloak
-enrollment is collision-free at the *domain* layer but NOT in *warm state* — `warmsnap.app_dir()` keys the
-slot by bare recipe, so the live-warm reconciler (`warm-keycloak…`, `last_good`) and the newly-enrolled
-data-warm canonical (`warm-canon-keycloak…`, `canonical.json` + `snapshot/`) share one slot
-`/var/lib/ci-warm/keycloak/`. I reproduced it independently (see below). The phase is **NOT done**; the
-keycloak DoD item is reopened. Do **not** merge `redfix-m2-harness` until this clears.
+Prior DONE was withdrawn under the standing VETO F-redfix-4 (REVIEW-redfix.md re-confirmation #8
+@2026-07-08T23:56Z). The remedy is in. Re-claiming M2 for the keycloak item only.
 
-Scope: keycloak only. The M1 classifications and the discourse / mattermost-lts / gitea / bluesky-pds /
-mumble fixes are unaffected and remain Adversary-verified (M1 PASS @2026-06-18T01:18Z, and the five
-non-keycloak M2 fixes content-verified through re-confirmation #7).
+**WHAT.** F-redfix-4 is fixed at `redfix-m2-harness` @ **`b5f2b10`** (parent `07fc6d4`, the sha the M2 PASS
+was given against). Warm state is now keyed by a **stack namespace**, not a bare recipe:
+`canonical.canonical_ns(r)` is the single source from which BOTH the canonical's domain and its warm-state
+slot derive. A live-warm provider (`r in warm.WARM_DOMAINS`) gets ns `canon-<r>` → domain
+`warm-canon-keycloak.ci.commoninternet.net` (**unchanged**) + slot `/var/lib/ci-warm/canon-keycloak/`.
+Every other recipe keeps ns `<r>` → on-disk layout of the 15 existing canonicals unchanged.
 
-Gate status: **M2 REOPENED** — will be re-claimed once the F-redfix-4 remedy is in and verified.
+Addresses all four consequences: (1)+(2) slots disjoint, neither deployment can replace the other's
+known-good; (3) the outage race is removed with the shared slot; (4) `prune_stale`'s reconciler-dir
+invariant is now structural — `<recipe>/` never gains a `canonical.json`.
+
+Also: `warmsnap._assert_slot_not_foreign()` refuses to snapshot/restore a slot recorded against a different
+domain (defence in depth; fails before the destructive swap, not at the next restore). The two comments
+asserting the deployments "can never touch each other" are corrected (`canonical.py`,
+`tests/keycloak/recipe_meta.py`). `WARM_CANONICAL = True` is retained — keycloak stays enrolled, no
+skip-guard, no silent de-enrollment.
+
+**MIGRATION: none required.** On cc-ci `/var/lib/ci-warm/keycloak/` contains only `last_good` (the
+canonical was never seeded), so no existing file changes slot. Verify: `ls -A /var/lib/ci-warm/keycloak/`
+→ `last_good` only.
+
+**WHERE.** cc-ci repo, branch `redfix-m2-harness` @ `b5f2b10`. Files: `runner/harness/warmsnap.py`,
+`runner/harness/canonical.py`, `runner/warm_reconcile.py`, `runner/run_recipe_ci.py`,
+`tests/keycloak/recipe_meta.py`, `tests/unit/test_warmsnap.py`, `tests/unit/test_canonical.py`.
+
+**HOW to verify (1) — unit suite, cold clone, no docker needed.**
+
+    git clone -q --branch redfix-m2-harness <cc-ci remote> /tmp/v && cd /tmp/v
+    /nix/store/x188l04r3gfkh18gy1dpf05fv3kkrgs7-python3-3.12.8-env/bin/python3 -m pytest tests/unit -q
+
+EXPECTED: `325 passed` (baseline at parent `07fc6d4` is `315 passed`; +10 F-redfix-4 regression tests).
+The 10 include `test_live_and_canonical_slots_are_disjoint`, `test_slot_maps_1to1_to_its_stack`,
+`test_registry_path_of_live_warm_provider_is_not_the_reconciler_dir`,
+`test_prune_stale_deenrolled_provider_spares_reconciler_last_good`,
+`test_snapshot_refuses_to_clobber_another_domains_slot`, `test_restore_refuses_a_foreign_slot`.
+
+**HOW to verify (2) — the VETO's clearing condition, on the real node.** Non-destructive: writes only to a
+scratch `CCCI_WARM_ROOT` + one throwaway docker volume. Uses the real idle `warm-canon-keycloak` stack for
+the canonical side and a throwaway `warm-fakelive_…_data` volume as the live-warm stand-in (the live
+`warm-keycloak` stack cannot be undeployed to snapshot it).
+
+    ssh cc-ci
+    docker volume create warm-fakelive_ci_commoninternet_net_data
+    MP=$(docker volume inspect -f '{{.Mountpoint}}' warm-fakelive_ci_commoninternet_net_data); echo x > $MP/live.txt
+    git clone -q --branch redfix-m2-harness <cc-ci remote> /tmp/v && mkdir -p /tmp/vw/keycloak
+    echo '10.7.1+26.6.2' > /tmp/vw/keycloak/last_good
+    cd /tmp/v/runner && CCCI_WARM_ROOT=/tmp/vw /nix/store/jag2131a95gw6ng7grig9pj3dn2q8vrv-python3-3.12.8-env/bin/python3 - <<'PY'
+    import sys; sys.path.insert(0, ".")
+    from harness import warmsnap as ws, canonical as c
+    LIVE, CANON = ws.live_slot("keycloak"), c.canonical_slot("keycloak")
+    CANON_DOM, LIVE_DOM = c.canonical_domain("keycloak"), "warm-fakelive.ci.commoninternet.net"
+    print(ws.snap_dir(LIVE), ws.snap_dir(CANON), c.registry_path("keycloak"), sep="\n")
+    ws.snapshot(CANON, CANON_DOM, version="canon-known-good")
+    ws.snapshot(LIVE,  LIVE_DOM,  version="live-last-good")     # used to clobber the canonical
+    print("restore(canon) ->", ws.restore(CANON, CANON_DOM)["volumes"])
+    print("restore(live)  ->", ws.restore(LIVE,  LIVE_DOM )["volumes"])
+    print("last_good:", open("/tmp/vw/keycloak/last_good").read().strip())
+    try: ws.snapshot(CANON, LIVE_DOM); print("FAIL: foreign snapshot allowed")
+    except ws.SnapshotError as e: print("foreign REFUSED:", str(e)[:60])
+    PY
+    docker volume rm warm-fakelive_ci_commoninternet_net_data; rm -rf /tmp/vw /tmp/v
+
+EXPECTED (observed 2026-07-09):
+
+    /tmp/vw/keycloak/snapshot                     <- live slot
+    /tmp/vw/canon-keycloak/snapshot               <- canonical slot: DISJOINT
+    /tmp/vw/canon-keycloak/canonical.json         <- registry NOT in the reconciler's dir
+    restore(canon) -> ['warm-canon-keycloak_ci_commoninternet_net_mariadb',
+                       'warm-canon-keycloak_ci_commoninternet_net_providers']
+    restore(live)  -> ['warm-fakelive_ci_commoninternet_net_data']
+    last_good: 10.7.1+26.6.2                      <- survived the canonical seed
+    foreign REFUSED: warm slot 'canon-keycloak' holds the known-good of ...
+
+Each `restore()` returns its OWN stack's volumes (the clearing condition). Pre-fix, the same script prints
+one shared slot `/tmp/vw/keycloak/snapshot` for both and `restore(canon)` raises `SnapshotError`.
+
+**Node integrity after my own run of the above (2026-07-09).** Real warm root untouched
+(`/var/lib/ci-warm/keycloak/` = `last_good` only); `warm-canon-keycloak` mariadb digest byte-identical
+across the restore round-trip (`1201440268 48846` before and after, 391 files / 2 files); live
+`warm-keycloak…/realms/master` → **200** throughout; throwaway volume removed; scratch removed.
+
+**Scope.** keycloak only. M1 classifications and the discourse / mattermost-lts / gitea / bluesky-pds /
+mumble fixes are untouched by this commit and remain Adversary-verified (M1 PASS @2026-06-18T01:18Z; the
+five non-keycloak M2 fixes content-verified through re-confirmation #7).
+
+**Known residual, NOT in F-redfix-4's clearing condition** (filed as B-redfix-5 in BACKLOG-redfix.md, not
+blocking): `warm_reconcile.py`'s rollback `warmsnap.restore()` still sits outside the `try/except` that
+guards the upgrade, so any restore failure (e.g. a corrupt/absent snapshot) leaves live keycloak undeployed
+after `abra.undeploy()`. The F-redfix-4 race that made this reachable is gone; the pre-existing robustness
+gap is not. Recorded, not silently dropped.
 
 The prior DONE text is retained verbatim below as the historical record of what was claimed on 2026-06-18.