review(redfix): wake #44 — I REFUTE my own A-redfix-2; nightly-sweep.timer is a weekly re-trigger, wedge is silent + recurring (A-redfix-4)

machine-docs/BUILDER-INBOX.md

@@ -0,0 +1,57 @@
+# BUILDER-INBOX
+
+## From Adversary — wake #44 (2026-07-09T08:47Z) — I was wrong, and `f64d102` inherited it
+
+Not a gate message. **Nothing reopens: M1 + M2 PASS stand, `## DONE` stands, no VETO, merge target `b5f2b10`
+unchanged.** This is a correction to operator-facing STATUS text — including text you correctly took on my
+authority. The error was mine, not yours.
+
+**What I got wrong.** A-redfix-2 told you the wedge persists because there is **no re-trigger**
+(`warm-keycloak.service`: oneshot, no timer, no `Restart=`) and heals on the next reboot / `nixos-rebuild
+switch`. **False.** `warm_reconcile.py` has a second invoker:
+
+    nightly-sweep.timer  (OnCalendar=Sun *-*-* 03:00:00, Persistent=true)
+      └─ nightly_sweep.py main():147 → roll_warm_infra() → subprocess warm_reconcile.py keycloak
+         (WARM_APPS = ["keycloak","traefik"], nightly_sweep.py:39,57-61)
+
+Proof that does not depend on reading code: `warm-keycloak.service` last ran **2026-06-17 17:29:31**, but
+`/var/lib/ci-warm/keycloak/last_good` was written **2026-07-05 03:04:51.2** — 1.2 s after the sweep's last
+trigger (03:04:50). `write_last_good()` is reachable only from `reconcile()`.
+
+**The two probes now in STATUS cannot fail** — please delete them. The timer is named `nightly-sweep`, not
+`*keycloak*`, so `list-timers | grep -i keycloak` and `git grep warm-keycloak -- nix/` are both empty **while a
+timer drives it weekly**. Sound probe greps the callee: `git grep warm_reconcile -- runner/ nix/`.
+
+**Corrected behaviour (once armed).** Sweep N: undeploy → `snapshot()` guard raises → keycloak DOWN. Sweep N+1
+(7 d): `is_deployed`=False → fresh-deploy branch → UP on the old version, `warmsnap` never called. Sweep N+2:
+DOWN again. **Alternating weeks, indefinitely** — not "recovery is manual" (your `68b51d5`) and not "heals on
+reboot" (my `b358b7e`). Worse: it is **silent**. `roll_warm_infra()` ignores the subprocess rc
+(`nightly_sweep.py:59-63`) so `nightly-sweep.service` reports `Result=success`, and the raise at `:514` is
+outside the `try/except` at `:520-525` (**that is B-redfix-5**) and upstream of every `write_alert()`.
+
+**The reassuring half — and I checked before alarming you.** I suspected Sunday's sweep (2026-07-12 03:00)
+could arm the trap by itself. It cannot. Arming needs the enrollment **without** the fix, and the deployed tree
+the sweep runs (`CCCI_REPO=/etc/cc-ci`, = `origin/main`) has `WARM_CANONICAL = False`; only `07fc6d4` and
+`b5f2b10` have `True`. **No autonomous node activity can arm it.** Slot verified clean (only `last_good`);
+keycloak deployed and healthy.
+
+So the precondition's landmark is wrong: a `git merge` executes nothing. The binding rule is **never deploy
+`07fc6d4`** — enrollment-without-`canonical_ns()` is the only state that can arm it. `b5f2b10` ships both, so
+seeds land in `canon-keycloak/`. The precondition is **self-maintaining**; the merge story is calmer than
+STATUS currently tells it.
+
+**Three asks — STATUS-redfix.md text only, no code:**
+1. Delete both "no re-trigger" probe commands (they are unfalsifiable).
+2. Replace "recovery is manual" / "heals on reboot / `nixos-rebuild switch`" with the weekly DOWN/UP
+   oscillation, and say it is **silent** (`Result=success`, no `write_alert`).
+3. Re-anchor the precondition: "slot clean at merge time" → "**never deploy `07fc6d4`**".
+
+Minor, same file: `nixos-rebuild switch` re-runs reconcile **iff `runner/**` changed** (`ExecStart` embeds
+`/nix/store/…-runner/warm_reconcile.py`, and `warmsnap.py` is in that same derivation). An unrelated switch
+heals nothing; the merge's own activation switch *is* a reconcile trigger.
+
+Full evidence + repro in **A-redfix-4** (BACKLOG-redfix.md). A-redfix-2 is partially withdrawn there: its
+first half (a code path *does* redeploy it) stands. Remediation if ever armed is unchanged: **delete the
+foreign `snapshot/`**, do not redeploy keycloak.
+
+Delete this file when consumed.