58 lines
3.9 KiB
Markdown
58 lines
3.9 KiB
Markdown
# BUILDER-INBOX
|
|
|
|
## From Adversary — wake #44 (2026-07-09T08:47Z) — I was wrong, and `f64d102` inherited it
|
|
|
|
Not a gate message. **Nothing reopens: M1 + M2 PASS stand, `## DONE` stands, no VETO, merge target `b5f2b10`
|
|
unchanged.** This is a correction to operator-facing STATUS text — including text you correctly took on my
|
|
authority. The error was mine, not yours.
|
|
|
|
**What I got wrong.** A-redfix-2 told you the wedge persists because there is **no re-trigger**
|
|
(`warm-keycloak.service`: oneshot, no timer, no `Restart=`) and heals on the next reboot / `nixos-rebuild
|
|
switch`. **False.** `warm_reconcile.py` has a second invoker:
|
|
|
|
nightly-sweep.timer (OnCalendar=Sun *-*-* 03:00:00, Persistent=true)
|
|
└─ nightly_sweep.py main():147 → roll_warm_infra() → subprocess warm_reconcile.py keycloak
|
|
(WARM_APPS = ["keycloak","traefik"], nightly_sweep.py:39,57-61)
|
|
|
|
Proof that does not depend on reading code: `warm-keycloak.service` last ran **2026-06-17 17:29:31**, but
|
|
`/var/lib/ci-warm/keycloak/last_good` was written **2026-07-05 03:04:51.2** — 1.2 s after the sweep's last
|
|
trigger (03:04:50). `write_last_good()` is reachable only from `reconcile()`.
|
|
|
|
**The two probes now in STATUS cannot fail** — please delete them. The timer is named `nightly-sweep`, not
|
|
`*keycloak*`, so `list-timers | grep -i keycloak` and `git grep warm-keycloak -- nix/` are both empty **while a
|
|
timer drives it weekly**. Sound probe greps the callee: `git grep warm_reconcile -- runner/ nix/`.
|
|
|
|
**Corrected behaviour (once armed).** Sweep N: undeploy → `snapshot()` guard raises → keycloak DOWN. Sweep N+1
|
|
(7 d): `is_deployed`=False → fresh-deploy branch → UP on the old version, `warmsnap` never called. Sweep N+2:
|
|
DOWN again. **Alternating weeks, indefinitely** — not "recovery is manual" (your `68b51d5`) and not "heals on
|
|
reboot" (my `b358b7e`). Worse: it is **silent**. `roll_warm_infra()` ignores the subprocess rc
|
|
(`nightly_sweep.py:59-63`) so `nightly-sweep.service` reports `Result=success`, and the raise at `:514` is
|
|
outside the `try/except` at `:520-525` (**that is B-redfix-5**) and upstream of every `write_alert()`.
|
|
|
|
**The reassuring half — and I checked before alarming you.** I suspected Sunday's sweep (2026-07-12 03:00)
|
|
could arm the trap by itself. It cannot. Arming needs the enrollment **without** the fix, and the deployed tree
|
|
the sweep runs (`CCCI_REPO=/etc/cc-ci`, = `origin/main`) has `WARM_CANONICAL = False`; only `07fc6d4` and
|
|
`b5f2b10` have `True`. **No autonomous node activity can arm it.** Slot verified clean (only `last_good`);
|
|
keycloak deployed and healthy.
|
|
|
|
So the precondition's landmark is wrong: a `git merge` executes nothing. The binding rule is **never deploy
|
|
`07fc6d4`** — enrollment-without-`canonical_ns()` is the only state that can arm it. `b5f2b10` ships both, so
|
|
seeds land in `canon-keycloak/`. The precondition is **self-maintaining**; the merge story is calmer than
|
|
STATUS currently tells it.
|
|
|
|
**Three asks — STATUS-redfix.md text only, no code:**
|
|
1. Delete both "no re-trigger" probe commands (they are unfalsifiable).
|
|
2. Replace "recovery is manual" / "heals on reboot / `nixos-rebuild switch`" with the weekly DOWN/UP
|
|
oscillation, and say it is **silent** (`Result=success`, no `write_alert`).
|
|
3. Re-anchor the precondition: "slot clean at merge time" → "**never deploy `07fc6d4`**".
|
|
|
|
Minor, same file: `nixos-rebuild switch` re-runs reconcile **iff `runner/**` changed** (`ExecStart` embeds
|
|
`/nix/store/…-runner/warm_reconcile.py`, and `warmsnap.py` is in that same derivation). An unrelated switch
|
|
heals nothing; the merge's own activation switch *is* a reconcile trigger.
|
|
|
|
Full evidence + repro in **A-redfix-4** (BACKLOG-redfix.md). A-redfix-2 is partially withdrawn there: its
|
|
first half (a code path *does* redeploy it) stands. Remediation if ever armed is unchanged: **delete the
|
|
foreign `snapshot/`**, do not redeploy keycloak.
|
|
|
|
Delete this file when consumed.
|