review(redfix): wake #43 — A-redfix-2 REFUTES "no code path redeploys it"; A-redfix-3 voids the "both clones" evidence

Probed the two operator-facing claims in the Builder's new commits (cbeceea, d5dc547). No inbox, no gate.

A-redfix-2 (LOW-MED) — "The wedge does not self-heal ... no code path redeploys it. Recovery is manual."
Conclusion right, mechanism wrong. abra.undeploy() does not remove the app .env, so the next reconcile()
sees current_version resolvable + is_deployed False and takes the fresh-deploy branch
(warm_reconcile.py:471-479), which redeploys and NEVER calls warmsnap -- the guard is unreachable there.
What actually blocks self-healing is the lack of a re-trigger: warm-keycloak.service is Type=oneshot with
no timer and no Restart=. So it heals on the next reboot / nixos-rebuild switch, then RE-WEDGES on the next
due upgrade because the foreign snapshot/meta.json is still in the slot. The failure therefore presents as
an intermittent flake, and the correct remediation is removing the foreign snapshot/, not redeploying
keycloak. Asked the Builder to correct two sentences; the precondition itself is unchanged and still right.

A-redfix-3 (INFO) — "main.go present in both clones => written outside git". Evidence void: /srv/cc-ci is a
SYMLINK to /srv/cc-ci-orch, so both paths are the same inode (2049:3254604, links=1). One file, one clone.
Risk bounded: 281-byte hello-world net/http listener, no go.mod, go not installed, nothing on :8080, in no
commit, unreferenced, inert. Concur with leaving it in place.

No VETO. DONE stands, M1+M2 PASS stand, merge target b5f2b10 unchanged. B-redfix-8 unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SoraVF2hkEMExHswFX1g4M
This commit is contained in:
autonomic-bot
2026-07-09 08:32:49 +00:00
parent d5dc547cd9
commit 10576574c5
2 changed files with 119 additions and 0 deletions

View File

@ -517,3 +517,84 @@ reachability from all refs.)
a sha alone is not durable evidence. Record the *content assertion* (file → expected line) alongside the sha,
or push a tag. The other three redfix fixes pinned exactly (`4ca7f418`, `a0f2db88`, `4987ba91`), as did cc-ci
`redfix-m2-harness`@`07fc6d4a` — discourse drifted only because a later phase reused its branch.
### A-redfix-2 [adversary] — STATUS's "no code path redeploys it / recovery is manual" is REFUTED: the B-redfix-5 wedge **does** self-heal on the next unit activation, which makes it look intermittent
Severity **LOW-MED**, **no VETO, does not reopen the phase** (operator-facing description of a *deferred*
item, not a DoD item). Merge target `b5f2b10` unchanged. Filed because the stated mechanism is wrong and the
wrong mechanism points the operator at the wrong remediation.
**What STATUS-redfix.md (`68b51d5`, lines ~91-94) claims:**
> "**The wedge does not self-heal.** … A raise at `:514` or `:536` therefore escapes to `SystemExit` with
> live keycloak already undeployed by the preceding `abra.undeploy()` — **no code path redeploys it.
> Recovery is manual.**"
**The conclusion "does not self-heal" is right; the reason given is wrong.** There *is* a code path that
redeploys it, and it runs automatically.
**Repro (static, from any clone; no node access needed):**
git show b5f2b10:runner/warm_reconcile.py | sed -n '468,479p' # the fresh-deploy branch
git show b5f2b10:runner/harness/abra.py | sed -n '295,298p' # undeploy does NOT remove the .env
sed -n '38,45p' nix/modules/warm-keycloak.nix # oneshot, RemainAfterExit, no Restart=
git grep -n "warm-keycloak" -- nix/ | grep -iE "timer|OnCalendar|Restart=" # -> empty
**Chain, each leg verified:**
1. `abra.undeploy()` runs only `abra app undeploy` — it does **not** remove `~/.abra/servers/default/<domain>.env`.
2. So on the next `reconcile("keycloak")`: `current_version(domain)` still resolves from `TYPE=` in that
`.env`, while `is_deployed(domain)` is `False` (no docker services for the stack).
3. `warm_reconcile.py:471-479` therefore takes the **fresh-deploy branch**: `target = current or latest`
`deploy_version(...)``wait_healthy``write_last_good``return deployed-fresh:<target>`.
**This branch never calls `warmsnap`**, so `_assert_slot_not_foreign` is never reached and the foreign
meta cannot block it. Live keycloak comes back up.
**What actually prevents self-healing** is not the absence of a redeploy path but the absence of a
**re-trigger**: `warm-keycloak.service` is `Type=oneshot`, `RemainAfterExit=true`, `wantedBy=multi-user.target`,
with **no timer and no `Restart=`** (the module comment says it "converges every activation/boot"). Nothing
re-runs it between activations.
**Why this matters operationally — the failure is intermittent, not sticky.**
- The wedge persists only until the next **reboot or `nixos-rebuild switch`**, at which point the unit
re-activates, hits the fresh-deploy branch and silently restores live keycloak. "Recovery is manual" will
mislead an operator into hand-redeploying something that a routine rebuild already fixed.
- But the foreign `snapshot/meta.json` is **still in the slot**. So the box heals, then **re-wedges the next
time an upgrade is actually due** (`current != latest` → upgrade path → `:514` `snapshot()` → guard raises
`abra.undeploy()` at `:512` has already run). Heal → re-wedge → heal: it will read as a *flake*.
- Therefore the correct remediation is **remove the foreign `snapshot/` from `/var/lib/ci-warm/keycloak/`**,
not "redeploy keycloak". Redeploying treats the symptom and the wedge returns on the next due upgrade.
**Ask of the Builder:** correct the two sentences in STATUS. Suggested: *"The wedge does not self-heal between
activations — `warm-keycloak.service` is a `oneshot` with no timer and no `Restart=`. The next reboot or
`nixos-rebuild switch` re-activates it and the fresh-deploy branch (`:471-479`, which never calls `warmsnap`)
does restore live keycloak — but the foreign meta remains, so it re-wedges on the next due upgrade. Remediation
is to remove the foreign `snapshot/`, not to redeploy."* The **precondition itself is unchanged and still
correct**; only the mechanism/remediation sentences are wrong.
### A-redfix-3 [adversary] — STATUS's `main.go` "present in both clones ⇒ written outside git" evidence is void: `/srv/cc-ci` is a **symlink** to `/srv/cc-ci-orch`, so there is one file, not two
Severity **INFO**. No VETO, no gate impact, no DoD impact. The *conclusion* (untracked, not from this phase,
leave it alone) is fine; the supporting evidence is an artifact.
**What STATUS-redfix.md (`42bc4e4`, lines ~470-472) claims:**
> "`/srv/cc-ci-orch/cc-ci/main.go` **and** `/srv/cc-ci/cc-ci/main.go` — present in **both** clones, identical
> size, identical mtime … i.e. written by something outside git (**a git operation cannot leave the same
> untracked file in two clones**)."
**Repro:**
ls -la /srv/ # cc-ci -> /srv/cc-ci-orch (symlink)
stat -c '%d:%i %n' /srv/cc-ci-orch/cc-ci/main.go /srv/cc-ci/cc-ci/main.go
# -> 2049:3254604 for BOTH; stat -c 'links=%h' -> links=1
Same device, same inode, link count 1: **one file, one clone, seen through a symlink.** The same holds for the
Adversary clones (`/srv/cc-ci-orch/cc-ci-adv` and `/srv/cc-ci/cc-ci-adv` are inode `3206945`). The "two clones"
observation carries **zero** information about the writer, and the identical size/mtime are tautological.
**Risk bounded (read-only):** contents are a 281-byte hello-world `net/http` listener; `sha256
bdbc3bf167cd20f30c00880005f4f994f17f3722660973c9c65c7bf33e81ffaf`; no `go.mod`; **`go` is not installed**;
**nothing is listening on `:8080`**; `git log --all -- main.go` is empty; not gitignored; unreferenced by any
tracked file. It cannot execute. Present in the **Builder's** clone only — **absent from both Adversary
clone paths**, which is consistent with a single stray write into one working tree.
**Ask of the Builder:** drop the "both clones ⇒ outside git" inference (keep the file, keep the flag). Agreed
on not deleting it: neither of us created it.

View File

@ -1321,3 +1321,41 @@ B-redfix-5 stays deferred/open (now correctly scoped to both call sites). B-redf
(OPEN/HIGH/operator-rotation-only; not re-probed, by design — Builder concurs).
Inbox consumed and deleted. Did not read JOURNAL-redfix.md before forming this verdict.
---
## Wake #43 — 2026-07-09T08:31Z — two new Builder claims probed; **one REFUTED** (A-redfix-2), one's evidence void (A-redfix-3). Still no VETO.
Autonomous tick. No inbox, no gate claimed. Two new Builder commits (`68b51d5`, `42bc4e4`) each assert an
operator-facing claim I had never verified. Probed both cold.
**1. `68b51d5` — "The wedge does not self-heal … no code path redeploys it. Recovery is manual."**
The *conclusion* holds; the *mechanism* is wrong → filed **A-redfix-2**.
`abra.undeploy()` (`abra.py:295-297`) does not remove the app `.env`, so the next `reconcile()` finds
`current_version` resolvable and `is_deployed` False, and takes the **fresh-deploy branch**
(`warm_reconcile.py:471-479`) which redeploys `current or latest`. That branch **never calls `warmsnap`**, so
the guard is never reached and the foreign meta cannot block it. A code path **does** redeploy it.
What actually blocks self-healing is the absence of a **re-trigger**: `warm-keycloak.service` is
`Type=oneshot`, `RemainAfterExit=true`, **no timer, no `Restart=`** (`git grep warm-keycloak -- nix/ | grep -iE
'timer|OnCalendar|Restart='` → empty). So it heals on the next reboot / `nixos-rebuild switch`, then
**re-wedges on the next due upgrade** because the foreign `snapshot/meta.json` is still in the slot.
Consequence the Builder's wording hides: **the failure presents as an intermittent flake, and the correct
remediation is deleting the foreign `snapshot/`, not redeploying keycloak.** I checked myself before filing
(wake-#38 taught me to): confirmed no timer, confirmed `.env` survives undeploy, read the branch verbatim.
**2. `42bc4e4` — untracked `main.go` "present in both clones ⇒ written by something outside git".**
Evidence void → filed **A-redfix-3**. `/srv/cc-ci` is a **symlink to `/srv/cc-ci-orch`**
(`ls -la /srv/`), so `/srv/cc-ci/cc-ci/main.go` and `/srv/cc-ci-orch/cc-ci/main.go` are the **same inode**
(`2049:3254604`, `links=1`). One file, one clone. The "two clones" premise — and with it the "a git operation
cannot leave the same untracked file in two clones" inference — carries no information. Same symlink applies
to my own clones (`cc-ci-adv` → inode `3206945` via both paths), which is why I saw the file in neither: it
sits in the Builder's tree only.
Risk bounded read-only: 281-byte hello-world `net/http` listener, `sha256 bdbc3bf1…`, no `go.mod`, `go` **not
installed**, **nothing listening on :8080**, in no commit on any ref, not gitignored, unreferenced by any
tracked file. Inert. Concur with leaving it in place — neither of us created it.
**Verdict: no VETO. `## DONE` stands; M1 + M2 PASS stand; merge target `b5f2b10` unchanged.** Both findings
are LOW-MED/INFO and concern operator-facing *descriptions*, not DoD items. The B-redfix-5 **precondition
itself is unchanged and still correct** — A-redfix-2 corrects only the mechanism + remediation sentences.
B-redfix-8 unchanged (OPEN/HIGH/operator-rotation-only; not re-probed, by design).
Did not read JOURNAL-redfix.md before forming these verdicts.