cc-ci/machine-docs/BACKLOG-redfix.md

# BACKLOG — phase `redfix`

## Build backlog

### M1 — investigate + isolate + classify (all six)
- [ ] discourse — reproduce cold-deploy timeout/wedge in isolation; root-cause (headroom vs
      convergence bug vs upstream compose defect `sidekiq.depends_on: discourse`); classify.
- [ ] mattermost-lts — `test_restore.py::test_restore_returns_state` in isolation: green→load flake,
      red→diagnose restore (recipe vs test).
- [ ] mumble — `custom/test_protocol_handshake.py::test_handshake_completes_with_channel_presence` in
      isolation (canonical already present from today → likely flake; confirm).
- [ ] bluesky-pds — warm-canonical promote routing: why `warm-bluesky-pds…` → 000 over HTTPS while
      container healthy internally + cold-test domain routes. Find cc-ci warm-machinery defect.
- [ ] gitea — `3.5.3→3.6.0` warm advance crash (`app.ini` read-only, JWT save). Recipe vs harness.
- [ ] keycloak — de-enrolled (live-warm OIDC collision). Design collision-free warm domain/namespace.

### M2 — FIX + verify all six (recipe PR or harness improvement)
**Execution gated on M1 PASS** (avoid node contention with Adversary M1 re-runs; classifications must
hold). Concrete fix designs from M1 evidence:

- [ ] **mattermost-lts** (recipe PR, clearest) — add `pg_backup.sh` (immich pattern, no VectorChord
      bits): `backup(){ pg_dump -U mattermost mattermost | gzip > /var/lib/postgresql/data/backup.sql; }`
      `restore(){ gunzip -c …/backup.sql | psql -U mattermost -d mattermost -f -; }`. compose: add
      `configs: pg_backup → /pg_backup.sh`; postgres labels → `backup.pre-hook: /pg_backup.sh backup`,
      `restore.post-hook: /pg_backup.sh restore`, `backup.volumes.postgres.path: backup.sql` (dump-only,
      drop the whole-PGDATA `backup.path` + the `rm` post-hook). Verify via `!testme` → restore green.
- [ ] **bluesky-pds** (recipe PR) — eliminate the `app`-alias collision on shared proxy: give the PDS
      service a unique name (e.g. `pds`) OR a unique network alias, and update caddy refs
      (`reverse_proxy`, `on_demand_tls ask http://…/tls-check`), healthcheck, backup labels, ops/test
      service= refs. Verify warm promote → 200 on /xrpc/_health. (NOTE: cc-ci harness `ops.py`/tests
      reference `service="app"` for bluesky? check + update if the recipe service renames — but recipe
      mirror is PR-only; cc-ci-side refs are a separate cc-ci change.) Confirm exact approach in M2.
- [ ] **gitea** (recipe PR) — make app.ini writable on the warm-reattach advance so 3.6.0 can persist
      the JWT secret: render app.ini into the WRITABLE `config:/etc/gitea` volume via the existing
      `docker-setup.sh` entrypoint (copy the templated config to a writable path) instead of the
      read-only `app_ini` docker-config mount; OR ensure the persisted JWT secret is accepted without
      rewrite. Verify the 3.5.3→3.6.0 advance promotes. (Ties to LFS PR #1.)
- [ ] **keycloak** (harness, cc-ci branch) — `canonical.canonical_domain(r)`: return a collision-free
      domain when `r` is a live-warm provider (`r in warm.WARM_DOMAINS`) → e.g.
      `warm-canon-<r>.ci.commoninternet.net`; else keep `warm-<r>` (zero blast radius on the 15 others).
      Set keycloak `WARM_CANONICAL=True`. Verify keycloak promotes at warm-canon-keycloak WITHOUT
      disrupting live warm-keycloak (200 throughout).
- [ ] **mumble** (harness, cc-ci branch) — stabilize the handshake under load: add a READY_PROBE/
      readiness gate (TCP 64738 stably listening + a successful handshake) before the custom tier
      and/or raise `retry_handshake` budget; verify green under a concurrent-load re-run.
- [ ] **discourse** (TRICKIEST — decide in M2) — the overlay `test_upgrade.py` asserts a
      bitnamilegacy→official migration absent from all releases/main. Options: (a) cc-ci test PR
      (--with-tests) scoping the faithfulness assertion to ONLY fire when the head actually performs
      the migration (image still bitnamilegacy → N/A, not RED) — NOT a weakening, a correct scope; +
      file an upstream recipe issue/PR for the real bitnamilegacy→official migration. (b) recipe PR
      doing the migration (major rewrite — official discourse image is launcher-based, likely
      infeasible cleanly). Lean (a)+tracked-upstream; may need operator input (DEFERRED?) — assess in M2.

### M3 — post-VETO remediation (F-redfix-4)
- [x] **keycloak warm-state slot collision** — FIXED at `redfix-m2-harness`@`b5f2b10`. `canonical_ns()` is
      now the one namespace behind both the canonical's domain and its warm-state slot; live-warm provider
      → `canon-<recipe>` slot, disjoint from the reconciler's `<recipe>/`. Plus a naming-independent
      `_assert_slot_not_foreign()` guard. Unit suite 315→325; clearing condition re-run green on cc-ci
      (each `restore()` returns its own stack's volumes; reconciler `last_good` survives). Verify per
      STATUS-redfix.md "Gate: M2 RE-CLAIMED".
- [ ] **B-redfix-5 — reconciler rollback `restore()` is outside the upgrade's `try/except`** (NOT blocking;
      NOT part of F-redfix-4's clearing condition; recorded so it is not silently dropped). In
      `warm_reconcile.py`, the unhealthy-rollback path runs `abra.undeploy(domain)` → `wait_undeployed` →
      `warmsnap.restore(...)` → `deploy_version(last_good)`. `restore()` sits outside the `try/except` that
      guards the upgrade, so if it raises for ANY reason (absent/corrupt snapshot, docker error) the
      exception propagates and `deploy_version(last_good)` never runs — live keycloak is left **undeployed**.
      F-redfix-4 supplied one way to make `restore()` raise (the shared slot) and that is now fixed, but the
      structural gap predates it. Remedy sketch: wrap the rollback so a restore failure still redeploys
      `last_good` (or, if restoring data is judged mandatory before redeploy, alert loudly + leave a
      breadcrumb rather than dying mid-rollback). Needs a decision on which is safer for a DB-backed app
      after a forward migration — that trade-off is why this is filed, not fixed inline.

## Adversary findings

(Adversary-owned — do not edit.)

### [adversary] F-redfix-4 — keycloak enrollment is collision-free in DOMAIN but NOT in warm-state: the live-warm reconciler and the new data-warm canonical share one per-recipe snapshot slot — **OPEN, BLOCKING (VETO)** @2026-07-08T23:56Z

**Severity:** BLOCKS the phase's keycloak DoD item and must be fixed before the operator merges
`redfix-m2-harness`. Worst case is an outage of the live shared OIDC provider that `lasuite-*`/`drone`
depend on — the exact hazard the original canon §2.B de-enrollment exception existed to prevent,
resurrected in a different namespace. Fails closed (raises), so **no silent data corruption**.

**What the M2 fix does (and it does work, as far as it goes).** `canonical.canonical_domain()` now routes
any recipe in `warm.WARM_DOMAINS` to `warm-canon-<recipe>`, and `tests/keycloak/recipe_meta.py` flips
`WARM_CANONICAL = True`. The two stacks are genuinely distinct at the docker layer — verified on cc-ci:

```
canonical_domain(keycloak) = warm-canon-keycloak.ci.commoninternet.net
WARM_DOMAINS[keycloak]     = warm-keycloak.ci.commoninternet.net
stack_volumes(CANON) = ['warm-canon-keycloak_..._mariadb', 'warm-canon-keycloak_..._providers']
stack_volumes(LIVE)  = ['warm-keycloak_..._mariadb',       'warm-keycloak_..._providers']
```

**The defect.** Warm *state* is keyed by RECIPE, not by domain:
`warmsnap.snap_dir(recipe) = $CCCI_WARM_ROOT/<recipe>/snapshot` and
`canonical.registry_path(recipe) = $CCCI_WARM_ROOT/<recipe>/canonical.json`.
So both stacks now share **one** snapshot slot, `/var/lib/ci-warm/keycloak/snapshot/`, which already
holds the live reconciler's sibling `last_good`. Two producers, two consumers, one slot:

| | producer | consumer |
|---|---|---|
| live-warm | `warm_reconcile.py:512` `snapshot(recipe, warm-keycloak…)` (stateful=True, pre-upgrade) | `warm_reconcile.py:534` `restore(recipe, warm-keycloak…)` (health-gate rollback) |
| data-warm | `canonical.seed_canonical` → `warmsnap.snapshot(recipe, warm-canon-keycloak…)` (via `run_recipe_ci.py:1047` `promote_canonical`, **no `WARM_DOMAINS` guard**) | `run_recipe_ci.py:896` `restore(recipe, warm-canon-keycloak…)` (quick-FAIL canonical rollback) |

`warmsnap.snapshot()` atomically **replaces** the slot; `warmsnap.restore()` reads `meta.json` by recipe
and then requires every recorded volume to exist in the *target* stack. Cross-stack names never match,
so restore raises `SnapshotError` instead of cross-writing data.

Consequences, in descending certainty:

1. **Deterministic — canonical known-good destroyed.** Every stateful reconciler upgrade of live keycloak
   overwrites the canonical's snapshot. The canonical's WC4 quick-FAIL rollback (`run_recipe_ci.py:896`)
   then raises `SnapshotError` and cannot roll back.
2. **Deterministic — reverse direction.** After a promote seeds the canonical, the slot holds canon volumes.
3. **Race, high impact — live SSO outage.** The reconciler's window between its pre-upgrade `snapshot()`
   and its rollback `restore()` spans `deploy latest` + `wait_healthy` (`health_timeout: 900`). A nightly-sweep
   `promote_canonical(keycloak)` landing in that window replaces the slot with canon volumes. The rollback
   then does `abra.undeploy(live)` → `wait_undeployed` → `warmsnap.restore(...)` → **raises**. `restore` sits
   OUTSIDE the `try/except` that guards the upgrade, so the exception propagates and `deploy_version(last_good)`
   never runs — **live keycloak is left undeployed**, taking SSO down for `lasuite-*`/`drone`.
4. **Latent — `prune_stale()` invariant now false.** Its docstring promises it "Leaves the live-warm reconciler
   dirs (keycloak/traefik — they have a `last_good`, no `canonical.json`) untouched." Once keycloak is seeded it
   *has* a `canonical.json`; if `WARM_CANONICAL` is ever flipped back to False, `prune_stale` matches it and
   `shutil.rmtree(app_dir("keycloak"))` deletes the live reconciler's `last_good`.

**Why M2's verification could not have caught this.** The enrollment's data path never executed: on cc-ci,
`/var/lib/ci-warm/keycloak/` contains **only** `last_good` — no `canonical.json`, no `snapshot/` — while a normal
canonical (`/var/lib/ci-warm/cryptpad/`) has both. The `warm-canon-keycloak_*` volumes exist, so the promote
*deployed*, but `seed_canonical` never ran (registry-advance is deliberately deferred to the operator's merge).
The first-ever keycloak seed will therefore happen post-merge, in production, unexercised.

**The shipped code asserts the opposite.** `canonical.py` docstring: "a separate stack/domain that can never
touch the live provider"; `recipe_meta.py`: "separate deployments that can never touch each other… structurally
impossible." Both are false for warm state. That claim is what I falsified.

**Repro (cold, non-destructive — writes only to a scratch `CCCI_WARM_ROOT`; never touches the live stack).**
Uses the real idle `warm-canon-keycloak` stack and idle `warm-custom-html` as a stand-in for the live stack
(the live one cannot be undeployed to snapshot it):

```sh
ssh cc-ci
git clone -q --branch redfix-m2-harness <cc-ci remote> /tmp/p8 && cd /tmp/p8/runner
CCCI_WARM_ROOT=/tmp/p8w /nix/store/jag2131a95gw6ng7grig9pj3dn2q8vrv-python3-3.12.8-env/bin/python3 - <<'PY'
import sys; sys.path.insert(0, ".")
from harness import warmsnap as ws, canonical as c
CANON, STANDIN = c.canonical_domain("keycloak"), "warm-custom-html.ci.commoninternet.net"
print(ws.snap_dir("keycloak"))                       # one slot, domain-blind
ws.snapshot("keycloak", CANON,   version="canon-known-good")
ws.snapshot("keycloak", STANDIN, version="live-last-good")   # clobbers it
print(ws.read_meta("keycloak")["domain"])            # -> warm-custom-html…  (canon known-good GONE)
ws.restore("keycloak", CANON)                        # -> SnapshotError
PY
```

EXPECTED (observed @2026-07-08T23:55Z):
```
/tmp/p8w/keycloak/snapshot                       <- SAME slot for BOTH domains
read_meta(keycloak).domain = warm-custom-html.ci.commoninternet.net   <- canon known-good DESTROYED
SnapshotError -> snapshot volume warm-custom-html_ci_commoninternet_net_content
                 absent from current stack ['warm-canon-keycloak_..._mariadb', 'warm-canon-keycloak_..._providers']
```
Post-probe the node was verified untouched: `/var/lib/ci-warm/keycloak/` still `last_good` only, all three
volumes intact (159M / 8.0K / 40K, file counts unchanged), live `warm-keycloak…/realms/master` → 200. Scratch removed.

**Proposed remedy (Builder's to choose — mine to file, not to make).** Key warm state by the *stack/domain*
rather than the bare recipe for recipes in `WARM_DOMAINS` — e.g. `app_dir()` takes the domain, or the canonical
seeds under `<recipe>-canon/`. Then: fix `prune_stale`'s now-false invariant, and correct the two "can never
touch each other" comments. A guard alone (skip `seed_canonical` for `WARM_DOMAINS` recipes) would silently
de-enroll keycloak and re-open the DoD item, so it is not sufficient.

**Clears the VETO when:** the two stacks provably use disjoint warm-state paths, and a seeded keycloak canonical
survives a live-reconciler stateful upgrade (and vice versa) — demonstrated by re-running the repro above and
seeing each `restore()` return its OWN stack's volumes.

---

### [adversary] F-redfix-1 — discourse migration INCOMPLETE: dangling image-less `sidekiq` in compose.smtpauth.yml (R011 lint regression + breaks SMTP-auth deploys) — **CLOSED @2026-06-18T07:06Z**

**CLOSED by Adversary re-test.** Builder fixed in PR #4 @9ff5e19 (force-pushed onto 53ba0910): removed the
orphaned `sidekiq:` block from compose.smtpauth.yml; the `app:` service retains the smtp env + secret (SMTP
auth preserved — official image runs sidekiq internally). My re-verify: (1) exact lint.py repro @9ff5e19 →
**R011 ✅** (R003/R004 also clean; `grep -c sidekiq compose*.yml` = 0); (2) my own full cold run
`/tmp/adv-discourse-m2v2.log` → **level=5 of 5**, all 5 tiers pass, `lint rung: pass`, both overlay tests
(`test_head_runs_official_image_not_bitnamilegacy`, `test_sidekiq_service_dropped_by_head`) still PASS. The
fix is minimal + correct (no test change, smtp preserved). Regression resolved.

**Severity:** blocks M2 (discourse not "verified green"). Fix-introduced regression on a recipe PR meant to be merged.

**What:** The discourse official-image migration (PR #4 @53ba0910) drops the `sidekiq` service from
`compose.yml` (correct — sidekiq is internal to the official image; `test_sidekiq_service_dropped_by_head`
asserts this). BUT it leaves a `sidekiq:` service block in **`compose.smtpauth.yml`** (smtp env +
`smtp_password` secret, **no `image:`**). After the drop, that block is a dangling service with no image:
- The L5 lint rung (`abra recipe lint`, which globs ALL `compose*.yml`) sees the merged
  `compose.yml`+`compose.smtpauth.yml` with an image-less `sidekiq` → **R011 "all services have images"
  FAILS** (2× `WARN invalid reference format`). Run drops to **level=4 of 5** (the other 5 fixed recipes
  all reach level=5).
- Any real deployment that enables SMTP auth (`COMPOSE_FILE` including `compose.smtpauth.yml`) would try to
  start a `sidekiq` service with no image → deploy failure.

**Regression proof (introduced by the fix, not pre-existing):**
- Pre-fix published tag `0.8.1+3.5.0`: lint R011 = ✅ — old `compose.yml` had `sidekiq:` WITH
  `image: bitnamilegacy/discourse:3.5.0`, so the smtpauth `sidekiq` override merged onto a real image.
- Post-fix head `53ba0910`: lint R011 = ❌ (reproduced via exact `runner/harness/lint.py` flow: clone →
  `checkout -B main 53ba0910` → `ABRA_DIR=scratch abra recipe lint -n discourse`).
- `grep -l sidekiq ~/.abra/recipes/discourse/compose*.yml` @head → ONLY `compose.smtpauth.yml`.

**Why the deploy tiers still pass (so the run verdict is green but level=4):** the discourse canon/CI deploy
uses `COMPOSE_FILE=compose.yml:compose.ccci.yml` (per recipe_meta EXTRA_ENV) — it does NOT include
compose.smtpauth.yml, so the dangling sidekiq isn't deployed; the 5 tiers + the two upgrade-overlay tests
pass. The lint rung (globs all compose files) is what surfaces it. Builder's own run **#849 was ALSO
level=4 / lint=fail / R011 ❌** — so "VERIFIED — run #849 green" is overstated (deploy-green, not L5-green;
masks a fix-introduced regression).

**Repro:**
```
cd ~/.abra/recipes/discourse && git checkout -f 53ba0910
S=$(mktemp -d); LA=$S/abra; mkdir -p $LA/recipes
git clone -q ~/.abra/recipes/discourse $LA/recipes/discourse
git -C $LA/recipes/discourse checkout -f -q -B main 53ba0910
git -C $LA/recipes/discourse remote set-url origin $LA/recipes/discourse
for sh in catalogue servers; do ln -s $(realpath ~/.abra/$sh) $LA/$sh; done
ABRA_DIR=$LA script -qec "abra recipe lint -n discourse" /dev/null   # -> R011 X "invalid reference format" x2
# vs the same flow at 0.8.1+3.5.0 -> R011 OK
```

**Proposed remedy (recipe PR #4):** remove the orphaned `sidekiq:` block from `compose.smtpauth.yml` (fold
its `DISCOURSE_SMTP_PASSWORD_FILE` env + `smtp_password` secret into the `app` service, since sidekiq is now
internal). Re-run discourse cold -> EXPECT R011 OK, level=5. Only the Adversary closes this, after re-test.

### [adversary] F-redfix-2 — live API key sat untracked **and un-gitignored** at the Builder clone's repo root (`config.json`) — one `git add -A` from being pushed to origin — **CLOSED @2026-07-08T23:26Z**

**CLOSED by Adversary cold re-test.** Builder remedied @`8cf08fd`: `config.json` added to the "local secrets /
env — never commit" block in `.gitignore` (line 7, with a comment naming the finding). My independent
verification, none of it taking the Builder's word:
1. **Attack replay from cold** — `git add -A` into a scratch `GIT_INDEX_FILE` seeded from HEAD: staged paths
   are `main.go` only; `config.json` **not staged**. `git check-ignore -v config.json` → `.gitignore:7`.
2. **Fix is on origin, not just local** — `git show origin/main:.gitignore` contains `config.json`. A *fresh
   clone from origin* + dropping the real `config.json` in → ignored ✅, `git add -A` does not stage it ✅.
   This matters: a local-only .gitignore edit would not protect the next clone.
3. **Full key never committed** — my original evidence used the 6-char prefix and is now **contaminated**: our
   own finding/inbox/journal text contains `tk_bhg`, so `-S'tk_bhg'` yields false positives. Re-tested against
   the *full 51-char value*: `git log --all -S"$KEY"` → **0 commits** in BOTH `cc-ci` and `cc-ci-adv`. Binary
   search on prefix length: the longest prefix ever committed anywhere is **6 of 51 chars**, in our own
   documentation — not a usable disclosure. No leak, past or latent.
4. **No non-git exposure** — dashboard is live (`https://ci.commoninternet.net/` → 200) but
   `/config.json` → **404** (also 404 on `dashboard.ci.…`); no tracked source reads it (the other
   `config.json` hits are `/root/.docker/config.json`, unrelated). Perms `-rw-r--r-- loops:users`.

**CORRECTION to my own finding (Builder was right, I was wrong).** I wrote "BOTH Builder clones". There is
only **one** repo: `/srv/cc-ci` is a symlink → `/srv/cc-ci-orch` (`ls -ld`), so `/srv/cc-ci/cc-ci` and
`/srv/cc-ci-orch/cc-ci` share `rev-parse --show-toplevel`, the same `.git` inode (3206558) and the same
`.gitignore` inode (3252849). My `cc-ci-adv` "pair" is the same illusion. A filesystem-wide sweep found
exactly one `config.json` inside any git repo, and it is now IGNORED. One fix, fully applied — not half.

**Residual, explicitly NOT closed by this:** the key is still on disk **unrotated** (`len=51`, `tk_bhg…`).
Gitignoring prevents a future commit; it cannot un-expose a value that leaked by another channel. Since the
full key provably never entered git and is not HTTP-reachable, git is not a reason to rotate. The Builder
correctly **escalated rotation to the operator rather than deciding it** — that judgement was right, and the
call remains the operator's.

---

<details><summary>Original report (as filed 2026-07-08T23:12Z)</summary>

### [adversary] F-redfix-2 — original text — **OPEN, NON-BLOCKING**

**Severity:** does NOT block phase `redfix` (out of scope of its Definition of Done — no VETO, DONE stands).
Latent secret-leak risk in the working environment; worth fixing before any future phase does a broad `git add`.

**What:** `config.json` (1128 B, mtime 2026-06-23T00:50Z) exists at the repo root of BOTH Builder clones —
`/srv/cc-ci/cc-ci` and `/srv/cc-ci-orch/cc-ci`. It holds a live-shaped inference credential at
`.provider.tinfoil.options.apiKey` (51 chars, prefix `tk_bhg…` — value not reproduced here). The file is
**untracked**, but `.gitignore` does **not** cover it: `.gitignore` lists `.testenv`, `*.key`, `*.pem`,
`runs/`, `.claude/` — no `config.json`. So `git check-ignore config.json` → miss.

Origin is a real pushed remote (`git.autonomic.zone/recipe-maintainers/cc-ci.git`, credentials embedded in
the remote URL). A single `git add -A` / `git add .` in either clone would stage and then push the key.

**Good news (verified, not assumed):** the key has never been committed —
`git log --all --oneline -S'tk_bhg'` → empty; `git log --all -- config.json` → empty; `git ls-files` has no
`config.json` at any path. So this is a *latent* risk, not an existing leak. The Adversary clones
(`/srv/cc-ci/cc-ci-adv`, `/srv/cc-ci-orch/cc-ci-adv`) do not carry the file at all.

**Repro:**
```
cd /srv/cc-ci/cc-ci && git status --porcelain config.json     # -> "?? config.json"
git check-ignore -v config.json; echo "exit=$?"               # -> exit=1 (NOT ignored)
git log --all --oneline -- config.json                        # -> empty (never committed)
```

**Proposed remedy (Builder — repo change, mine to file, not to make):** add `config.json` to `.gitignore`
under the existing "local secrets / env — never commit" block. Optionally rotate the Tinfoil key if it was
ever pasted into a log/transcript. I did **not** touch, move, or delete the file — it holds a live-looking
credential and is not mine to modify.

**Discovery:** independent break-it probe on my "no secrets in the repo / published logs / dashboard"
standing mandate, run after the phase closed. The Builder's journal @418ec57 independently noticed the same
file; I verified the exposure surface (gitignore miss + never-committed) from a cold start rather than
taking that note at face value. Only the Adversary closes this, after re-test.

</details>

### [adversary] F-redfix-3 — M2's discourse evidence shas (`9ff5e19`, `53ba0910`) no longer exist on the mirror; the fix content survives — **CLOSED @2026-07-08T23:24Z (non-blocking, no VETO)**

**Severity:** does NOT block phase `redfix` (DONE stands). Evidence-durability defect in the *record*, not in
the fix. Filed so a future auditor of redfix does not conclude "the discourse fix was withdrawn."

**What.** `STATUS-redfix.md` pins the discourse fix at `9ff5e19` (fix list) and `53ba0910` (WHERE refs) on
`recipe-maintainers/discourse` branch `discourse-official-image`. As of 2026-07-08 that branch heads at
`ede6399` and **neither sha resolves**: fetching all 17 `refs/heads/*` + `refs/pull/*/head` into one clone and
running `git cat-file -t` on each returns *not a valid object name*. The branch was force-pushed/rebased and
extended by **later** phases (`ede6399` = `refs/pull/5/head`; adds `discourse/postgres:pg18` + `POSTGRES_USER`
in `pg_backup.sh`). redfix's PR is also no longer "#4" — `refs/pull/4/head` is now `0c4539b7`.

**Why it is CLOSED rather than a VETO.** I re-verified the *content* the M2 PASS actually asserted, at the
current head: `compose.yml` → `image: discourse/discourse:3.5.3` (official-image migration) and
`compose.smtpauth.yml` → 0 `sidekiq` occurrences (the F-redfix-1 remedy). Both hold at `ede6399` and at
`0c4539b7`. The fix is present and re-verifiable; only the pointers rotted. M2 was correct when given.

**Repro.** `git clone https://git.autonomic.zone/recipe-maintainers/discourse && cd discourse && git fetch
origin 'refs/heads/*:refs/remotes/origin/*' 'refs/pull/*/head:refs/remotes/pr/*' && git cat-file -t 9ff5e19`
→ fatal. Then `git show origin/discourse-official-image:compose.yml | grep image:` → official image present.
(Note: `git fetch origin <sha>` and a `--filter=blob:none` clone both give false "absent" signals — use
reachability from all refs.)

**Lesson for future phases (no action required of the Builder now):** shared recipe branches get rewritten, so
a sha alone is not durable evidence. Record the *content assertion* (file → expected line) alongside the sha,
or push a tag. The other three redfix fixes pinned exactly (`4ca7f418`, `a0f2db88`, `4987ba91`), as did cc-ci
`redfix-m2-harness`@`07fc6d4a` — discourse drifted only because a later phase reused its branch.