Files
cc-ci/machine-docs/ADVERSARY-INBOX.md

1.9 KiB

[builder heads-up @2026-06-10T20:35Z] restore-cluster root cause refined: REF mismatch vs baseline — not load, possibly not harness at all

Serial re-runs reproduced the restore RED for discourse AND plausible (m2rr-* logs/results) — so NOT load flake for those two. But before calling it a harness regression I checked the REFs:

  • ALL FOUR failing recipes' baseline evidence (my matrix) came from PR-head runs: plausible run 308 ref=13458fac56a1, immich 307 ref=107d7220adce, discourse 184 ref=7ae7b0f76efb, mattermost-lts 196 ref=a333e31a6002.
  • My sweep ran mirror default-branch heads (per plan M2.3 primary form): plausible head da159375 is from 2025-01, discourse head 7d53d4ec from 2025-12 — none moved today (Gitea API).
  • So the sweep tested DIFFERENT (much older) recipe content than every baseline row — the baseline comparison for those four is invalid as run. matrix-synapse also ran content newer than its baseline and PASSED, so the harness handles ref drift fine elsewhere.

Pre-existing-mode corroboration (you found the docstring; I found live logs): pre-rcust full runs ~/ccci-discourse-full6/7/8.log (05-31) show the SAME backup-verify attempt-1 failure, with the retry then succeeding — same race, old code, default-branch-era content.

Plan to resolve (running now, serially):

  1. Finish the in-flight m2rr re-runs (bluesky-pds, mattermost-lts, immich, lasuite-drive).
  2. Re-run the four restore-RED recipes at their baseline REFs (the PR heads above), CCCI_RUN_ID=m2b- — the true apples-to-apples. Green there = harness matches its pre-change baseline on identical content.
  3. Your explicit A/B ask regardless: discourse on PRE-MERGE main (c2508c7 clone at /root/m2-oldmain) at the default-branch REF, CCCI_RUN_ID=ab-discourse-oldmain — old harness + same (old) content. RED there = content/env, cleared harness. All evidence to STATUS-rcust.md before any M2 claim. No fix-forward, no test edits.