From 66a6a59212032dbd0bb521ed921f2888ac27df6a Mon Sep 17 00:00:00 2001 From: autonomic-bot Date: Mon, 1 Jun 2026 14:14:44 +0000 Subject: [PATCH] review(5): flag matrix-synapse stale-test status gap --- machine-docs/BACKLOG-5.md | 26 +++++++++++++++++++++++++ machine-docs/REVIEW-5.md | 40 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 66 insertions(+) diff --git a/machine-docs/BACKLOG-5.md b/machine-docs/BACKLOG-5.md index b487b3c..4350202 100644 --- a/machine-docs/BACKLOG-5.md +++ b/machine-docs/BACKLOG-5.md @@ -25,6 +25,32 @@ Single-writer: `## Build backlog` = Builder-only; `## Adversary findings` = Adve ## Adversary findings +### [adversary] A5-4 — `matrix-synapse` stale-test/default path leaves no recipe commit status +**Status:** OPEN — found 2026-06-01T14:16:00Z; see `REVIEW-5.md`. + +On the live V5 stale-test candidate `recipe-maintainers/matrix-synapse` PR `#1`, the PR comments show a +terminal failed `!testme` result for build `#53` plus the default-mode explanatory stale-test comment, +but the recipe PR head has **no** `cc-ci/testme` commit status at all. As a result, the helper cannot +read the verdict back from the PR and poll-only returns `PENDING` even though the PR already shows the +terminal outcome. + +**Cold repro:** +1. Use `recipe-maintainers/matrix-synapse` PR `#1`, head + `21e5d84430bdc52f8fa8aa9a40fa5bda8adf06c0`. +2. Confirm PR comments include: + - failure result comment for build `#53` (`#13872`), and + - explanatory stale-test comment (`#13877`). +3. Run: + `POST=0 MAX_WAIT=20 INTERVAL=5 /srv/cc-ci/.claude/skills/recipe-upgrade/testme-on-pr.sh matrix-synapse 1` +4. Observe: + - helper returns `VERDICT=PENDING` and `BUILD=?`; + - `GET /repos/recipe-maintainers/matrix-synapse/commits/21e5d84430bdc52f8fa8aa9a40fa5bda8adf06c0/status` + returns `{"state":"","total_count":0,"statuses":null}`. + +**Impact:** this breaks the Phase-5 requirement that the upgrade tooling read the verdict back from the +PR on the live stale-test/default path. The comment surface says the run is terminal; the status surface +still says nothing. + ### [adversary] A5-3 — `POST=1 testme-on-pr.sh` can return a stale prior GREEN on re-runs **Status:** CLOSED — re-tested 2026-06-01T03:31:30Z; see `REVIEW-5.md` follow-up entry. diff --git a/machine-docs/REVIEW-5.md b/machine-docs/REVIEW-5.md index 7a8c0fe..52cf03d 100644 --- a/machine-docs/REVIEW-5.md +++ b/machine-docs/REVIEW-5.md @@ -227,3 +227,43 @@ poll-only probe against the Builder's current V5/V6 sandbox candidate. I did **n showed `cc-ci/testme status=success` with target URL `/47`. **Verdict:** V2's poll-only path still holds on the live `n8n` sandbox PR. No new defect found. + +--- + +## Cold-verify finding — 2026-06-01T14:16:00Z + +No `Gate: CLAIMED` was pending in `STATUS-5.md`, so I used the idle slot for a fresh cold probe of +the Builder's current V5 stale-test candidate plus the newly-fixed `lasuite-meet` enrollment. I did +**not** read `JOURNAL-5.md` before forming this verdict. + +### Control probe: `lasuite-meet` enrollment fix still holds +- Cold-shell invocation: + `POST=0 MAX_WAIT=20 INTERVAL=5 /srv/cc-ci/.claude/skills/recipe-upgrade/testme-on-pr.sh lasuite-meet 2` +- The helper returned: + `VERDICT=GREEN` + `BUILD=https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/58` +- PR comment count stayed unchanged across that call (`4 -> 4`), confirming `POST=0` still polls without + re-triggering. +- `GET /repos/recipe-maintainers/lasuite-meet/commits/2d0c70779e7a87dfc240b69606c7bcff2472d720/status` + still shows `cc-ci/testme status=success` with target URL `/58`. + +### A5-4: stale-test/default path on `matrix-synapse` leaves no recipe commit status, so poll-only reports `PENDING` +- Probe target: `recipe-maintainers/matrix-synapse` PR `#1`, head + `21e5d84430bdc52f8fa8aa9a40fa5bda8adf06c0`. +- Cold-shell invocation: + `POST=0 MAX_WAIT=20 INTERVAL=5 /srv/cc-ci/.claude/skills/recipe-upgrade/testme-on-pr.sh matrix-synapse 1` +- The helper returned: + `VERDICT=PENDING` + `BUILD=?` +- Live PR comments at verify time show the run has already reached a terminal outcome on the PR: + - `#13872` (`2026-06-01T13:48:21Z`): + `cc-ci: run for matrix-synapse @ 21e5d844 ❌ failure -> .../53` + - `#13877` (`2026-06-01T14:03:04Z`): explanatory stale-test/default-mode comment telling the operator + to re-run `/recipe-upgrade matrix-synapse --with-tests`. +- But the recipe head's combined status endpoint is empty: + `GET /repos/recipe-maintainers/matrix-synapse/commits/21e5d84430bdc52f8fa8aa9a40fa5bda8adf06c0/status` + returned `{"state":"","total_count":0,"statuses":null}`. + +**Verdict:** FAIL for this live V5/V2 intersection. The PR comment surface reflects the terminal +stale-test result, but the commit-status surface is absent, so `testme-on-pr.sh` cannot read the verdict +back from the PR and incorrectly reports `PENDING`. Filed as `BACKLOG-5.md` item **A5-4**.