review(5): flag matrix-synapse stale-test status gap
This commit is contained in:
@ -25,6 +25,32 @@ Single-writer: `## Build backlog` = Builder-only; `## Adversary findings` = Adve
|
||||
|
||||
## Adversary findings
|
||||
|
||||
### [adversary] A5-4 — `matrix-synapse` stale-test/default path leaves no recipe commit status
|
||||
**Status:** OPEN — found 2026-06-01T14:16:00Z; see `REVIEW-5.md`.
|
||||
|
||||
On the live V5 stale-test candidate `recipe-maintainers/matrix-synapse` PR `#1`, the PR comments show a
|
||||
terminal failed `!testme` result for build `#53` plus the default-mode explanatory stale-test comment,
|
||||
but the recipe PR head has **no** `cc-ci/testme` commit status at all. As a result, the helper cannot
|
||||
read the verdict back from the PR and poll-only returns `PENDING` even though the PR already shows the
|
||||
terminal outcome.
|
||||
|
||||
**Cold repro:**
|
||||
1. Use `recipe-maintainers/matrix-synapse` PR `#1`, head
|
||||
`21e5d84430bdc52f8fa8aa9a40fa5bda8adf06c0`.
|
||||
2. Confirm PR comments include:
|
||||
- failure result comment for build `#53` (`#13872`), and
|
||||
- explanatory stale-test comment (`#13877`).
|
||||
3. Run:
|
||||
`POST=0 MAX_WAIT=20 INTERVAL=5 /srv/cc-ci/.claude/skills/recipe-upgrade/testme-on-pr.sh matrix-synapse 1`
|
||||
4. Observe:
|
||||
- helper returns `VERDICT=PENDING` and `BUILD=?`;
|
||||
- `GET /repos/recipe-maintainers/matrix-synapse/commits/21e5d84430bdc52f8fa8aa9a40fa5bda8adf06c0/status`
|
||||
returns `{"state":"","total_count":0,"statuses":null}`.
|
||||
|
||||
**Impact:** this breaks the Phase-5 requirement that the upgrade tooling read the verdict back from the
|
||||
PR on the live stale-test/default path. The comment surface says the run is terminal; the status surface
|
||||
still says nothing.
|
||||
|
||||
### [adversary] A5-3 — `POST=1 testme-on-pr.sh` can return a stale prior GREEN on re-runs
|
||||
**Status:** CLOSED — re-tested 2026-06-01T03:31:30Z; see `REVIEW-5.md` follow-up entry.
|
||||
|
||||
|
||||
@ -227,3 +227,43 @@ poll-only probe against the Builder's current V5/V6 sandbox candidate. I did **n
|
||||
showed `cc-ci/testme status=success` with target URL `/47`.
|
||||
|
||||
**Verdict:** V2's poll-only path still holds on the live `n8n` sandbox PR. No new defect found.
|
||||
|
||||
---
|
||||
|
||||
## Cold-verify finding — 2026-06-01T14:16:00Z
|
||||
|
||||
No `Gate: <Mn> CLAIMED` was pending in `STATUS-5.md`, so I used the idle slot for a fresh cold probe of
|
||||
the Builder's current V5 stale-test candidate plus the newly-fixed `lasuite-meet` enrollment. I did
|
||||
**not** read `JOURNAL-5.md` before forming this verdict.
|
||||
|
||||
### Control probe: `lasuite-meet` enrollment fix still holds
|
||||
- Cold-shell invocation:
|
||||
`POST=0 MAX_WAIT=20 INTERVAL=5 /srv/cc-ci/.claude/skills/recipe-upgrade/testme-on-pr.sh lasuite-meet 2`
|
||||
- The helper returned:
|
||||
`VERDICT=GREEN`
|
||||
`BUILD=https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/58`
|
||||
- PR comment count stayed unchanged across that call (`4 -> 4`), confirming `POST=0` still polls without
|
||||
re-triggering.
|
||||
- `GET /repos/recipe-maintainers/lasuite-meet/commits/2d0c70779e7a87dfc240b69606c7bcff2472d720/status`
|
||||
still shows `cc-ci/testme status=success` with target URL `/58`.
|
||||
|
||||
### A5-4: stale-test/default path on `matrix-synapse` leaves no recipe commit status, so poll-only reports `PENDING`
|
||||
- Probe target: `recipe-maintainers/matrix-synapse` PR `#1`, head
|
||||
`21e5d84430bdc52f8fa8aa9a40fa5bda8adf06c0`.
|
||||
- Cold-shell invocation:
|
||||
`POST=0 MAX_WAIT=20 INTERVAL=5 /srv/cc-ci/.claude/skills/recipe-upgrade/testme-on-pr.sh matrix-synapse 1`
|
||||
- The helper returned:
|
||||
`VERDICT=PENDING`
|
||||
`BUILD=?`
|
||||
- Live PR comments at verify time show the run has already reached a terminal outcome on the PR:
|
||||
- `#13872` (`2026-06-01T13:48:21Z`):
|
||||
`cc-ci: run for matrix-synapse @ 21e5d844 ❌ failure -> .../53`
|
||||
- `#13877` (`2026-06-01T14:03:04Z`): explanatory stale-test/default-mode comment telling the operator
|
||||
to re-run `/recipe-upgrade matrix-synapse --with-tests`.
|
||||
- But the recipe head's combined status endpoint is empty:
|
||||
`GET /repos/recipe-maintainers/matrix-synapse/commits/21e5d84430bdc52f8fa8aa9a40fa5bda8adf06c0/status`
|
||||
returned `{"state":"","total_count":0,"statuses":null}`.
|
||||
|
||||
**Verdict:** FAIL for this live V5/V2 intersection. The PR comment surface reflects the terminal
|
||||
stale-test result, but the commit-status surface is absent, so `testme-on-pr.sh` cannot read the verdict
|
||||
back from the PR and incorrectly reports `PENDING`. Filed as `BACKLOG-5.md` item **A5-4**.
|
||||
|
||||
Reference in New Issue
Block a user