review(5 init): Phase 5 Adversary init — break-it probes + two blocking findings
Some checks failed
continuous-integration/drone/push Build is failing

Break-it probes (V1):
- !testmexyz on custom-html PR#2 (watched repo): correctly ignored — no Drone trigger ✓
- Non-collaborator auth: GET /orgs/recipe-maintainers/members/nonexistent-user-999 → 404 ✓
- bridge source: parse_body("!testmexyz") → (False, False) ✓

CRITICAL finding A5-2 (blocks V2–V8): testme-on-pr.sh reads Gitea commit statuses on the recipe
PR head SHA, but the bridge NEVER posts commit statuses — only PR comments. Drone posts statuses
on cc-ci repo only. POST=0 testme-on-pr.sh custom-html 2 → VERDICT=PENDING always. Fix: bridge
must POST /repos/{owner}/{recipe}/statuses/{sha} on build start/finish.

Finding A5-1: custom-html-tiny not in bridge POLL_REPOS — testme on tiny PRs would silently do
nothing. Must enroll it or use custom-html as sandbox instead.

BUILDER-INBOX.md: heads-up to Builder with both findings.
This commit is contained in:
autonomic-bot
2026-05-31 13:37:08 +00:00
parent e487b7febd
commit dbe08e4ea7
3 changed files with 178 additions and 0 deletions

61
machine-docs/BACKLOG-5.md Normal file
View File

@ -0,0 +1,61 @@
# Phase 5 — BACKLOG
SSOT: `/srv/cc-ci/cc-ci-plan/plan-phase5-verify-upgrade-flow.md`. DoD = V1V9.
Single-writer: `## Build backlog` = Builder-only; `## Adversary findings` = Adversary-only.
---
## Build backlog
(Builder-owned. Adversary reads but does not write this section.)
---
## Adversary findings
### [adversary] A5-2 — CRITICAL: testme-on-pr.sh cannot read verdicts (commit status vs comment mismatch)
**Status:** OPEN — Builder must fix before V2V8 can pass.
`testme-on-pr.sh` reads Gitea commit statuses on the recipe PR's head SHA. But the bridge NEVER
sets Gitea commit statuses on recipe repos — it only posts PR comments (the YunoHost card+badge).
Drone posts commit statuses on the `cc-ci` repo (its own repo), not on recipe repos.
**Evidence:**
- `GET /repos/recipe-maintainers/custom-html/commits/db9a95024e9d.../status``state:'', statuses:0`
- `POST=0 testme-on-pr.sh custom-html 2``VERDICT=PENDING BUILD=?` (always, on any known-green PR)
- Bridge source `bridge.py`: no call to `POST /repos/{owner}/{recipe}/statuses/{sha}` anywhere
**Required fix (one of):**
1. (Preferred) Bridge: after triggering a Drone build, POST `state=pending` on the recipe PR's head
SHA; on build completion, POST `state=success` or `state=failure` with the build URL as
`target_url`. This makes `testme-on-pr.sh` work unmodified, adds a native SCM status indicator.
2. `testme-on-pr.sh`: scan the recipe PR's comments for the `<!-- cc-ci:testme -->` marker and parse
the result from the comment body (fragile but avoids bridge changes).
**Repro:** `POST=0 MAX_WAIT=60 INTERVAL=5 /srv/cc-ci/.claude/skills/recipe-upgrade/testme-on-pr.sh custom-html 2`
→ always `VERDICT=PENDING` even after a green Drone build.
(Only Adversary closes this, after re-testing with a VERDICT=GREEN on a real green build.)
### [adversary] A5-1 — custom-html-tiny not in bridge poll list
**Status:** OPEN — needs Builder attention or plan clarification.
The Phase 5 plan specifies using `custom-html-tiny` as the sandbox recipe for V3V8 tests.
However the bridge's poll list (from live container logs) does NOT include `recipe-maintainers/custom-html-tiny`:
```
poller (primary) watching ['recipe-maintainers/cc-ci', 'recipe-maintainers/custom-html',
'recipe-maintainers/keycloak', 'recipe-maintainers/cryptpad', 'recipe-maintainers/matrix-synapse',
'recipe-maintainers/lasuite-docs', 'recipe-maintainers/n8n', 'recipe-maintainers/hedgedoc'] every 30s
```
This means `!testme` on a `custom-html-tiny` PR will NOT trigger a Drone build. Either:
1. The builder must add `custom-html-tiny` to the bridge's enrolled repos list (and enroll its tests), OR
2. Use `custom-html` (which IS enrolled) as the sandbox recipe instead, OR
3. The plan's V3V8 tests must first enroll the sandbox recipe as part of Phase 5 setup
**Repro:** `docker logs ccci-bridge_app.1.<id> 2>&1 | head -3` on cc-ci shows the poll list.
**Impact:** V3, V4, V5, V8 tests using `custom-html-tiny` as sandbox will fail silently (the `!testme`
comment is posted but the bridge never sees it → VERDICT stays PENDING forever).
(Only Adversary closes this after re-test.)

View File

@ -0,0 +1,42 @@
# BUILDER-INBOX
**From Adversary, 2026-05-31T13:45Z — Phase 5 orientation findings**
Phase 5 Adversary loop initialized. Phase 5 state files created in my clone (REVIEW-5.md,
BACKLOG-5.md). Phase 5 Builder hasn't started yet — write STATUS-5.md when you begin.
## Critical finding: testme-on-pr.sh CANNOT read verdicts (A5-2, BLOCKING for V2V8)
`testme-on-pr.sh` reads Gitea commit statuses on the recipe PR's head SHA. But:
- The bridge NEVER posts commit statuses on recipe repo commits
- Drone posts statuses only on the `cc-ci` repo (its own pipeline builds)
- Result: `POST=0 testme-on-pr.sh custom-html 2` → always `VERDICT=PENDING`, even on a known-green PR
Confirmed cold: `GET /repos/recipe-maintainers/custom-html/commits/db9a95024e9d.../status`
`state:'', statuses:0` (zero statuses on the recipe PR head SHA after Drone build #7).
This is the EXACT gap the Phase 5 plan anticipated (§2: "commit status vs comment — reconcile here").
**Recommended fix:** Modify the bridge to POST a Gitea commit status on the recipe PR's head SHA:
- On build trigger: `POST /repos/{owner}/{recipe}/statuses/{sha}` with `state=pending`, `target_url=<drone-build-url>`
- On build finish: POST `state=success` or `state=failure` with the same target_url
This makes `testme-on-pr.sh` work unmodified and adds the native Gitea PR status indicator.
Record the decision in DECISIONS.md.
## Secondary finding: custom-html-tiny not in bridge poll list (A5-1)
The plan uses `custom-html-tiny` as the sandbox recipe. It's NOT in the bridge's POLL_REPOS.
A `!testme` on a custom-html-tiny PR will silently do nothing. You'll need to either:
1. Add `custom-html-tiny` to POLL_REPOS (enroll it), OR
2. Use `custom-html` (already enrolled) as the sandbox recipe instead
Both are small fixes; document the decision.
## V1 break-it probes (no Builder action needed)
- `!testmexyz` on watched repo (custom-html PR#2): correctly rejected — 9 bridge log lines, no match ✓
- Non-collaborator auth: `GET /orgs/recipe-maintainers/members/nonexistent-user-999` → 404 ✓
- Bridge source: `parse_body("!testmexyz") → (False, False)` correctly ✓
I will verify V1 fully once you demonstrate a `!testme` → GREEN build on a recipe PR.

75
machine-docs/REVIEW-5.md Normal file
View File

@ -0,0 +1,75 @@
# Phase 5 — REVIEW (Adversary)
SSOT: `/srv/cc-ci/cc-ci-plan/plan-phase5-verify-upgrade-flow.md`. DoD = V1V9.
State files (this phase): `machine-docs/{STATUS,BACKLOG,REVIEW,JOURNAL}-5.md`. DECISIONS.md shared.
This file is **Adversary-owned** (append-only log). Builder owns STATUS-5, JOURNAL-5.
---
## Orientation — 2026-05-31T13:30Z
Phase 5 initiated (Adversary loop start). Current system state:
- Phase 3: ## DONE (all R1R8 Adversary-verified per STATUS-3.md)
- Phase 4: not started (no STATUS-4.md exists anywhere)
- Phase 5 Builder: not started (no STATUS-5.md exists)
- cc-ci services: bridge (1/1), dashboard (1/1), drone (1/1), traefik (2/2) — all healthy
- Bridge poll list: recipe-maintainers/{cc-ci, custom-html, keycloak, cryptpad, matrix-synapse, lasuite-docs, n8n, hedgedoc}
- `custom-html-tiny` (the Phase 5 sandbox recipe per the plan) is NOT in the bridge poll list
- Open PRs: custom-html-tiny PR#1 exists (chore: publish 1.0.2+2.38.0); custom-html PR#2 exists
## Break-it probes initiated — 2026-05-31T13:30Z
### V1 probe 1: !testmexyz on unmonitored repo (custom-html-tiny PR#1)
- Comment #13795 posted: `!testmexyz`
- Bridge does NOT poll custom-html-tiny (not in poll list)
- Result: no trigger expected (but not a useful V1 test — wrong repo)
- Action: re-ran probe on custom-html PR#2 (a watched repo)
### V1 probe 2: !testmexyz on watched repo (custom-html PR#2)
- Comment #13796 posted: `!testmexyz` on recipe-maintainers/custom-html PR#2
- Bridge source confirmed: `parse_body("!testmexyz") → (False, False)` — explicitly filtered
- After multiple 30s poll cycles: bridge logs still at 9 lines, ZERO match for "13796" or "testmexyz"
- `!testmexyz` CORRECTLY IGNORED by bridge — does not trigger a Drone build ✓
- V1 partial evidence: `!testmexyz` does NOT fire (confirmed cold by Adversary)
### V1 auth probe: non-collaborator rejection
- Auth endpoint verified directly: `GET /orgs/recipe-maintainers/members/nonexistent-user-999` → 404
- Bot auth: `GET /orgs/recipe-maintainers/members/autonomic-bot` → 204
- Bridge source: `is_authorized()` returns False for 404 → triggers `log("rejected: ... not authorized")`
- V1 partial evidence: non-collaborator rejection logic confirmed by source + auth endpoint test ✓
### V2 probe: testme-on-pr.sh reads verdict — CRITICAL GAP FOUND
**Problem:** `testme-on-pr.sh POST=0` on known-green custom-html PR#2 (head `db9a95024e9d`) returns:
```
VERDICT=PENDING
BUILD=?
```
**Root cause:** The script reads `GET /repos/recipe-maintainers/custom-html/commits/{sha}/status`
Gitea commit statuses. But the bridge NEVER posts commit statuses on recipe repo commits:
- Bridge `trigger_build()` fires a Drone build on the `cc-ci` repo (not the recipe repo)
- Drone posts `continuous-integration/drone/push` status on `cc-ci` commits ONLY
- Recipe PR head SHA has ZERO commit statuses (confirmed: `state: ''`, `statuses: 0`)
The bridge only posts PR comments (the YunoHost card+badge comment, U3). It does not call
`POST /repos/{owner}/{recipe}/statuses/{sha}`.
This is the EXACT gap Phase 5 §2 anticipated: "commit status vs comment — reconcile here."
**Fix options:**
1. Bridge: add Gitea commit status posting for the recipe PR's head SHA (pending → on build start;
success/failure → on build finish). This makes `testme-on-pr.sh` work unmodified.
2. `testme-on-pr.sh`: read the bridge's PR comment (scan for the `<!-- cc-ci:testme -->` marker
and parse the result text/level from the comment body) instead of commit statuses.
Option 1 is preferable (proper SCM integration; commit status is visible in the Gitea PR UI as a
checkbox next to the commit). The Builder must implement one and record the decision in DECISIONS.md.
**Impact on Phase 5:** V2 CANNOT be verified green until this is fixed. All of V3V8 also depend on
`testme-on-pr.sh` being able to return VERDICT=GREEN.
---
## Adversary findings
(Tracked in BACKLOG-5.md)