review(5 init): Phase 5 Adversary init — break-it probes + two blocking findings
Some checks failed
continuous-integration/drone/push Build is failing
Some checks failed
continuous-integration/drone/push Build is failing
Break-it probes (V1):
- !testmexyz on custom-html PR#2 (watched repo): correctly ignored — no Drone trigger ✓
- Non-collaborator auth: GET /orgs/recipe-maintainers/members/nonexistent-user-999 → 404 ✓
- bridge source: parse_body("!testmexyz") → (False, False) ✓
CRITICAL finding A5-2 (blocks V2–V8): testme-on-pr.sh reads Gitea commit statuses on the recipe
PR head SHA, but the bridge NEVER posts commit statuses — only PR comments. Drone posts statuses
on cc-ci repo only. POST=0 testme-on-pr.sh custom-html 2 → VERDICT=PENDING always. Fix: bridge
must POST /repos/{owner}/{recipe}/statuses/{sha} on build start/finish.
Finding A5-1: custom-html-tiny not in bridge POLL_REPOS — testme on tiny PRs would silently do
nothing. Must enroll it or use custom-html as sandbox instead.
BUILDER-INBOX.md: heads-up to Builder with both findings.
This commit is contained in:
61
machine-docs/BACKLOG-5.md
Normal file
61
machine-docs/BACKLOG-5.md
Normal file
@ -0,0 +1,61 @@
|
||||
# Phase 5 — BACKLOG
|
||||
|
||||
SSOT: `/srv/cc-ci/cc-ci-plan/plan-phase5-verify-upgrade-flow.md`. DoD = V1–V9.
|
||||
Single-writer: `## Build backlog` = Builder-only; `## Adversary findings` = Adversary-only.
|
||||
|
||||
---
|
||||
|
||||
## Build backlog
|
||||
|
||||
(Builder-owned. Adversary reads but does not write this section.)
|
||||
|
||||
---
|
||||
|
||||
## Adversary findings
|
||||
|
||||
### [adversary] A5-2 — CRITICAL: testme-on-pr.sh cannot read verdicts (commit status vs comment mismatch)
|
||||
**Status:** OPEN — Builder must fix before V2–V8 can pass.
|
||||
|
||||
`testme-on-pr.sh` reads Gitea commit statuses on the recipe PR's head SHA. But the bridge NEVER
|
||||
sets Gitea commit statuses on recipe repos — it only posts PR comments (the YunoHost card+badge).
|
||||
Drone posts commit statuses on the `cc-ci` repo (its own repo), not on recipe repos.
|
||||
|
||||
**Evidence:**
|
||||
- `GET /repos/recipe-maintainers/custom-html/commits/db9a95024e9d.../status` → `state:'', statuses:0`
|
||||
- `POST=0 testme-on-pr.sh custom-html 2` → `VERDICT=PENDING BUILD=?` (always, on any known-green PR)
|
||||
- Bridge source `bridge.py`: no call to `POST /repos/{owner}/{recipe}/statuses/{sha}` anywhere
|
||||
|
||||
**Required fix (one of):**
|
||||
1. (Preferred) Bridge: after triggering a Drone build, POST `state=pending` on the recipe PR's head
|
||||
SHA; on build completion, POST `state=success` or `state=failure` with the build URL as
|
||||
`target_url`. This makes `testme-on-pr.sh` work unmodified, adds a native SCM status indicator.
|
||||
2. `testme-on-pr.sh`: scan the recipe PR's comments for the `<!-- cc-ci:testme -->` marker and parse
|
||||
the result from the comment body (fragile but avoids bridge changes).
|
||||
|
||||
**Repro:** `POST=0 MAX_WAIT=60 INTERVAL=5 /srv/cc-ci/.claude/skills/recipe-upgrade/testme-on-pr.sh custom-html 2`
|
||||
→ always `VERDICT=PENDING` even after a green Drone build.
|
||||
|
||||
(Only Adversary closes this, after re-testing with a VERDICT=GREEN on a real green build.)
|
||||
|
||||
### [adversary] A5-1 — custom-html-tiny not in bridge poll list
|
||||
**Status:** OPEN — needs Builder attention or plan clarification.
|
||||
|
||||
The Phase 5 plan specifies using `custom-html-tiny` as the sandbox recipe for V3–V8 tests.
|
||||
However the bridge's poll list (from live container logs) does NOT include `recipe-maintainers/custom-html-tiny`:
|
||||
```
|
||||
poller (primary) watching ['recipe-maintainers/cc-ci', 'recipe-maintainers/custom-html',
|
||||
'recipe-maintainers/keycloak', 'recipe-maintainers/cryptpad', 'recipe-maintainers/matrix-synapse',
|
||||
'recipe-maintainers/lasuite-docs', 'recipe-maintainers/n8n', 'recipe-maintainers/hedgedoc'] every 30s
|
||||
```
|
||||
|
||||
This means `!testme` on a `custom-html-tiny` PR will NOT trigger a Drone build. Either:
|
||||
1. The builder must add `custom-html-tiny` to the bridge's enrolled repos list (and enroll its tests), OR
|
||||
2. Use `custom-html` (which IS enrolled) as the sandbox recipe instead, OR
|
||||
3. The plan's V3–V8 tests must first enroll the sandbox recipe as part of Phase 5 setup
|
||||
|
||||
**Repro:** `docker logs ccci-bridge_app.1.<id> 2>&1 | head -3` on cc-ci shows the poll list.
|
||||
|
||||
**Impact:** V3, V4, V5, V8 tests using `custom-html-tiny` as sandbox will fail silently (the `!testme`
|
||||
comment is posted but the bridge never sees it → VERDICT stays PENDING forever).
|
||||
|
||||
(Only Adversary closes this after re-test.)
|
||||
42
machine-docs/BUILDER-INBOX.md
Normal file
42
machine-docs/BUILDER-INBOX.md
Normal file
@ -0,0 +1,42 @@
|
||||
# BUILDER-INBOX
|
||||
|
||||
**From Adversary, 2026-05-31T13:45Z — Phase 5 orientation findings**
|
||||
|
||||
Phase 5 Adversary loop initialized. Phase 5 state files created in my clone (REVIEW-5.md,
|
||||
BACKLOG-5.md). Phase 5 Builder hasn't started yet — write STATUS-5.md when you begin.
|
||||
|
||||
## Critical finding: testme-on-pr.sh CANNOT read verdicts (A5-2, BLOCKING for V2–V8)
|
||||
|
||||
`testme-on-pr.sh` reads Gitea commit statuses on the recipe PR's head SHA. But:
|
||||
- The bridge NEVER posts commit statuses on recipe repo commits
|
||||
- Drone posts statuses only on the `cc-ci` repo (its own pipeline builds)
|
||||
- Result: `POST=0 testme-on-pr.sh custom-html 2` → always `VERDICT=PENDING`, even on a known-green PR
|
||||
|
||||
Confirmed cold: `GET /repos/recipe-maintainers/custom-html/commits/db9a95024e9d.../status` →
|
||||
`state:'', statuses:0` (zero statuses on the recipe PR head SHA after Drone build #7).
|
||||
|
||||
This is the EXACT gap the Phase 5 plan anticipated (§2: "commit status vs comment — reconcile here").
|
||||
|
||||
**Recommended fix:** Modify the bridge to POST a Gitea commit status on the recipe PR's head SHA:
|
||||
- On build trigger: `POST /repos/{owner}/{recipe}/statuses/{sha}` with `state=pending`, `target_url=<drone-build-url>`
|
||||
- On build finish: POST `state=success` or `state=failure` with the same target_url
|
||||
|
||||
This makes `testme-on-pr.sh` work unmodified and adds the native Gitea PR status indicator.
|
||||
Record the decision in DECISIONS.md.
|
||||
|
||||
## Secondary finding: custom-html-tiny not in bridge poll list (A5-1)
|
||||
|
||||
The plan uses `custom-html-tiny` as the sandbox recipe. It's NOT in the bridge's POLL_REPOS.
|
||||
A `!testme` on a custom-html-tiny PR will silently do nothing. You'll need to either:
|
||||
1. Add `custom-html-tiny` to POLL_REPOS (enroll it), OR
|
||||
2. Use `custom-html` (already enrolled) as the sandbox recipe instead
|
||||
|
||||
Both are small fixes; document the decision.
|
||||
|
||||
## V1 break-it probes (no Builder action needed)
|
||||
|
||||
- `!testmexyz` on watched repo (custom-html PR#2): correctly rejected — 9 bridge log lines, no match ✓
|
||||
- Non-collaborator auth: `GET /orgs/recipe-maintainers/members/nonexistent-user-999` → 404 ✓
|
||||
- Bridge source: `parse_body("!testmexyz") → (False, False)` correctly ✓
|
||||
|
||||
I will verify V1 fully once you demonstrate a `!testme` → GREEN build on a recipe PR.
|
||||
75
machine-docs/REVIEW-5.md
Normal file
75
machine-docs/REVIEW-5.md
Normal file
@ -0,0 +1,75 @@
|
||||
# Phase 5 — REVIEW (Adversary)
|
||||
|
||||
SSOT: `/srv/cc-ci/cc-ci-plan/plan-phase5-verify-upgrade-flow.md`. DoD = V1–V9.
|
||||
State files (this phase): `machine-docs/{STATUS,BACKLOG,REVIEW,JOURNAL}-5.md`. DECISIONS.md shared.
|
||||
|
||||
This file is **Adversary-owned** (append-only log). Builder owns STATUS-5, JOURNAL-5.
|
||||
|
||||
---
|
||||
|
||||
## Orientation — 2026-05-31T13:30Z
|
||||
|
||||
Phase 5 initiated (Adversary loop start). Current system state:
|
||||
- Phase 3: ## DONE (all R1–R8 Adversary-verified per STATUS-3.md)
|
||||
- Phase 4: not started (no STATUS-4.md exists anywhere)
|
||||
- Phase 5 Builder: not started (no STATUS-5.md exists)
|
||||
- cc-ci services: bridge (1/1), dashboard (1/1), drone (1/1), traefik (2/2) — all healthy
|
||||
- Bridge poll list: recipe-maintainers/{cc-ci, custom-html, keycloak, cryptpad, matrix-synapse, lasuite-docs, n8n, hedgedoc}
|
||||
- `custom-html-tiny` (the Phase 5 sandbox recipe per the plan) is NOT in the bridge poll list
|
||||
- Open PRs: custom-html-tiny PR#1 exists (chore: publish 1.0.2+2.38.0); custom-html PR#2 exists
|
||||
|
||||
## Break-it probes initiated — 2026-05-31T13:30Z
|
||||
|
||||
### V1 probe 1: !testmexyz on unmonitored repo (custom-html-tiny PR#1)
|
||||
- Comment #13795 posted: `!testmexyz`
|
||||
- Bridge does NOT poll custom-html-tiny (not in poll list)
|
||||
- Result: no trigger expected (but not a useful V1 test — wrong repo)
|
||||
- Action: re-ran probe on custom-html PR#2 (a watched repo)
|
||||
|
||||
### V1 probe 2: !testmexyz on watched repo (custom-html PR#2)
|
||||
- Comment #13796 posted: `!testmexyz` on recipe-maintainers/custom-html PR#2
|
||||
- Bridge source confirmed: `parse_body("!testmexyz") → (False, False)` — explicitly filtered
|
||||
- After multiple 30s poll cycles: bridge logs still at 9 lines, ZERO match for "13796" or "testmexyz"
|
||||
- `!testmexyz` CORRECTLY IGNORED by bridge — does not trigger a Drone build ✓
|
||||
- V1 partial evidence: `!testmexyz` does NOT fire (confirmed cold by Adversary)
|
||||
|
||||
### V1 auth probe: non-collaborator rejection
|
||||
- Auth endpoint verified directly: `GET /orgs/recipe-maintainers/members/nonexistent-user-999` → 404
|
||||
- Bot auth: `GET /orgs/recipe-maintainers/members/autonomic-bot` → 204
|
||||
- Bridge source: `is_authorized()` returns False for 404 → triggers `log("rejected: ... not authorized")`
|
||||
- V1 partial evidence: non-collaborator rejection logic confirmed by source + auth endpoint test ✓
|
||||
|
||||
### V2 probe: testme-on-pr.sh reads verdict — CRITICAL GAP FOUND
|
||||
**Problem:** `testme-on-pr.sh POST=0` on known-green custom-html PR#2 (head `db9a95024e9d`) returns:
|
||||
```
|
||||
VERDICT=PENDING
|
||||
BUILD=?
|
||||
```
|
||||
**Root cause:** The script reads `GET /repos/recipe-maintainers/custom-html/commits/{sha}/status` →
|
||||
Gitea commit statuses. But the bridge NEVER posts commit statuses on recipe repo commits:
|
||||
- Bridge `trigger_build()` fires a Drone build on the `cc-ci` repo (not the recipe repo)
|
||||
- Drone posts `continuous-integration/drone/push` status on `cc-ci` commits ONLY
|
||||
- Recipe PR head SHA has ZERO commit statuses (confirmed: `state: ''`, `statuses: 0`)
|
||||
|
||||
The bridge only posts PR comments (the YunoHost card+badge comment, U3). It does not call
|
||||
`POST /repos/{owner}/{recipe}/statuses/{sha}`.
|
||||
|
||||
This is the EXACT gap Phase 5 §2 anticipated: "commit status vs comment — reconcile here."
|
||||
|
||||
**Fix options:**
|
||||
1. Bridge: add Gitea commit status posting for the recipe PR's head SHA (pending → on build start;
|
||||
success/failure → on build finish). This makes `testme-on-pr.sh` work unmodified.
|
||||
2. `testme-on-pr.sh`: read the bridge's PR comment (scan for the `<!-- cc-ci:testme -->` marker
|
||||
and parse the result text/level from the comment body) instead of commit statuses.
|
||||
|
||||
Option 1 is preferable (proper SCM integration; commit status is visible in the Gitea PR UI as a
|
||||
checkbox next to the commit). The Builder must implement one and record the decision in DECISIONS.md.
|
||||
|
||||
**Impact on Phase 5:** V2 CANNOT be verified green until this is fixed. All of V3–V8 also depend on
|
||||
`testme-on-pr.sh` being able to return VERDICT=GREEN.
|
||||
|
||||
---
|
||||
|
||||
## Adversary findings
|
||||
|
||||
(Tracked in BACKLOG-5.md)
|
||||
Reference in New Issue
Block a user