230 lines
12 KiB
Markdown
230 lines
12 KiB
Markdown
# Phase 5 — REVIEW (Adversary)
|
||
|
||
SSOT: `/srv/cc-ci/cc-ci-plan/plan-phase5-verify-upgrade-flow.md`. DoD = V1–V9.
|
||
State files (this phase): `machine-docs/{STATUS,BACKLOG,REVIEW,JOURNAL}-5.md`. DECISIONS.md shared.
|
||
|
||
This file is **Adversary-owned** (append-only log). Builder owns STATUS-5, JOURNAL-5.
|
||
|
||
---
|
||
|
||
## Orientation — 2026-05-31T13:30Z
|
||
|
||
Phase 5 initiated (Adversary loop start). Current system state:
|
||
- Phase 3: ## DONE (all R1–R8 Adversary-verified per STATUS-3.md)
|
||
- Phase 4: not started (no STATUS-4.md exists anywhere)
|
||
- Phase 5 Builder: not started (no STATUS-5.md exists)
|
||
- cc-ci services: bridge (1/1), dashboard (1/1), drone (1/1), traefik (2/2) — all healthy
|
||
- Bridge poll list: recipe-maintainers/{cc-ci, custom-html, keycloak, cryptpad, matrix-synapse, lasuite-docs, n8n, hedgedoc}
|
||
- `custom-html-tiny` (the Phase 5 sandbox recipe per the plan) is NOT in the bridge poll list
|
||
- Open PRs: custom-html-tiny PR#1 exists (chore: publish 1.0.2+2.38.0); custom-html PR#2 exists
|
||
|
||
## Break-it probes initiated — 2026-05-31T13:30Z
|
||
|
||
### V1 probe 1: !testmexyz on unmonitored repo (custom-html-tiny PR#1)
|
||
- Comment #13795 posted: `!testmexyz`
|
||
- Bridge does NOT poll custom-html-tiny (not in poll list)
|
||
- Result: no trigger expected (but not a useful V1 test — wrong repo)
|
||
- Action: re-ran probe on custom-html PR#2 (a watched repo)
|
||
|
||
### V1 probe 2: !testmexyz on watched repo (custom-html PR#2)
|
||
- Comment #13796 posted: `!testmexyz` on recipe-maintainers/custom-html PR#2
|
||
- Bridge source confirmed: `parse_body("!testmexyz") → (False, False)` — explicitly filtered
|
||
- After multiple 30s poll cycles: bridge logs still at 9 lines, ZERO match for "13796" or "testmexyz"
|
||
- `!testmexyz` CORRECTLY IGNORED by bridge — does not trigger a Drone build ✓
|
||
- V1 partial evidence: `!testmexyz` does NOT fire (confirmed cold by Adversary)
|
||
|
||
### V1 auth probe: non-collaborator rejection
|
||
- Auth endpoint verified directly: `GET /orgs/recipe-maintainers/members/nonexistent-user-999` → 404
|
||
- Bot auth: `GET /orgs/recipe-maintainers/members/autonomic-bot` → 204
|
||
- Bridge source: `is_authorized()` returns False for 404 → triggers `log("rejected: ... not authorized")`
|
||
- V1 partial evidence: non-collaborator rejection logic confirmed by source + auth endpoint test ✓
|
||
|
||
### V2 probe: testme-on-pr.sh reads verdict — CRITICAL GAP FOUND
|
||
**Problem:** `testme-on-pr.sh POST=0` on known-green custom-html PR#2 (head `db9a95024e9d`) returns:
|
||
```
|
||
VERDICT=PENDING
|
||
BUILD=?
|
||
```
|
||
**Root cause:** The script reads `GET /repos/recipe-maintainers/custom-html/commits/{sha}/status` →
|
||
Gitea commit statuses. But the bridge NEVER posts commit statuses on recipe repo commits:
|
||
- Bridge `trigger_build()` fires a Drone build on the `cc-ci` repo (not the recipe repo)
|
||
- Drone posts `continuous-integration/drone/push` status on `cc-ci` commits ONLY
|
||
- Recipe PR head SHA has ZERO commit statuses (confirmed: `state: ''`, `statuses: 0`)
|
||
|
||
The bridge only posts PR comments (the YunoHost card+badge comment, U3). It does not call
|
||
`POST /repos/{owner}/{recipe}/statuses/{sha}`.
|
||
|
||
This is the EXACT gap Phase 5 §2 anticipated: "commit status vs comment — reconcile here."
|
||
|
||
**Builder fix (`5d48436`):** Added `post_commit_status()` to bridge.py; calls it from:
|
||
- `process_testme()`: posts `cc-ci/testme: pending` on build trigger ✓
|
||
- `watch_and_reflect()`: posts `cc-ci/testme: success/failure` on build completion ✓
|
||
Fix uses `owner, name, sha` from the RECIPE repo (not the cc-ci repo) — correctly targets the recipe PR ✓
|
||
|
||
**Bot permission verified:** `POST /repos/recipe-maintainers/custom-html-tiny/statuses/{sha}` → HTTP 201 ✓
|
||
(tested directly via bot basic auth; bot has write access to org repos)
|
||
|
||
**Deployment pending:** Bridge NOT yet deployed (deployed hash `6377f9571f3b` ≠ source hash `3761c4221042`).
|
||
The `!testme` on custom-html-tiny PR#2 (comment #13802) is pending bridge update + redeploy.
|
||
|
||
**Probe artifact:** I accidentally posted `cc-ci/testme-adv-probe: success` on custom-html-tiny
|
||
PR#2 head (`156a49ac`) while testing permissions. Alerted Builder in BUILDER-INBOX. Impact: false-
|
||
positive window before bridge deployment; clears once bridge posts real `cc-ci/testme` status.
|
||
|
||
---
|
||
|
||
## Cold-verify findings — 2026-05-31T14:10Z (V1/V2/V3/V7 partial)
|
||
|
||
**System state at verify time:**
|
||
- Bridge: `cc-ci-bridge:3761c4221042` (updated, A5-1+A5-2 fix deployed) ✓
|
||
- Bridge poll list: includes `recipe-maintainers/custom-html-tiny` ✓
|
||
- Drone build #29: `success` for `custom-html-tiny@156a49ac` (PR #2)
|
||
|
||
### V1 evidence (cold-verified)
|
||
- `!testme` on custom-html-tiny PR#2 (comment #13803 by `autonomic-bot`): bridge triggered build #29 within the next poll cycle (30s window)
|
||
- Bridge log: `[poll] triggered build 29 for custom-html-tiny@156a49ac (PR #2, comment 13803) by autonomic-bot` ✓
|
||
- Bridge log: `reflected outcome build 29 (custom-html-tiny PR #2): success` ✓
|
||
- Result comment #13804 posted on PR#2: `<!-- cc-ci:testme -->\n🌻 **cc-ci** — custom-html-tiny @ 156a49ac ✅ **passed**` ✓
|
||
- Commit status `cc-ci/testme` on PR#2 head: `state=success`, `target_url=https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/29` ✓
|
||
- V1 non-trigger probes (from earlier): `!testmexyz` — no build triggered ✓; auth endpoint verifies non-member → 404 ✓
|
||
- **V1: PASS (partial — !testme trigger + result-back to PR verified; non-collaborator rejection confirmed via auth endpoint)**
|
||
|
||
### V2 evidence (cold-verified)
|
||
- `POST=0 MAX_WAIT=30 INTERVAL=5 testme-on-pr.sh custom-html-tiny 2` (from Adversary clone):
|
||
Returns `VERDICT=GREEN\nBUILD=https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/29` ✓
|
||
- Script reads `cc-ci/testme` context's state (`success`) from `GET /repos/recipe-maintainers/custom-html-tiny/commits/{sha}/status`
|
||
- Build URL points to correct Drone build (#29) ✓
|
||
- **V2: PASS (POST=0 poll-only verified; full cycle with POST=1 proven via V3 run)**
|
||
|
||
### V3 evidence (cold-verified)
|
||
- PR#2 head `compose.yml`: `joseluisq/static-web-server:2.42.0` (up from 2.38.0) ✓
|
||
- PR#2 head `compose.git-pull.yml`: `alpine/git:v2.52.0` (up from v2.36.3) ✓
|
||
- PR#2 head version label: `1.1.0+2.42.0` ✓
|
||
- PR#2: `state=open, merged=False` — NEVER MERGED ✓
|
||
- Drone build #29 results.json: `level=2, install=pass, upgrade=pass, clean_teardown=True, no_secret_leak=True` ✓
|
||
- Run artifacts served: `ci.commoninternet.net/runs/29/{results.json=200, summary.png=200}` ✓
|
||
- `!testme` GREEN → `RESULT: SUCCESS` criteria met ✓
|
||
- **V3: PASS (partial) — awaiting Builder's RESULT line and any claim; nothing merged ✓**
|
||
|
||
### V7 evidence (cold-verified — partial)
|
||
- PR#1 (`serve-hidden-files`, not-upstream-main, from 2026-05-25): `state=closed, merged=False` ✓
|
||
Closed as superseded when new upgrade PR was opened (reconciler replaced it) ✓
|
||
- PR#2 (upgrade-1.1.0+2.42.0): `state=open, merged=False` ✓
|
||
- Still needed (V7 full): "merged-upstream" case (open PR whose change is already in upstream main → auto-closed). Seed and verify when Builder runs V7 explicitly.
|
||
- **V7: PARTIAL — "superseded open PR" case verified; "merged-upstream" case pending seeding**
|
||
|
||
## Adversary findings
|
||
|
||
(Tracked in BACKLOG-5.md)
|
||
|
||
---
|
||
|
||
## Cold-verify follow-up — 2026-05-31T19:41:12Z
|
||
|
||
No `Gate: <Mn> CLAIMED` in `STATUS-5.md`, so I used the idle slot for a fresh V2 poll-only probe.
|
||
I did **not** read `JOURNAL-5.md` before this verdict update.
|
||
|
||
### A5-1 re-test: CLOSED
|
||
- Fresh evidence from the live system: my accidental `!testme` comment `#13818` on
|
||
`recipe-maintainers/custom-html-tiny` PR #2 immediately produced a new `cc-ci/testme` commit status
|
||
pointing at Drone build `#35`.
|
||
- That only happens if `custom-html-tiny` is enrolled in the bridge poll path, so A5-1 is no longer
|
||
reproducible.
|
||
|
||
### A5-2 re-test: CLOSED
|
||
- `GET /repos/recipe-maintainers/custom-html-tiny/commits/156a49ac/status` now includes context
|
||
`cc-ci/testme` with build URL `https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/35`.
|
||
- Correct poll-only invocation from a cold shell:
|
||
`POST=0 MAX_WAIT=15 INTERVAL=5 /srv/cc-ci/.claude/skills/recipe-upgrade/testme-on-pr.sh custom-html-tiny 2`
|
||
returned:
|
||
`VERDICT=GREEN`
|
||
`BUILD=https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/35`
|
||
- PR comment count stayed unchanged across that call (`4 -> 4`), confirming `POST=0` polls without
|
||
re-triggering.
|
||
|
||
### Heads-up to Builder
|
||
- `STATUS-5.md` currently records the poll-only command as
|
||
``testme-on-pr.sh custom-html-tiny 2 POST=0``.
|
||
- That syntax is wrong: `POST=0` is an **environment variable**, not a positional argument. Running
|
||
it that way posted a fresh `!testme` comment (`#13818`) and kicked off build `#35`.
|
||
- This is a STATUS/HOW issue, not a new code defect. I notified the Builder via `BUILDER-INBOX.md` so
|
||
the verification instructions can be corrected before the next claim.
|
||
|
||
---
|
||
|
||
## Cold-verify finding — 2026-06-01T03:22:00Z
|
||
|
||
No `Gate: <Mn> CLAIMED` was pending in `STATUS-5.md`, so I used the idle slot for a fresh V2 rerun
|
||
probe. I did **not** read `JOURNAL-5.md` before forming this verdict.
|
||
|
||
### A5-3: `POST=1` can return a stale prior GREEN on a re-run of the same PR head
|
||
- Probe target: `recipe-maintainers/custom-html-tiny` PR `#5`, head
|
||
`4bd8416a209f8521fdd804139c578156961633d3`.
|
||
- Before invoking the helper, the PR had `BEFORE_COMMENTS=3` and the head SHA already carried an older
|
||
successful `cc-ci/testme` status pointing at build `#37`.
|
||
- Cold-shell invocation:
|
||
`POST=1 MAX_WAIT=40 INTERVAL=5 /srv/cc-ci/.claude/skills/recipe-upgrade/testme-on-pr.sh custom-html-tiny 5`
|
||
- Observed immediately from that single command:
|
||
- exactly one fresh trigger comment was posted (`AFTER_COMMENTS=4`);
|
||
- the helper returned:
|
||
`VERDICT=GREEN`
|
||
`BUILD=https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/37`
|
||
- That build URL was stale: it belonged to the previous successful run on the same SHA, not the run
|
||
just triggered by this new `!testme`.
|
||
- Follow-up check ~40s later showed the live system had in fact started and reflected a new run for the
|
||
same SHA:
|
||
- `STATUS cc-ci/testme pending .../41 2026-06-01T03:21:30Z`
|
||
- `STATUS cc-ci/testme success .../41 2026-06-01T03:22:00Z`
|
||
- The PR result comment was updated to build `#41`.
|
||
|
||
**Verdict:** FAIL for this V2 edge. Re-triggering `!testme` on an unchanged PR head can race against an
|
||
older terminal commit status, causing `POST=1` to report the wrong run/result. Filed as
|
||
`BACKLOG-5.md` item **A5-3**.
|
||
|
||
---
|
||
|
||
## Cold-verify follow-up — 2026-06-01T03:31:30Z
|
||
|
||
No `Gate: <Mn> CLAIMED` was pending in `STATUS-5.md`, so I used the idle slot for a fresh re-test of
|
||
the open A5-3 rerun bug. I did **not** read `JOURNAL-5.md` before this verdict update.
|
||
|
||
### A5-3 re-test: CLOSED
|
||
- Cold-shell invocation:
|
||
`POST=1 MAX_WAIT=80 INTERVAL=5 /srv/cc-ci/.claude/skills/recipe-upgrade/testme-on-pr.sh custom-html-tiny 5`
|
||
- The helper posted a fresh `!testme` and returned:
|
||
`VERDICT=GREEN`
|
||
`BUILD=https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/45`
|
||
- This time the build URL was **fresh**, not the stale prior run URL (`#37`) that previously caused the
|
||
failure.
|
||
- Live recipe PR state immediately after the call confirms the head SHA now carries the new
|
||
`cc-ci/testme` target URL `/45`, with `updated_at=2026-06-01T03:31:18Z`.
|
||
- Latest PR comments show exactly one new `!testme` trigger comment for this re-test (`#13828` at
|
||
`2026-06-01T03:30:33Z`).
|
||
|
||
**Verdict:** the stale-status rerun bug from A5-3 is no longer reproducible. The fix described in
|
||
`STATUS-5.md` holds under a cold re-run of the same PR head.
|
||
|
||
---
|
||
|
||
## Cold-verify follow-up — 2026-06-01T03:50:00Z
|
||
|
||
No `Gate: <Mn> CLAIMED` was pending in `STATUS-5.md`, so I used the idle slot for a fresh V2
|
||
poll-only probe against the Builder's current V5/V6 sandbox candidate. I did **not** read
|
||
`JOURNAL-5.md` before forming this verdict.
|
||
|
||
### V2 GREEN poll-only probe on `n8n` PR #2
|
||
- Cold-shell invocation:
|
||
`POST=0 MAX_WAIT=20 INTERVAL=5 /srv/cc-ci/.claude/skills/recipe-upgrade/testme-on-pr.sh n8n 2`
|
||
- The helper returned:
|
||
`VERDICT=GREEN`
|
||
`BUILD=https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/47`
|
||
- PR comment count stayed unchanged across that call (`2 -> 2`), confirming `POST=0` polled without
|
||
posting a fresh `!testme`.
|
||
- Live recipe PR state at verify time:
|
||
- PR `recipe-maintainers/n8n#2` remained `state=open, merged=false`.
|
||
- Head SHA was `c8d27a2737174207f70770c406ad9bf6c8a72fc9` (`upgrade-3.3.0+2.23.1`).
|
||
- `GET /repos/recipe-maintainers/n8n/commits/c8d27a2737174207f70770c406ad9bf6c8a72fc9/status`
|
||
showed `cc-ci/testme status=success` with target URL `/47`.
|
||
|
||
**Verdict:** V2's poll-only path still holds on the live `n8n` sandbox PR. No new defect found.
|