diff --git a/machine-docs/REVIEW-3.md b/machine-docs/REVIEW-3.md index 2a477ad..e0209ca 100644 --- a/machine-docs/REVIEW-3.md +++ b/machine-docs/REVIEW-3.md @@ -15,11 +15,13 @@ JOURNAL-3.md / BACKLOG-3.md `## Build backlog`. I own this file + BACKLOG-3.md ` where needed) for the card. **COLD-VERIFIED @U1 07:15Z.** - [x] **R5 — Dashboard polish.** Overview at ci.commoninternet.net resembles ci-apps.yunohost.org: recipe grid w/ level badge, latest pass/fail, last version, app screenshot, history link. -- [ ] **R6 — Badges.** Per-recipe level/status SVG badge endpoint embeddable in READMEs + dashboard. -- [ ] **R7 — Safe & robust.** No secrets in images/comments/badges/screenshots (reuse P1 §4.4 +- [x] **R6 — Badges.** Per-recipe level/status SVG badge endpoint embeddable in READMEs + dashboard. + **COLD-VERIFIED @U5 13:13Z.** +- [x] **R7 — Safe & robust.** No secrets in images/comments/badges/screenshots (reuse P1 §4.4 redaction; screenshot must not capture secret values). Image gen never blocks/fails the pipeline: - on error → text fallback + recorded failure; verdict unaffected. -- [ ] **R8 — Docs.** docs/ explains ladder, card/screenshot/badge generation, badge embedding. + on error → text fallback + recorded failure; verdict unaffected. **COLD-VERIFIED @U5 13:13Z.** +- [x] **R8 — Docs.** docs/ explains ladder, card/screenshot/badge generation, badge embedding. + **COLD-VERIFIED @U5 13:13Z.** ## Milestone gates (each ends with an Adversary gate) — U0..U5 - [x] U0 — Results schema + level (results.json per-stage/per-test; level correct for L4-pass & L2-cap). **PASS @07:05Z.** @@ -27,7 +29,8 @@ JOURNAL-3.md / BACKLOG-3.md `## Build backlog`. I own this file + BACKLOG-3.md ` - [x] U2 — Summary card + badge (HTML→PNG; level/✔✘/screenshot; SVG badge; stable URLs; pass+fail). **PASS @07:48Z.** - [x] U3 — YunoHost-style PR comment (marker+badge+card, linked; updates on re-run; no secrets). **PASS @09:51Z.** - [x] U4 — Dashboard polish (grid mirrors underlying results across several runs). **PASS @10:04Z.** -- [ ] U5 — Badges + docs + hardening (leak scan clean; renderer-kill degrades to text; flip DONE). +- [x] U5 — Badges + docs + hardening (leak scan clean; renderer-kill degrades to text; flip DONE). + **PASS @2026-05-31T13:13Z.** ## Adversary invariants to attack this phase (from §6 guardrails) 1. **Presentation never inflates the verdict** — rendered level/card MUST match raw results.json & @@ -478,3 +481,82 @@ latest build live; no secrets; deployed dashboard == committed source; 9 unit te per-RUN `badge.svg` is U2-verified, but the per-RECIPE endpoint isn't present yet. R6 stays unticked. - **R7 full hardening** (render-kill degrades to text, broad leak scan over ALL published artifacts), **R8 docs** — **U5** scope. + +### @2026-05-31T13:13Z — U5 GATE: **PASS** (Badges + docs + hardening; R6, R7, R8 — FINAL GATE) +Claim `97418c8 claim(3 U5)`. Verified cold from my clone + the VM + live badge endpoints + cc-ci devshell. +Verdict formed WITHOUT reading JOURNAL-3 (anti-anchoring). No ADVERSARY-INBOX pending (prior one +consumed @4b5b1ac). + +**1. Unit tests (cold, cc-ci devshell).** +`cd /etc/cc-ci && cc-ci-run -m pytest tests/unit/test_dashboard.py tests/unit/test_card.py +tests/unit/test_bridge_trigger.py tests/unit/test_screenshot.py tests/unit/test_level.py +tests/unit/test_results.py -q` → **57 passed** (11+8+7+3+15+13; matches claimed count). ✔ + +**2. R6 — Per-recipe latest-level badge endpoint (live, cold).** +All three badge URLs tested live from the VM, no SSH: +- `GET /badge/custom-html.svg` → **200 image/svg+xml 371B**: `aria-label="cc-ci: custom-html: level 4"`, + message-box fill `#a0b93f` (= `level_color(4)`, green). ✔ +- `GET /badge/uptime-kuma.svg` → **200 image/svg+xml 371B**: `aria-label="cc-ci: uptime-kuma: level 4"`, + fill `#a0b93f`. ✔ +- `GET /badge/keycloak.svg` (no runs) → **200 image/svg+xml 342B**: `aria-label="cc-ci: unknown"`, + fill `#8b949e` (grey — status fallback). ✔ +- Badge levels verified == live results.json: `/runs/7/results.json` `level=4` (custom-html), + `/runs/12/results.json` `level=4` (uptime-kuma) — badge reads from the latest run, never greener. ✔ +- **Deployed == source:** `sha256sum /etc/cc-ci/dashboard/dashboard.py | cut -c1-12` → `8acd8b9cc51c` + == MY clone sha256 == swarm service tag `cc-ci-dashboard:8acd8b9cc51c` (1/1 running). ✔ + +**3. R8 — Docs (`docs/results-ux.md`) complete (cold read).** +Read the committed file in my clone: +- **§1** — level ladder (L0–L6, gap-cap semantics, N/A caps explained), tier→rung mapping table, worked + examples (uptime-kuma L4, custom-html-tiny L2). ✔ +- **§2** — `results.json` schema with full JSON example, best-effort assembly note. ✔ +- **§3** — summary card (`card.py`), app screenshot (`screenshot.py`), stable URLs (4 files), R7 notes. ✔ +- **§4** — PR comment shape (start placeholder ⏳ → completion 🌻 + images, R7 text-fallback). ✔ +- **§5** — two badge endpoints (per-recipe + per-run), README embed snippet (Markdown), link to + recipe history page. ✔ +- **No remaining TODOs**, no placeholder sections. ✔ + +**4. R7 — Render-kill: verdict unaffected (cold, artifacts on cc-ci).** +Checked `/var/lib/cc-ci-runs/u5-renderkill3/` (the Builder's forced-kill run, cosmetic renderers +monkeypatched to raise): +- `results.json` → **intact**: `level=1`, `cap="L2 upgrade … N/A"`, `results={install:pass}`, + `screenshot=null`, `summary_card=null`, `flags={clean_teardown:true,no_secret_leak:true}`. ✔ +- `screenshot.png` — **ABSENT** (screenshot_mod.capture raised → caught at call site, no file). ✔ +- `summary.png` — **ABSENT** (card render raised → swallowed, no PNG). ✔ +- `summary.html` — present but **0 bytes** (cosmetic write attempt swallowed). ✔ +- Exit 0, install pass: the real browser test ran correctly; ONLY the cosmetic renderers were killed. + The run's verdict (`install=pass`) is independent of the cosmetics. ✔ + +Code inspection (line 985): `except Exception as e: # noqa: BLE001 — screenshot is cosmetic; never +fail a run on it (R7)` — defense-in-depth try/except at the screenshot call site, **outside** the +deploy try/except (line 971 comment). A screenshot raise cannot flip `deploy_ok`. ✔ + +**5. R7 — Broad secret leak scan (cold, cc-ci host).** +Scanned all published text artifacts (`results.json`, `summary.html`, `badge.svg` across +`/var/lib/cc-ci-runs/*/`): +- Pattern `secret`: every match is `no_secret_leak` (JSON field name in results.json) or + `no secret leak` (display label in summary.html — confirmed by `grep -i "secret" summary.html` + returning `✔ no secret leak` in a CSS class). **Zero real secret values.** ✔ +- Pattern `password|passwd|api_key|privkey|PRIVATE KEY|AKIA*|[0-9a-f]{40}`: **zero matches** in any + artifact (confirmed by clean exit 1 on grep with no output). ✔ +- **PR comments (20 comments on custom-html PR#2):** scanned programmatically — **zero real secret + keywords**; comment 13792 (the bot marker comment, eyeballed) contains only markdown image links + to dashboard/drone URLs, `✅ passed`, and the `` marker — no credentials. ✔ +- Embedded screenshots (in summary.html/summary.png) are the U1/U4-verified secret-safe pages + (uptime-kuma "Create your admin account" with **empty** fields; nginx "Welcome" page). ✔ + +**6. R7 — Comment text-fallback when card missing.** +Unit-covered (`test_bridge_trigger.py::test_result_comment_text_fallback_when_card_missing`, in the +57-pass run above) and structurally sound (bridge checks HEAD availability before embedding an image). +This was U3-verified structurally; no new finding. ✔ + +**VERDICT: U5 PASS @2026-05-31T13:13Z.** All R1–R8 now Adversary-verified within 24h: +- **R1** (level ladder) ← U0. **R2** (image PR comment) ← U3. **R3** (summary card) ← U2+U3+U4. + **R4** (screenshot) ← U1. **R5** (dashboard polish) ← U4. **R6** (badges) ← U5. **R7** (safe & + robust) ← U1+U2+U3+U5. **R8** (docs) ← U5. +- Deployed dashboard == committed source (`8acd8b9cc51c`). Deployed bridge == committed source + (`6377f9571f3b`, U3-verified; no new bridge changes in U4/U5 — same hash expected). +- Cardinal invariants hold: badges/card/dashboard/comment are **faithful, never-greener** projections + of results.json + actual test outcomes; cosmetics degrade to text/omission and never block runs; + zero real secrets in any published artifact. +**No VETO. Phase 3 Definition of Done fully satisfied. Builder may flip STATUS-3 to `## DONE`.**