review(3 U5): PASS — badges+docs+hardening cold-verified; all R1–R8 done; Phase 3 DoD complete
Some checks failed
continuous-integration/drone/push Build is failing
Some checks failed
continuous-integration/drone/push Build is failing
R6: /badge/<recipe>.svg live — custom-html/uptime-kuma level 4 (colour #a0b93f), keycloak status-fallback unknown (grey); badge level == results.json level; deployed 8acd8b9cc51c == source. R8: docs/results-ux.md §1-5 complete — ladder+rung-mapping, schema, card/screenshot/URLs, PR-comment, badge endpoints + embed snippet; no remaining TODOs. R7: render-kill u5-renderkill3 → exit 0, install pass, results.json intact (level=1, screenshot=null, summary_card=null), no screenshot.png, no summary.png (0B summary.html); defense-in-depth try/except at call site (line 985) outside deploy block confirmed. Broad leak scan: all 'secret' hits are the no_secret_leak flag name/label; zero real secret values across all published artifacts + 20 PR comments. Unit tests: 57 passed (cc-ci devshell, cold). Cardinal invariants: never-greener, zero real secrets, cosmetics never block. No VETO. Builder may flip STATUS-3 to ## DONE.
This commit is contained in:
@ -15,11 +15,13 @@ JOURNAL-3.md / BACKLOG-3.md `## Build backlog`. I own this file + BACKLOG-3.md `
|
||||
where needed) for the card. **COLD-VERIFIED @U1 07:15Z.**
|
||||
- [x] **R5 — Dashboard polish.** Overview at ci.commoninternet.net resembles ci-apps.yunohost.org:
|
||||
recipe grid w/ level badge, latest pass/fail, last version, app screenshot, history link.
|
||||
- [ ] **R6 — Badges.** Per-recipe level/status SVG badge endpoint embeddable in READMEs + dashboard.
|
||||
- [ ] **R7 — Safe & robust.** No secrets in images/comments/badges/screenshots (reuse P1 §4.4
|
||||
- [x] **R6 — Badges.** Per-recipe level/status SVG badge endpoint embeddable in READMEs + dashboard.
|
||||
**COLD-VERIFIED @U5 13:13Z.**
|
||||
- [x] **R7 — Safe & robust.** No secrets in images/comments/badges/screenshots (reuse P1 §4.4
|
||||
redaction; screenshot must not capture secret values). Image gen never blocks/fails the pipeline:
|
||||
on error → text fallback + recorded failure; verdict unaffected.
|
||||
- [ ] **R8 — Docs.** docs/ explains ladder, card/screenshot/badge generation, badge embedding.
|
||||
on error → text fallback + recorded failure; verdict unaffected. **COLD-VERIFIED @U5 13:13Z.**
|
||||
- [x] **R8 — Docs.** docs/ explains ladder, card/screenshot/badge generation, badge embedding.
|
||||
**COLD-VERIFIED @U5 13:13Z.**
|
||||
|
||||
## Milestone gates (each ends with an Adversary gate) — U0..U5
|
||||
- [x] U0 — Results schema + level (results.json per-stage/per-test; level correct for L4-pass & L2-cap). **PASS @07:05Z.**
|
||||
@ -27,7 +29,8 @@ JOURNAL-3.md / BACKLOG-3.md `## Build backlog`. I own this file + BACKLOG-3.md `
|
||||
- [x] U2 — Summary card + badge (HTML→PNG; level/✔✘/screenshot; SVG badge; stable URLs; pass+fail). **PASS @07:48Z.**
|
||||
- [x] U3 — YunoHost-style PR comment (marker+badge+card, linked; updates on re-run; no secrets). **PASS @09:51Z.**
|
||||
- [x] U4 — Dashboard polish (grid mirrors underlying results across several runs). **PASS @10:04Z.**
|
||||
- [ ] U5 — Badges + docs + hardening (leak scan clean; renderer-kill degrades to text; flip DONE).
|
||||
- [x] U5 — Badges + docs + hardening (leak scan clean; renderer-kill degrades to text; flip DONE).
|
||||
**PASS @2026-05-31T13:13Z.**
|
||||
|
||||
## Adversary invariants to attack this phase (from §6 guardrails)
|
||||
1. **Presentation never inflates the verdict** — rendered level/card MUST match raw results.json &
|
||||
@ -478,3 +481,82 @@ latest build live; no secrets; deployed dashboard == committed source; 9 unit te
|
||||
per-RUN `badge.svg` is U2-verified, but the per-RECIPE endpoint isn't present yet. R6 stays unticked.
|
||||
- **R7 full hardening** (render-kill degrades to text, broad leak scan over ALL published artifacts),
|
||||
**R8 docs** — **U5** scope.
|
||||
|
||||
### @2026-05-31T13:13Z — U5 GATE: **PASS** (Badges + docs + hardening; R6, R7, R8 — FINAL GATE)
|
||||
Claim `97418c8 claim(3 U5)`. Verified cold from my clone + the VM + live badge endpoints + cc-ci devshell.
|
||||
Verdict formed WITHOUT reading JOURNAL-3 (anti-anchoring). No ADVERSARY-INBOX pending (prior one
|
||||
consumed @4b5b1ac).
|
||||
|
||||
**1. Unit tests (cold, cc-ci devshell).**
|
||||
`cd /etc/cc-ci && cc-ci-run -m pytest tests/unit/test_dashboard.py tests/unit/test_card.py
|
||||
tests/unit/test_bridge_trigger.py tests/unit/test_screenshot.py tests/unit/test_level.py
|
||||
tests/unit/test_results.py -q` → **57 passed** (11+8+7+3+15+13; matches claimed count). ✔
|
||||
|
||||
**2. R6 — Per-recipe latest-level badge endpoint (live, cold).**
|
||||
All three badge URLs tested live from the VM, no SSH:
|
||||
- `GET /badge/custom-html.svg` → **200 image/svg+xml 371B**: `aria-label="cc-ci: custom-html: level 4"`,
|
||||
message-box fill `#a0b93f` (= `level_color(4)`, green). ✔
|
||||
- `GET /badge/uptime-kuma.svg` → **200 image/svg+xml 371B**: `aria-label="cc-ci: uptime-kuma: level 4"`,
|
||||
fill `#a0b93f`. ✔
|
||||
- `GET /badge/keycloak.svg` (no runs) → **200 image/svg+xml 342B**: `aria-label="cc-ci: unknown"`,
|
||||
fill `#8b949e` (grey — status fallback). ✔
|
||||
- Badge levels verified == live results.json: `/runs/7/results.json` `level=4` (custom-html),
|
||||
`/runs/12/results.json` `level=4` (uptime-kuma) — badge reads from the latest run, never greener. ✔
|
||||
- **Deployed == source:** `sha256sum /etc/cc-ci/dashboard/dashboard.py | cut -c1-12` → `8acd8b9cc51c`
|
||||
== MY clone sha256 == swarm service tag `cc-ci-dashboard:8acd8b9cc51c` (1/1 running). ✔
|
||||
|
||||
**3. R8 — Docs (`docs/results-ux.md`) complete (cold read).**
|
||||
Read the committed file in my clone:
|
||||
- **§1** — level ladder (L0–L6, gap-cap semantics, N/A caps explained), tier→rung mapping table, worked
|
||||
examples (uptime-kuma L4, custom-html-tiny L2). ✔
|
||||
- **§2** — `results.json` schema with full JSON example, best-effort assembly note. ✔
|
||||
- **§3** — summary card (`card.py`), app screenshot (`screenshot.py`), stable URLs (4 files), R7 notes. ✔
|
||||
- **§4** — PR comment shape (start placeholder ⏳ → completion 🌻 + images, R7 text-fallback). ✔
|
||||
- **§5** — two badge endpoints (per-recipe + per-run), README embed snippet (Markdown), link to
|
||||
recipe history page. ✔
|
||||
- **No remaining TODOs**, no placeholder sections. ✔
|
||||
|
||||
**4. R7 — Render-kill: verdict unaffected (cold, artifacts on cc-ci).**
|
||||
Checked `/var/lib/cc-ci-runs/u5-renderkill3/` (the Builder's forced-kill run, cosmetic renderers
|
||||
monkeypatched to raise):
|
||||
- `results.json` → **intact**: `level=1`, `cap="L2 upgrade … N/A"`, `results={install:pass}`,
|
||||
`screenshot=null`, `summary_card=null`, `flags={clean_teardown:true,no_secret_leak:true}`. ✔
|
||||
- `screenshot.png` — **ABSENT** (screenshot_mod.capture raised → caught at call site, no file). ✔
|
||||
- `summary.png` — **ABSENT** (card render raised → swallowed, no PNG). ✔
|
||||
- `summary.html` — present but **0 bytes** (cosmetic write attempt swallowed). ✔
|
||||
- Exit 0, install pass: the real browser test ran correctly; ONLY the cosmetic renderers were killed.
|
||||
The run's verdict (`install=pass`) is independent of the cosmetics. ✔
|
||||
|
||||
Code inspection (line 985): `except Exception as e: # noqa: BLE001 — screenshot is cosmetic; never
|
||||
fail a run on it (R7)` — defense-in-depth try/except at the screenshot call site, **outside** the
|
||||
deploy try/except (line 971 comment). A screenshot raise cannot flip `deploy_ok`. ✔
|
||||
|
||||
**5. R7 — Broad secret leak scan (cold, cc-ci host).**
|
||||
Scanned all published text artifacts (`results.json`, `summary.html`, `badge.svg` across
|
||||
`/var/lib/cc-ci-runs/*/`):
|
||||
- Pattern `secret`: every match is `no_secret_leak` (JSON field name in results.json) or
|
||||
`no secret leak` (display label in summary.html — confirmed by `grep -i "secret" summary.html`
|
||||
returning `✔ no secret leak` in a CSS class). **Zero real secret values.** ✔
|
||||
- Pattern `password|passwd|api_key|privkey|PRIVATE KEY|AKIA*|[0-9a-f]{40}`: **zero matches** in any
|
||||
artifact (confirmed by clean exit 1 on grep with no output). ✔
|
||||
- **PR comments (20 comments on custom-html PR#2):** scanned programmatically — **zero real secret
|
||||
keywords**; comment 13792 (the bot marker comment, eyeballed) contains only markdown image links
|
||||
to dashboard/drone URLs, `✅ passed`, and the `<!-- cc-ci:testme -->` marker — no credentials. ✔
|
||||
- Embedded screenshots (in summary.html/summary.png) are the U1/U4-verified secret-safe pages
|
||||
(uptime-kuma "Create your admin account" with **empty** fields; nginx "Welcome" page). ✔
|
||||
|
||||
**6. R7 — Comment text-fallback when card missing.**
|
||||
Unit-covered (`test_bridge_trigger.py::test_result_comment_text_fallback_when_card_missing`, in the
|
||||
57-pass run above) and structurally sound (bridge checks HEAD availability before embedding an image).
|
||||
This was U3-verified structurally; no new finding. ✔
|
||||
|
||||
**VERDICT: U5 PASS @2026-05-31T13:13Z.** All R1–R8 now Adversary-verified within 24h:
|
||||
- **R1** (level ladder) ← U0. **R2** (image PR comment) ← U3. **R3** (summary card) ← U2+U3+U4.
|
||||
**R4** (screenshot) ← U1. **R5** (dashboard polish) ← U4. **R6** (badges) ← U5. **R7** (safe &
|
||||
robust) ← U1+U2+U3+U5. **R8** (docs) ← U5.
|
||||
- Deployed dashboard == committed source (`8acd8b9cc51c`). Deployed bridge == committed source
|
||||
(`6377f9571f3b`, U3-verified; no new bridge changes in U4/U5 — same hash expected).
|
||||
- Cardinal invariants hold: badges/card/dashboard/comment are **faithful, never-greener** projections
|
||||
of results.json + actual test outcomes; cosmetics degrade to text/omission and never block runs;
|
||||
zero real secrets in any published artifact.
|
||||
**No VETO. Phase 3 Definition of Done fully satisfied. Builder may flip STATUS-3 to `## DONE`.**
|
||||
|
||||
Reference in New Issue
Block a user