From 7c77aec7abfba0cfe7c4bb59e07d962feda8340a Mon Sep 17 00:00:00 2001 From: autonomic-bot Date: Wed, 27 May 2026 04:01:21 +0100 Subject: [PATCH] Add Phase-3 plan: beautiful YunoHost-style results (levels + image comment + dashboard) Phase 3 (after Phase-2 DONE, manual transition): compute a per-run quality LEVEL, post an image-forward Gitea PR comment in the YunoHost shape (marker + status/level badge + a rendered summary card containing a real app screenshot, linking to the run), and polish the overview dashboard to a ci-apps.yunohost.org look/feel with per-recipe level badges + screenshots. Reuses the Phase-1 dashboard/bridge/Playwright; presentation never changes the verdict; no secrets in any artifact; cosmetics never block the pipeline. Linked from README. Co-Authored-By: Claude Opus 4.7 (1M context) --- cc-ci-plan/README.md | 3 +- cc-ci-plan/plan-phase3-results-ux.md | 176 +++++++++++++++++++++++++++ 2 files changed, 178 insertions(+), 1 deletion(-) create mode 100644 cc-ci-plan/plan-phase3-results-ux.md diff --git a/cc-ci-plan/README.md b/cc-ci-plan/README.md index f30eae3..bee897c 100644 --- a/cc-ci-plan/README.md +++ b/cc-ci-plan/README.md @@ -15,7 +15,8 @@ autonomous Claude loops (a Builder and an adversarial Reviewer) running over day | File | Purpose | |---|---| | `plan.md` | The Phase-1 plan (build the CI server). Agents treat it as their single source of truth. | -| `plan-phase2-recipe-tests.md` | **Phase 2** (starts after Phase-1 `## DONE`): author comprehensive per-recipe tests — port every recipe-maintainer test + ≥2 recipe-specific tests per app. | +| `plan-phase2-recipe-tests.md` | **Phase 2** (after Phase-1 `## DONE`): author comprehensive per-recipe tests — port every recipe-maintainer test + ≥2 recipe-specific tests per app. | +| `plan-phase3-results-ux.md` | **Phase 3** (after Phase-2 `## DONE`): beautiful YunoHost-style results — per-run **level**, image-forward PR comment (badge + summary card + app screenshot), polished dashboard. | | `IDEAS.md` | Deferred/future ideas, parked out of current scope. | | `brief.md` | The original one-page brief (context only; `plan.md` supersedes it). | | `kickoff.md` | Launch & supervision guide. | diff --git a/cc-ci-plan/plan-phase3-results-ux.md b/cc-ci-plan/plan-phase3-results-ux.md new file mode 100644 index 0000000..9fcc518 --- /dev/null +++ b/cc-ci-plan/plan-phase3-results-ux.md @@ -0,0 +1,176 @@ +# cc-ci Phase 3 — Beautiful YunoHost-style results (Autonomous Build Plan) + +**Status:** QUEUED — starts only after Phase 2 (`plan-phase2-recipe-tests.md`) reaches `## DONE`. +**Transition:** **manual** (operator kicks it off; check in / test between phases). +**Builds on:** Phase 1 (`plan.md` — dashboard `dashboard/`, the `!testme` bridge's PR comment, +the runner, Playwright in the harness) and Phase 2 (the rich per-recipe test taxonomy → meaningful +levels). +**Reference style:** the YunoHost app-CI result comment, e.g. +`https://github.com/YunoHost-Apps/lichenmarkdown_ynh/pull/20#issuecomment-3543928229` — see §3. +**Owner agents:** same Builder + Adversary loops + protocol as Phase 1 (`plan.md` §6/§7). +**This file's path:** `/srv/cc-ci/cc-ci-plan/plan-phase3-results-ux.md` + +--- + +## 0. Relationship to earlier phases + +Phase 1 gave us a *functional* results surface (Drone's per-run logs, a basic overview dashboard, and +a PR comment with run URL + pass/fail — D7). Phase 2 gave us a *rich, layered* test taxonomy per +recipe (install / upgrade / backup-data-integrity / recipe-specific / SSO-integration / recipe-local). + +Phase 3 makes the results **beautiful and YunoHost-style**: a computed **level** per run, an +**image-forward PR comment** (status badge + a rendered summary card with an app screenshot), and a +**polished overview dashboard** comparable to `ci-apps.yunohost.org`. It is presentation + +level-scoring on top of existing data — it must **not** change what the tests assert. + +Do not start until Phase 2 `STATUS.md` shows `## DONE` (Adversary-verified). Same loop protocol. + +--- + +## 1. Mission + +Turn cc-ci results into something a maintainer is happy to see on a PR and on a status page: +- a **level** (0–N) summarizing how far up the quality ladder the recipe got, +- an **image-forward PR comment** like YunoHost's (🌻 + status badge + summary image that includes a + screenshot of the actually-deployed app), linking to the full run, +- a **dashboard** that looks and feels like the YunoHost app list (per-recipe level badges, latest + status, screenshots, history). + +--- + +## 2. Definition of Done (Phase 3 exit condition) + +Terminates only when every item holds **and the Adversary has independently re-verified each within +24h** (logged in `REVIEW.md`): + +- [ ] **R1 — Level ladder.** A documented level ladder (§4.1) maps which test sets passed → a single + integer **level**, computed per run. Missing a lower rung caps the level (YunoHost semantics). +- [ ] **R2 — Image-forward PR comment.** On a `!testme` run, the bridge posts/updates a Gitea PR + comment in the YunoHost shape: a marker (🌻), a **status/level badge**, and a **summary image**, + both linking to the full run/dashboard. Re-running updates the same comment. +- [ ] **R3 — Summary card image.** Each run renders a PNG summary card showing: recipe + version, + the **level**, a per-stage/per-test **✔/✘ breakdown**, and an embedded **screenshot of the + deployed app**. Served at a stable URL; embedded in the comment and the dashboard. +- [ ] **R4 — App screenshot.** The runner captures a real screenshot of the deployed app (Playwright, + reusing the Phase-1 harness) — post-login where the landing page requires it — for the card. +- [ ] **R5 — Dashboard polish.** The overview at `ci.commoninternet.net` looks/feels like + `ci-apps.yunohost.org`: a table/grid of recipes with **level badge**, latest pass/fail, last + tested version, app screenshot/thumbnail, and a link to history. Regenerated on completion. +- [ ] **R6 — Badges.** A per-recipe **level/status badge** endpoint (SVG) embeddable in recipe + READMEs and the dashboard. +- [ ] **R7 — Safe & robust.** No secrets in images, comments, badges, or screenshots (reuse Phase-1 + §4.4 redaction; the screenshot step must not capture secret values — e.g. don't shoot pages + showing generated admin passwords). Image/screenshot generation **never blocks or fails the + pipeline**: on error it falls back to a text comment + records the failure, and the test verdict + is unaffected. +- [ ] **R8 — Docs.** `docs/` explains the level ladder, how the card/screenshot/badge are generated, + and how to embed a badge. + +When R1–R8 hold and are Adversary-verified, write `## DONE` to Phase-3 `STATUS.md`. + +--- + +## 3. Reference: the YunoHost comment style + +The linked YunoHost CI comment is deliberately **minimal and visual** (verified by fetching it): +- A header marker (🌻). +- A **shield-style test badge** linking to the CI job (`ci-apps.yunohost.org/ci/job/`). +- A **summary image (PNG)** — a rendered card with the result/level — also linking to the job. +- **No verbose inline table**; the per-test breakdown + level live *inside the rendered image* and on + the dashboard. Users click through for full logs. + +Mirror this shape for cc-ci (Gitea renders markdown images in comments): marker + badge + summary +PNG, both linking to the cc-ci run/dashboard. YunoHost also shows a **screenshot of the app** — we do +the same in the card. + +--- + +## 4. Design + +### 4.1 The level ladder (proposed default — finalize in `DECISIONS.md`) +A single integer; each rung requires all lower rungs (a gap caps the level, like YunoHost): + +- **L0** — install failed / app never became healthy. +- **L1 — Installs:** deploys and passes health/readiness. +- **L2 — Upgrades:** previous published version → PR version, stays healthy, data intact. +- **L3 — Backup/restore:** seeded data survives backup → wipe → restore (real data-integrity, P4). +- **L4 — Functional:** the recipe-specific functional tests pass (Phase-2 parity + ≥2 specific). +- **L5 — Integration:** SSO/OIDC and cross-app integration tests pass (for recipes that have them; + recipes with no integration surface cap at L4 by definition — record this so the level is fair). +- **L6 — Recipe-local:** the recipe repo's own `tests/` (D4) pass and are merged. + +(Also surface, as badges/flags rather than levels: clean-teardown ✔, no-secret-leak ✔ — these are +gating invariants from Phase 1, shown but not part of the climb.) + +### 4.2 Data flow +``` +run_recipe_ci.py emits a structured results.json per run + { recipe, version, pr, stages:[{name,status,tests:[{name,status,ms}]}], level, screenshot.png } + │ + ├─► summary-card renderer: HTML template (recipe, level badge, ✔/✘ table, app screenshot) + │ → render to PNG (Playwright screenshot of the HTML, reusing the harness browser) + │ → publish at ci.commoninternet.net/runs//summary.png (+ badge.svg) + │ + ├─► bridge updates the Gitea PR comment: 🌻 + [badge] + [summary.png], linking to the run + │ + └─► dashboard generator: overview grid (per-recipe level badge, screenshot, last status, + version, history) regenerated on build-completion → ci.commoninternet.net +``` +- **Summary image:** render an HTML results card → PNG via Playwright (already in the harness — no + new heavy dep). Keep a deterministic template; embed the app screenshot. +- **App screenshot:** Playwright navigates the live `-pr-.ci.commoninternet.net` + (logging in via the test user where needed) and screenshots the main view — captured during the + run while the app is up, before teardown. +- **Badges:** generate SVG (shields-style) per run + a per-recipe latest-level badge endpoint. +- **Hosting:** the dashboard service (Phase-1 `dashboard/`) serves `/runs//...` and `/badge/...`; + Gitea comments embed them by URL. + +### 4.3 PR comment (YunoHost-shaped) +On run start: a placeholder comment ("⏳ testing … level pending", link to live logs). On completion: +update the same comment to 🌻 + level/status **badge** + **summary card image**, linking to the run +and the dashboard. One comment per PR, updated in place; re-`!testme` refreshes it. + +--- + +## 5. Milestones (each ends with an Adversary gate) + +- **U0 — Results schema + level.** `run_recipe_ci.py` emits `results.json` (per-stage/per-test) and + computes the level (§4.1). *Accept:* level is correct for a recipe that passes through L4 and one + that fails at L2 (capped). +- **U1 — App screenshot.** Harness captures a real screenshot of the deployed app (post-login where + needed), secret-safe. *Accept:* screenshot of a sample recipe shows the working UI, no secrets. +- **U2 — Summary card + badge.** Render the HTML card → PNG (level, ✔/✘ table, screenshot) + SVG + badge, served at stable URLs. *Accept:* card + badge render correctly for pass and fail runs. +- **U3 — YunoHost-style PR comment.** Bridge posts/updates the image-forward comment (marker + badge + + card, linked). *Accept:* live on a scratch PR — comment shows badge + card + screenshot, updates + on re-run, contains no secrets. +- **U4 — Dashboard polish.** Overview grid with per-recipe level badges, screenshots, status, version, + history — comparable look/feel to `ci-apps.yunohost.org`. *Accept:* matches reality across several + runs; Adversary confirms it mirrors the underlying results. +- **U5 — Badges + docs + hardening.** Embeddable per-recipe badges; docs for the ladder + embedding; + fallback-to-text on render failure; secret-scan over images/screenshots/comments. *Accept:* + Adversary's leak scan over published images/comments finds nothing; killing the renderer degrades + gracefully to text without affecting the verdict; flip Phase-3 `STATUS.md` to `## DONE`. + +--- + +## 6. Guardrails (inherit Phase 1 §9 + Phase 2 §7.1) + +- **Presentation never changes the verdict.** The level and card *report* test outcomes; they must + not let a run look greener than its tests. The Adversary checks the rendered level/card against the + raw `results.json` and the actual test outcomes — a card that overstates the result is a FAIL. +- **No secrets in any artifact** (R7) — comments, badges, summary cards, app screenshots. The + screenshot step must avoid pages that display generated credentials. +- **Never block the pipeline on cosmetics** — image/screenshot/badge generation failures degrade to a + text comment and a recorded warning; they never fail or hang a test run (respect Phase-1 timeouts). +- **Don't weaken tests to raise a level** (carry-over of the cardinal rule) — the Adversary watches + for tests softened or levels mis-mapped to inflate the displayed quality. + +--- + +## 7. Open decisions (log in DECISIONS.md) +- Exact level ladder + how recipes without an integration/SSO surface are scored fairly (cap vs N/A). +- Summary-card rendering: HTML→Playwright-PNG (default, reuses the harness) vs a dedicated image lib. +- Where app screenshots are hosted/retained and for how long (retention/cleanup, like run logs). +- Badge implementation: self-rendered SVG vs a shields.io endpoint pattern. +- Whether to also post a compact markdown fallback table beneath the image for accessibility.