Files
cc-ci-orchestrator/.claude/skills/recipe-report/SKILL.md
T

149 lines
12 KiB
Markdown

---
name: recipe-report
description: Generate the weekly public "Recipe Report" after the cc-ci /upgrade-all run. Reviews how the upgrade went and the state of every recipe + open PR, and publishes a self-contained HTML page to report.ci.commoninternet.net (one page per week + a home index). Page order: short lead → the full wire table (priority-sorted, CVE-count column, CVE recipes first) → Addendum of special issues → Security Bulletin → per-recipe "what changed". Read-only — reports, never merges or edits PRs. Runs as its own agent (model separately overridable via REPORT_BACKEND/REPORT_MODEL; defaults to the upgrader's backend+model, opencode-go/glm-5.2). Invoke as /recipe-report [YYYY-MM-DD].
---
# recipe-report
Produce the weekly **"The Recipe Report"** — a public HTML page reviewing the latest `/upgrade-all` run
and the live state of every recipe + open PR. Served at **https://report.ci.commoninternet.net** (one
`week-<date>.html` per run + a home-page index). You are **read-only**: you report, you never merge,
edit, or comment on PRs. The page is **public + unauthenticated** — include only public-safe data
(recipe names, version bumps, PR titles/links, CI verdicts, concise error summaries); **never** secrets,
tokens, internal IPs, or raw logs.
Helper: `python3 /srv/cc-ci/cc-ci-plan/recipe-report.py {survey|render|publish}` (see its header).
**Deterministic formatting — spec in, page out (NON-NEGOTIABLE).** ALL layout, CSS, and HTML is owned by
`recipe-report.py`'s `render()`. Your ONLY output artifact is the **spec JSON** (data + editorial text).
You **never** write, edit, post-process, or templatize HTML yourself; you never put markup/inline styles
in spec text fields (they are HTML-escaped on purpose); the page is produced **solely** by
`render <spec> <out>`. Same spec shape every week → a structurally identical page. If the layout itself
must change, edit `recipe-report.py` (and `example-spec.json`) — never the generated page. This is what
keeps every weekly edition looking the same regardless of which model writes the spec.
## Procedure
1. **Pick the date.** `DATE` = the argument, else today (UTC). The `/upgrade-all` summary for that run is
`/srv/cc-ci/.cc-ci-logs/upgrades/upgrade-all-<DATE>.md`.
2. **Survey.** `python3 /srv/cc-ci/cc-ci-plan/recipe-report.py survey <DATE> > /tmp/survey.json`. This
gives you: the raw `/upgrade-all` summary; for every recipe mirror its open PRs + the `cc-ci/testme`
verdict; and each recipe's **per-recipe upgrade notes** (`upgrade_notes_md` — the breaking-change /
**CVE / security** analysis the upgrade did). Read it all. Also run
`python3 /srv/cc-ci/cc-ci-plan/audit-sources.py --security-sources`; a recipe with no verified
security source or a failed source must be called out, not presented as a clean CVE scan.
3. **Review & classify.** The page order is: **short lead → the full wire table (priority-sorted) →
Addendum → Security Bulletin → per-recipe "What changed".** Same newspaper style; this order.
- **Security analysis.** Scan the per-recipe `upgrade_notes_md` + the summary (and use your own
knowledge of the version bumps) for upgrades that fix **CVEs / security issues**. For each recipe,
**count the CVEs** the PR fixes — this drives both the table's `cve` column and the priority sort.
Before crediting a CVE to an open PR, compare that PR's head `compose.yml` with the **current
target-branch tip**, both pinned by SHA at report time. Run
`advisory-scan.py <recipe> --compose-from <current-target-sha-raw-url>
--compose-to <head-sha-raw-url>` when an upgrade log lacks that exact scan, or when its listed
image windows disagree with the PR diff. Credit image CVEs only for image version changes in
that diff, and only if the PR head is the fixed version. An upgrade-time merge base may now be
stale if upstream advanced; a previous release or warm canonical can also include fixes already
present in the target branch. For example, lasuite-docs PR #11 did not change Valkey,
so its 8.1.11→9.1.3 CVE must not be credited to that PR. Record a security-relevant config
fix separately with direct evidence that the target branch lacks it. If the current diff cannot
be verified, use `?` and explain why. If the PR head is behind the target branch or unmergeable,
say so in the same row; its old CI result is not proof that the current combined recipe works.
- **ADDITIONALLY, and never instead:** each per-recipe log carries an `### Advisory scan
(deterministic pre-step)` block (from `cc-ci-plan/advisory-scan.py` — GitHub Security
Advisories + vendor security pages + OSV, with severities and fixed-in versions). Treat its
CVE list as a **further source** and report the **UNION** of it and what you found by reading.
Its entries are machine-derived with advisory IDs, so prefer them for CVE ids / severities /
fixed-in versions, and cite the GHSA where present in the Security Bulletin. If the block
lists **failed sources**, the count is **not** authoritative: render the cve cell as `?`
(unknown), never `none` — a blank that reads as "clean" is exactly how two CVSS-9.8 gitea
RCEs were reported as "none" on 2026-08-07.
- The table's **CVEs** column counts known CVEs fixed by the **current open PR's diff against
current target branch**, even if `/upgrade-all` found no new image version this week. Keep the
existing one-row-per-recipe table and show that PR in the row. `SKIPPED — up-to-date` describes
this week's image survey, not the absence of a relevant open PR; use the PR's actual CI verdict
in TESTS when one exists. A zero means the current PR fixes no verified CVEs (or there is no
PR); it does not certify that deployed images are vulnerability-free. If the PR's advisory
scan is absent, a source failed, or an image window was unmatched, use `?`.
If a confirmed security advisory is fixed but has no CVE ID, put `"YES"` in `cve` rather
than `0`; describe its advisory ID and severity in Notes. If CVE IDs are also fixed, show
the numeric CVE count and mention the additional non-CVE advisories in Notes.
- **`?` means the count is not verified.** Use it when an upgrade has no scan, a source failed,
an image window was unmatched, or the scan block says COUNT UNKNOWN. In that case publish
`?` and say why in
the notes; publishing `0` would assert a clean bill of health nothing supports. Note a
**version-scheme change is no longer a reason for `?`** — the scan resolves semver→calver jumps
(discourse 3.5.3 → 2026.7.1) by falling back to advisory publish dates and reports a real number.
- **A count with undetermined advisories is a FLOOR.** If the scan block says N advisories
"could NOT be judged", report the number but say in the notes that it is a floor — those
advisories are neither fixed nor safe, they are unmeasured. Never round them away.
- **Counts span every image, each judged by its own window.** A scan block lists one line per
image with its version range and classification method; the headline is their union. So a
recipe's count legitimately includes **sidecar** CVEs (discourse's 128 = 123 app + 5 redis).
When a sidecar contributes a critical/high, name the image in the bulletin — CVE-2025-49844 is
a redis flaw, not a discourse one, and an operator reading "discourse" needs to know that.
(The scan headline itself now says `UNKNOWN` rather than a number in that case.)
In particular: **no new image upgrade this run does not force `0`** if an open PR still has
an unmerged security change. Recheck its current diff and credit only fixes absent from the
current target branch. A recipe with no relevant PR reports `0`; a clean PR scan reports its
number (including `0`). Benign notes in a scan
block (`no-advisories-published`, `skipped: template URL`) are NOT failures and must not
trigger `?`. If you find yourself rendering `?` for many recipes, that is a bug to report in
the Addendum, not a normal outcome.
Anything **critical/high** also gets a `security` bulletin entry (recipe · CVE id(s) + severity ·
what it fixes · PR link); be specific about severity and what's exposed if not merged.
- **Lead — ONE short paragraph.** A tight, concrete opener in opus's voice: fleet state in a sentence
(how healthy, what moved) and what to address first. Lead with the single most important thing; the
table and sections carry all the detail. Do NOT write multiple paragraphs.
- **Priority sort for the table.** Order rows by **recommended priority to address**:
1. recipes **with CVEs** first (more/higher-severity CVEs higher),
2. then **FAILED** (RED) recipes,
3. then **STALE**-test recipes,
4. then routine **GREEN** bumps,
5. then **UPTODATE / SKIPPED** last.
- **Addendum — special issues to look into.** A bullet list of real anomalies worth an operator's
eye: a recipe carrying **two open PRs** to reconcile, a **CI-run oddity** (flaky tier, needed
re-runs), drift between the summary and reality, a **misleading branch name**, a tooling snag that
silently drops a recipe, leftover artifact PRs, or **something in the test run that could be
improved**. **Only real issues — if everything looks clean, leave it empty; do NOT invent items.**
- **What changed — per recipe with a relevant open PR.** Describe the PR's current diff (versions,
configuration, migrations/gotchas, reconcile notes), including an older PR still awaiting merge.
Say when it was last updated and whether it can merge cleanly. Skip recipes with no relevant PR.
4. **Write the spec** `/tmp/report-spec.json` (shape in the helper header):
- `date`, `subtitle` ("Week of <human date>"),
- `lead` — **one short paragraph**,
- `table[]` — EVERY recipe, **sorted by the priority above**. Fields: `recipe`, `change` "a → b",
`status` GREEN/FAILED/STALE/SKIPPED/UPTODATE, **`cve`** (integer count of CVEs this PR fixes; `0`
or omit for none), `ci` as a level/result number+info string e.g. `build 154 ✓` or `RED 200 ·
install` with `ci_url`, `pr`/`pr_url`, `notes`. **CI = link + number/info only; no images.**
- The table renders **two** status columns: **TESTS** (the `status` field — the CI/test verdict)
and **STATUS** (the PR's LIVE state, fetched client-side). You only supply `status`; the live
STATUS column is derived automatically from `recipe` + `pr` and needs no spec field — it shows
`open` vs a ✓ (any not-open state), refreshing ~30s via the same-origin `/pr/<recipe>/<n>` proxy
(cc-ci `nix/modules/reports.nix`); it degrades to a muted "?" if the proxy/repo is unreachable.
- `addendum[]` — list of bullet strings (may be empty),
- `security[]` — critical-CVE bulletin entries (may be empty),
- `changes[]` — `{recipe, body, links[]}`, one per recipe that has a PR.
**Template — match this.** `.claude/skills/recipe-report/example-spec.json` is the gold-standard
example (operator-approved style/format/voice). **Match its editorial voice, field shapes, and level
of specificity**, following the section order above. Read it before writing your spec.
5. **Render & publish — via the script only; never hand-edit the HTML.**
`python3 .../recipe-report.py render /tmp/report-spec.json /tmp/week-<DATE>.html`
`python3 .../recipe-report.py publish /tmp/week-<DATE>.html <DATE>`
(publish copies the page to `cc-ci:/var/lib/cc-ci-reports/` and regenerates the index.)
If `render` errors, **fix the spec JSON** (a malformed field) and re-run render — do NOT work around
it by writing HTML by hand. The rendered file is an artifact of the spec, never edited directly.
6. **Verify & stop.** `curl -fsS https://report.ci.commoninternet.net/week-<DATE>.html` renders and the
index lists it. Print the report URL and `RECIPE REPORT COMPLETE`, then go idle (one-shot — do not loop).
## Guardrails
- **Read-only.** Never merge/edit/comment on PRs or touch recipes/tests. You only describe + link.
- **Public-safe.** No secrets/tokens/internal hostnames/raw logs on the page. Summarize errors.
- One page per run; always regenerate the index so all weeks are listed (newest first).