autonomic-bot
fc36d0e10f
Operator: 'the scanner should not say 0 when it was not able to scan.' Correct — the previous patch still led with '0 identified' and relegated the caveat to a footnote, so the headline number was wrong even though the prose was right. A 0 in a security column is an assertion of safety; it must never be emitted for an undetermined result. Now: cve_count_fixed is null (not 0) in JSON, a count_known flag distinguishes 'counted zero' from 'could not count', and the markdown headline reads 'CVEs fixed by this upgrade: UNKNOWN — the scan could NOT determine a count' with an explicit 'This is NOT zero' and instructions to render '?'. Verified: discourse 3.5.3 -> 2026.7.1 (semver->calver) now reports UNKNOWN; gitea 1.27.0 -> 1.27.1 still reports 2.
114 lines
9.1 KiB
Markdown
114 lines
9.1 KiB
Markdown
---
|
|
name: recipe-report
|
|
description: Generate the weekly public "Recipe Report" after the cc-ci /upgrade-all run. Reviews how the upgrade went and the state of every recipe + open PR, and publishes a self-contained HTML page to report.ci.commoninternet.net (one page per week + a home index). Page order: short lead → the full wire table (priority-sorted, CVE-count column, CVE recipes first) → Addendum of special issues → Security Bulletin → per-recipe "what changed". Read-only — reports, never merges or edits PRs. Runs as its own agent (model separately overridable via REPORT_BACKEND/REPORT_MODEL; defaults to the upgrader's backend+model, opencode-go/glm-5.2). Invoke as /recipe-report [YYYY-MM-DD].
|
|
---
|
|
|
|
# recipe-report
|
|
|
|
Produce the weekly **"The Recipe Report"** — a public HTML page reviewing the latest `/upgrade-all` run
|
|
and the live state of every recipe + open PR. Served at **https://report.ci.commoninternet.net** (one
|
|
`week-<date>.html` per run + a home-page index). You are **read-only**: you report, you never merge,
|
|
edit, or comment on PRs. The page is **public + unauthenticated** — include only public-safe data
|
|
(recipe names, version bumps, PR titles/links, CI verdicts, concise error summaries); **never** secrets,
|
|
tokens, internal IPs, or raw logs.
|
|
|
|
Helper: `python3 /srv/cc-ci/cc-ci-plan/recipe-report.py {survey|render|publish}` (see its header).
|
|
|
|
**Deterministic formatting — spec in, page out (NON-NEGOTIABLE).** ALL layout, CSS, and HTML is owned by
|
|
`recipe-report.py`'s `render()`. Your ONLY output artifact is the **spec JSON** (data + editorial text).
|
|
You **never** write, edit, post-process, or templatize HTML yourself; you never put markup/inline styles
|
|
in spec text fields (they are HTML-escaped on purpose); the page is produced **solely** by
|
|
`render <spec> <out>`. Same spec shape every week → a structurally identical page. If the layout itself
|
|
must change, edit `recipe-report.py` (and `example-spec.json`) — never the generated page. This is what
|
|
keeps every weekly edition looking the same regardless of which model writes the spec.
|
|
|
|
## Procedure
|
|
|
|
1. **Pick the date.** `DATE` = the argument, else today (UTC). The `/upgrade-all` summary for that run is
|
|
`/srv/cc-ci/.cc-ci-logs/upgrades/upgrade-all-<DATE>.md`.
|
|
|
|
2. **Survey.** `python3 /srv/cc-ci/cc-ci-plan/recipe-report.py survey <DATE> > /tmp/survey.json`. This
|
|
gives you: the raw `/upgrade-all` summary; for every recipe mirror its open PRs + the `cc-ci/testme`
|
|
verdict; and each recipe's **per-recipe upgrade notes** (`upgrade_notes_md` — the breaking-change /
|
|
**CVE / security** analysis the upgrade did). Read it all.
|
|
|
|
3. **Review & classify.** The page order is: **short lead → the full wire table (priority-sorted) →
|
|
Addendum → Security Bulletin → per-recipe "What changed".** Same newspaper style; this order.
|
|
- **Security analysis.** Scan the per-recipe `upgrade_notes_md` + the summary (and use your own
|
|
knowledge of the version bumps) for upgrades that fix **CVEs / security issues**. For each recipe,
|
|
**count the CVEs** the PR fixes — this drives both the table's `cve` column and the priority sort.
|
|
- **ADDITIONALLY, and never instead:** each per-recipe log carries an `### Advisory scan
|
|
(deterministic pre-step)` block (from `cc-ci-plan/advisory-scan.py` — GitHub Security
|
|
Advisories + vendor security pages + OSV, with severities and fixed-in versions). Treat its
|
|
CVE list as a **further source** and report the **UNION** of it and what you found by reading.
|
|
Its entries are machine-derived with advisory IDs, so prefer them for CVE ids / severities /
|
|
fixed-in versions, and cite the GHSA where present in the Security Bulletin. If the block
|
|
lists **failed sources**, the count is **not** authoritative: render the cve cell as `?`
|
|
(unknown), never `none` — a blank that reads as "clean" is exactly how two CVSS-9.8 gitea
|
|
RCEs were reported as "none" on 2026-08-07.
|
|
- **`?` must stay RARE — it means "we tried and could not tell", not "we didn't look".** Use it
|
|
ONLY when a scan ran and reported genuinely failed sources, **or when the scan block says
|
|
COUNT UNKNOWN** (it refuses to classify across a version-scheme change, e.g. semver → calver —
|
|
discourse 3.5.3 → 2026.7.1). In that case the scan's `0` means *not determined*: publish `?`
|
|
and say so in the notes; publishing `0` would assert a clean bill of health nothing supports.
|
|
(The scan headline itself now says `UNKNOWN` rather than a number in that case.)
|
|
In particular: a recipe with **no upgrade this run** (up-to-date/skipped) has nothing an
|
|
upgrade could have fixed — report `0`, not `?`. A recipe with a clean scan reports its number (including `0`). Benign notes in a scan
|
|
block (`no-advisories-published`, `skipped: template URL`) are NOT failures and must not
|
|
trigger `?`. If you find yourself rendering `?` for many recipes, that is a bug to report in
|
|
the Addendum, not a normal outcome.
|
|
Anything **critical/high** also gets a `security` bulletin entry (recipe · CVE id(s) + severity ·
|
|
what it fixes · PR link); be specific about severity and what's exposed if not merged.
|
|
- **Lead — ONE short paragraph.** A tight, concrete opener in opus's voice: fleet state in a sentence
|
|
(how healthy, what moved) and what to address first. Lead with the single most important thing; the
|
|
table and sections carry all the detail. Do NOT write multiple paragraphs.
|
|
- **Priority sort for the table.** Order rows by **recommended priority to address**:
|
|
1. recipes **with CVEs** first (more/higher-severity CVEs higher),
|
|
2. then **FAILED** (RED) recipes,
|
|
3. then **STALE**-test recipes,
|
|
4. then routine **GREEN** bumps,
|
|
5. then **UPTODATE / SKIPPED** last.
|
|
- **Addendum — special issues to look into.** A bullet list of real anomalies worth an operator's
|
|
eye: a recipe carrying **two open PRs** to reconcile, a **CI-run oddity** (flaky tier, needed
|
|
re-runs), drift between the summary and reality, a **misleading branch name**, a tooling snag that
|
|
silently drops a recipe, leftover artifact PRs, or **something in the test run that could be
|
|
improved**. **Only real issues — if everything looks clean, leave it empty; do NOT invent items.**
|
|
- **What changed — per recipe with a PR.** One short section per recipe that has a PR this week
|
|
describing what the upgrade actually changes (versions, notable deps, migrations/gotchas, reconcile
|
|
notes). Skip recipes with no PR.
|
|
|
|
4. **Write the spec** `/tmp/report-spec.json` (shape in the helper header):
|
|
- `date`, `subtitle` ("Week of <human date>"),
|
|
- `lead` — **one short paragraph**,
|
|
- `table[]` — EVERY recipe, **sorted by the priority above**. Fields: `recipe`, `change` "a → b",
|
|
`status` GREEN/FAILED/STALE/SKIPPED/UPTODATE, **`cve`** (integer count of CVEs this PR fixes; `0`
|
|
or omit for none), `ci` as a level/result number+info string e.g. `build 154 ✓` or `RED 200 ·
|
|
install` with `ci_url`, `pr`/`pr_url`, `notes`. **CI = link + number/info only; no images.**
|
|
- The table renders **two** status columns: **TESTS** (the `status` field — the CI/test verdict)
|
|
and **STATUS** (the PR's LIVE state, fetched client-side). You only supply `status`; the live
|
|
STATUS column is derived automatically from `recipe` + `pr` and needs no spec field — it shows
|
|
`open` vs a ✓ (any not-open state), refreshing ~30s via the same-origin `/pr/<recipe>/<n>` proxy
|
|
(cc-ci `nix/modules/reports.nix`); it degrades to a muted "?" if the proxy/repo is unreachable.
|
|
- `addendum[]` — list of bullet strings (may be empty),
|
|
- `security[]` — critical-CVE bulletin entries (may be empty),
|
|
- `changes[]` — `{recipe, body, links[]}`, one per recipe that has a PR.
|
|
|
|
**Template — match this.** `.claude/skills/recipe-report/example-spec.json` is the gold-standard
|
|
example (operator-approved style/format/voice). **Match its editorial voice, field shapes, and level
|
|
of specificity**, following the section order above. Read it before writing your spec.
|
|
|
|
5. **Render & publish — via the script only; never hand-edit the HTML.**
|
|
`python3 .../recipe-report.py render /tmp/report-spec.json /tmp/week-<DATE>.html`
|
|
`python3 .../recipe-report.py publish /tmp/week-<DATE>.html <DATE>`
|
|
(publish copies the page to `cc-ci:/var/lib/cc-ci-reports/` and regenerates the index.)
|
|
If `render` errors, **fix the spec JSON** (a malformed field) and re-run render — do NOT work around
|
|
it by writing HTML by hand. The rendered file is an artifact of the spec, never edited directly.
|
|
|
|
6. **Verify & stop.** `curl -fsS https://report.ci.commoninternet.net/week-<DATE>.html` renders and the
|
|
index lists it. Print the report URL and `RECIPE REPORT COMPLETE`, then go idle (one-shot — do not loop).
|
|
|
|
## Guardrails
|
|
- **Read-only.** Never merge/edit/comment on PRs or touch recipes/tests. You only describe + link.
|
|
- **Public-safe.** No secrets/tokens/internal hostnames/raw logs on the page. Summarize errors.
|
|
- One page per run; always regenerate the index so all weeks are listed (newest first).
|