Files
cc-ci-orchestrator/.claude/skills/recipe-report/SKILL.md
T
autonomic-bot 318d09bdab advisory-scan: eliminate spurious '?' — url punctuation, stale URL, and ? semantics
The 2026-08-07 regeneration rendered '?' for 5 of 21 recipes. '?' is meant to be a
rare 'we tried and could not tell'; at that rate it is indistinguishable from noise
and hides the real unknowns. Three causes, none of them genuine uncertainty:

1. URL EXTRACTION BUG (mine). The registry is markdown, so urls appear inside
   `backticks` and 'quotes'. The extractor captured the trailing punctuation, so
   it fetched https://docs.n8n.io/release-notes/` and https://git.autonomic.zone'`
   — both 404 on the malformed url, both 200 when clean. Trailing markdown
   punctuation is now stripped. Fixed immich + n8n.
2. STALE REGISTRY URL. mattermost-lts pointed at
   docs.mattermost.com/about/mattermost-changelog.html, which 404s; the page moved
   to /deploy/. Corrected (same class as the pgautoupgrade fix).
3. WRONG SEMANTICS FOR 'NO UPGRADE'. lasuite-docs and custom-html-tiny were
   up-to-date this run, so no scan block existed and the report fell back to '?'.
   But a recipe with no upgrade has nothing an upgrade could have fixed — that is
   0, not unknown. The report skill now says so explicitly, restricts '?' to scans
   that RAN and reported genuinely failed sources, states that benign notes
   (no-advisories-published / template URL) never trigger '?', and instructs that
   many '?' is itself a bug to raise in the Addendum.

Result across all 16 scanned recipes of that run: 0 failed sources (was 5).
Counts also improved with the classifier fix: discourse 130->133, keycloak ->7.
2026-08-10 20:23:10 +00:00

8.6 KiB


name: recipe-report description: Generate the weekly public "Recipe Report" after the cc-ci /upgrade-all run. Reviews how the upgrade went and the state of every recipe + open PR, and publishes a self-contained HTML page to report.ci.commoninternet.net (one page per week + a home index). Page order: short lead → the full wire table (priority-sorted, CVE-count column, CVE recipes first) → Addendum of special issues → Security Bulletin → per-recipe "what changed". Read-only — reports, never merges or edits PRs. Runs as its own agent (model separately overridable via REPORT_BACKEND/REPORT_MODEL; defaults to the upgrader's backend+model, opencode-go/glm-5.2). Invoke as /recipe-report [YYYY-MM-DD].

recipe-report

Produce the weekly "The Recipe Report" — a public HTML page reviewing the latest /upgrade-all run and the live state of every recipe + open PR. Served at https://report.ci.commoninternet.net (one week-<date>.html per run + a home-page index). You are read-only: you report, you never merge, edit, or comment on PRs. The page is public + unauthenticated — include only public-safe data (recipe names, version bumps, PR titles/links, CI verdicts, concise error summaries); never secrets, tokens, internal IPs, or raw logs.

Helper: python3 /srv/cc-ci/cc-ci-plan/recipe-report.py {survey|render|publish} (see its header).

Deterministic formatting — spec in, page out (NON-NEGOTIABLE). ALL layout, CSS, and HTML is owned by recipe-report.py's render(). Your ONLY output artifact is the spec JSON (data + editorial text). You never write, edit, post-process, or templatize HTML yourself; you never put markup/inline styles in spec text fields (they are HTML-escaped on purpose); the page is produced solely by render <spec> <out>. Same spec shape every week → a structurally identical page. If the layout itself must change, edit recipe-report.py (and example-spec.json) — never the generated page. This is what keeps every weekly edition looking the same regardless of which model writes the spec.

Procedure

  1. Pick the date. DATE = the argument, else today (UTC). The /upgrade-all summary for that run is /srv/cc-ci/.cc-ci-logs/upgrades/upgrade-all-<DATE>.md.

  2. Survey. python3 /srv/cc-ci/cc-ci-plan/recipe-report.py survey <DATE> > /tmp/survey.json. This gives you: the raw /upgrade-all summary; for every recipe mirror its open PRs + the cc-ci/testme verdict; and each recipe's per-recipe upgrade notes (upgrade_notes_md — the breaking-change / CVE / security analysis the upgrade did). Read it all.

  3. Review & classify. The page order is: short lead → the full wire table (priority-sorted) → Addendum → Security Bulletin → per-recipe "What changed". Same newspaper style; this order.

    • Security analysis. Scan the per-recipe upgrade_notes_md + the summary (and use your own knowledge of the version bumps) for upgrades that fix CVEs / security issues. For each recipe, count the CVEs the PR fixes — this drives both the table's cve column and the priority sort.
      • ADDITIONALLY, and never instead: each per-recipe log carries an ### Advisory scan (deterministic pre-step) block (from cc-ci-plan/advisory-scan.py — GitHub Security Advisories + vendor security pages + OSV, with severities and fixed-in versions). Treat its CVE list as a further source and report the UNION of it and what you found by reading. Its entries are machine-derived with advisory IDs, so prefer them for CVE ids / severities / fixed-in versions, and cite the GHSA where present in the Security Bulletin. If the block lists failed sources, the count is not authoritative: render the cve cell as ? (unknown), never none — a blank that reads as "clean" is exactly how two CVSS-9.8 gitea RCEs were reported as "none" on 2026-08-07.
      • ? must stay RARE — it means "we tried and could not tell", not "we didn't look". Use it ONLY when a scan ran and reported genuinely failed sources. In particular: a recipe with no upgrade this run (up-to-date/skipped) has nothing an upgrade could have fixed — report 0, not ?. A recipe with a clean scan reports its number (including 0). Benign notes in a scan block (no-advisories-published, skipped: template URL) are NOT failures and must not trigger ?. If you find yourself rendering ? for many recipes, that is a bug to report in the Addendum, not a normal outcome. Anything critical/high also gets a security bulletin entry (recipe · CVE id(s) + severity · what it fixes · PR link); be specific about severity and what's exposed if not merged.
    • Lead — ONE short paragraph. A tight, concrete opener in opus's voice: fleet state in a sentence (how healthy, what moved) and what to address first. Lead with the single most important thing; the table and sections carry all the detail. Do NOT write multiple paragraphs.
    • Priority sort for the table. Order rows by recommended priority to address:
      1. recipes with CVEs first (more/higher-severity CVEs higher),
      2. then FAILED (RED) recipes,
      3. then STALE-test recipes,
      4. then routine GREEN bumps,
      5. then UPTODATE / SKIPPED last.
    • Addendum — special issues to look into. A bullet list of real anomalies worth an operator's eye: a recipe carrying two open PRs to reconcile, a CI-run oddity (flaky tier, needed re-runs), drift between the summary and reality, a misleading branch name, a tooling snag that silently drops a recipe, leftover artifact PRs, or something in the test run that could be improved. Only real issues — if everything looks clean, leave it empty; do NOT invent items.
    • What changed — per recipe with a PR. One short section per recipe that has a PR this week describing what the upgrade actually changes (versions, notable deps, migrations/gotchas, reconcile notes). Skip recipes with no PR.
  4. Write the spec /tmp/report-spec.json (shape in the helper header):

    • date, subtitle ("Week of "),
    • lead — one short paragraph,
    • table[] — EVERY recipe, sorted by the priority above. Fields: recipe, change "a → b", status GREEN/FAILED/STALE/SKIPPED/UPTODATE, cve (integer count of CVEs this PR fixes; 0 or omit for none), ci as a level/result number+info string e.g. build 154 ✓ or RED 200 · install with ci_url, pr/pr_url, notes. CI = link + number/info only; no images.
      • The table renders two status columns: TESTS (the status field — the CI/test verdict) and STATUS (the PR's LIVE state, fetched client-side). You only supply status; the live STATUS column is derived automatically from recipe + pr and needs no spec field — it shows open vs a ✓ (any not-open state), refreshing ~30s via the same-origin /pr/<recipe>/<n> proxy (cc-ci nix/modules/reports.nix); it degrades to a muted "?" if the proxy/repo is unreachable.
    • addendum[] — list of bullet strings (may be empty),
    • security[] — critical-CVE bulletin entries (may be empty),
    • changes[] — {recipe, body, links[]}, one per recipe that has a PR.

    Template — match this. .claude/skills/recipe-report/example-spec.json is the gold-standard example (operator-approved style/format/voice). Match its editorial voice, field shapes, and level of specificity, following the section order above. Read it before writing your spec.

  5. Render & publish — via the script only; never hand-edit the HTML. python3 .../recipe-report.py render /tmp/report-spec.json /tmp/week-<DATE>.html python3 .../recipe-report.py publish /tmp/week-<DATE>.html <DATE> (publish copies the page to cc-ci:/var/lib/cc-ci-reports/ and regenerates the index.) If render errors, fix the spec JSON (a malformed field) and re-run render — do NOT work around it by writing HTML by hand. The rendered file is an artifact of the spec, never edited directly.

  6. Verify & stop. curl -fsS https://report.ci.commoninternet.net/week-<DATE>.html renders and the index lists it. Print the report URL and RECIPE REPORT COMPLETE, then go idle (one-shot — do not loop).

Guardrails

  • Read-only. Never merge/edit/comment on PRs or touch recipes/tests. You only describe + link.
  • Public-safe. No secrets/tokens/internal hostnames/raw logs on the page. Summarize errors.
  • One page per run; always regenerate the index so all weeks are listed (newest first).