recipe-maintainers/cc-ci-orchestrator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
cf26ef863a58e8abc23bd2b55ef6be8dbec7e4a7
cc-ci-orchestrator/.claude/skills/recipe-report/SKILL.md
T
autonomic-bot 5775fe23f8 security: deterministic advisory scan as an ADDITIVE pre-step 
Why: gitea 1.27.1 fixed CVE-2026-60004 + CVE-2026-59774 (both CVSS 9.8). The
2026-08-03 report printed gitea's CVE count as '1', the 2026-08-07 report as
'none'. Cause chain: the upgrade subagent read the GitHub release notes, which
name NEITHER cve (they are announced only in the vendor blog's security section),
so it recorded one unrelated minor item; the report then derived security content
from those notes plus model knowledge, and the model's training predates the CVEs.
Nothing in the pipeline ever queried an advisory source.

cc-ci-plan/advisory-scan.py — deterministic, per recipe, per upgrade window:
  1. GitHub Security Advisories API for every source repo in the upstream registry.
     PRIMARY: CVE + GHSA + severity + vulnerable/patched ranges, so 'fixed by THIS
     upgrade' is computed. Needs no new per-recipe config (134 registry URLs are
     already github.com).
  2. Vendor release/security pages — every registry URL, fetched + regex-scanned.
     This is the source that actually had the gitea CVEs.
  3. OSV where a package mapping exists — supplementary.
Each source reports its own status so 'checked, none found' is never confused with
'not checked'. Source selection was measured, not assumed: for these two CVEs OSV
404'd and NVD's API had them by neither CPE, id, nor keyword — advisory DBs lag the
vendor, hence 1+2 lead.

Wiring is strictly ADDITIVE:
- /recipe-upgrade gains step 2a: run the scan, paste the block into the per-recipe
  log, and report the UNION of it and the existing release-note reading. The scan
  may never lower a count established by reading.
- /recipe-report treats the block as a FURTHER source, prefers its advisory ids /
  severities / fixed-in versions for citation, and must render '?' (not 'none')
  when a scan is absent or has failed sources — the false-clean 'none' is exactly
  what happened on 2026-08-07.
- upstream/gitea.md records blog.gitea.com as the security-announcement URL.

Verified on the real regression: 1.27.0 -> 1.27.1 now yields exactly the 2 missed
criticals with their GHSA ids; the wider 1.26.2 -> 1.27.1 window yields 62.
2026-08-10 18:25:58 +00:00

8.0 KiB
Raw Blame History

name: recipe-report description: Generate the weekly public "Recipe Report" after the cc-ci /upgrade-all run. Reviews how the upgrade went and the state of every recipe + open PR, and publishes a self-contained HTML page to report.ci.commoninternet.net (one page per week + a home index). Page order: short lead → the full wire table (priority-sorted, CVE-count column, CVE recipes first) → Addendum of special issues → Security Bulletin → per-recipe "what changed". Read-only — reports, never merges or edits PRs. Runs as its own agent (model separately overridable via REPORT_BACKEND/REPORT_MODEL; defaults to the upgrader's backend+model, opencode-go/glm-5.2). Invoke as /recipe-report [YYYY-MM-DD].

recipe-report

Produce the weekly "The Recipe Report" — a public HTML page reviewing the latest /upgrade-all run and the live state of every recipe + open PR. Served at https://report.ci.commoninternet.net (one week-<date>.html per run + a home-page index). You are read-only: you report, you never merge, edit, or comment on PRs. The page is public + unauthenticated — include only public-safe data (recipe names, version bumps, PR titles/links, CI verdicts, concise error summaries); never secrets, tokens, internal IPs, or raw logs.

Helper: python3 /srv/cc-ci/cc-ci-plan/recipe-report.py {survey|render|publish} (see its header).

Deterministic formatting — spec in, page out (NON-NEGOTIABLE). ALL layout, CSS, and HTML is owned by recipe-report.py's render(). Your ONLY output artifact is the spec JSON (data + editorial text). You never write, edit, post-process, or templatize HTML yourself; you never put markup/inline styles in spec text fields (they are HTML-escaped on purpose); the page is produced solely by render <spec> <out>. Same spec shape every week → a structurally identical page. If the layout itself must change, edit recipe-report.py (and example-spec.json) — never the generated page. This is what keeps every weekly edition looking the same regardless of which model writes the spec.

Procedure

  1. Pick the date. DATE = the argument, else today (UTC). The /upgrade-all summary for that run is /srv/cc-ci/.cc-ci-logs/upgrades/upgrade-all-<DATE>.md.

  2. Survey. python3 /srv/cc-ci/cc-ci-plan/recipe-report.py survey <DATE> > /tmp/survey.json. This gives you: the raw /upgrade-all summary; for every recipe mirror its open PRs + the cc-ci/testme verdict; and each recipe's per-recipe upgrade notes (upgrade_notes_md — the breaking-change / CVE / security analysis the upgrade did). Read it all.

  3. Review & classify. The page order is: short lead → the full wire table (priority-sorted) → Addendum → Security Bulletin → per-recipe "What changed". Same newspaper style; this order.

    • Security analysis. Scan the per-recipe upgrade_notes_md + the summary (and use your own knowledge of the version bumps) for upgrades that fix CVEs / security issues. For each recipe, count the CVEs the PR fixes — this drives both the table's cve column and the priority sort.
      • ADDITIONALLY, and never instead: each per-recipe log carries an ### Advisory scan (deterministic pre-step) block (from cc-ci-plan/advisory-scan.py — GitHub Security Advisories + vendor security pages + OSV, with severities and fixed-in versions). Treat its CVE list as a further source and report the UNION of it and what you found by reading. Its entries are machine-derived with advisory IDs, so prefer them for CVE ids / severities / fixed-in versions, and cite the GHSA where present in the Security Bulletin. If a recipe has no scan block, or the block lists failed sources, the count is not authoritative: render the cve cell as ? (unknown), never none — a blank that reads as "clean" is exactly how two CVSS-9.8 gitea RCEs were reported as "none" on 2026-08-07. Anything critical/high also gets a security bulletin entry (recipe · CVE id(s) + severity · what it fixes · PR link); be specific about severity and what's exposed if not merged.
    • Lead — ONE short paragraph. A tight, concrete opener in opus's voice: fleet state in a sentence (how healthy, what moved) and what to address first. Lead with the single most important thing; the table and sections carry all the detail. Do NOT write multiple paragraphs.
    • Priority sort for the table. Order rows by recommended priority to address:
      1. recipes with CVEs first (more/higher-severity CVEs higher),
      2. then FAILED (RED) recipes,
      3. then STALE-test recipes,
      4. then routine GREEN bumps,
      5. then UPTODATE / SKIPPED last.
    • Addendum — special issues to look into. A bullet list of real anomalies worth an operator's eye: a recipe carrying two open PRs to reconcile, a CI-run oddity (flaky tier, needed re-runs), drift between the summary and reality, a misleading branch name, a tooling snag that silently drops a recipe, leftover artifact PRs, or something in the test run that could be improved. Only real issues — if everything looks clean, leave it empty; do NOT invent items.
    • What changed — per recipe with a PR. One short section per recipe that has a PR this week describing what the upgrade actually changes (versions, notable deps, migrations/gotchas, reconcile notes). Skip recipes with no PR.

  4. Write the spec /tmp/report-spec.json (shape in the helper header):

    • date, subtitle ("Week of "),
    • lead — one short paragraph,
    • table[] — EVERY recipe, sorted by the priority above. Fields: recipe, change "a → b", status GREEN/FAILED/STALE/SKIPPED/UPTODATE, cve (integer count of CVEs this PR fixes; 0 or omit for none), ci as a level/result number+info string e.g. build 154 ✓ or RED 200 · install with ci_url, pr/pr_url, notes. CI = link + number/info only; no images.
      • The table renders two status columns: TESTS (the status field — the CI/test verdict) and STATUS (the PR's LIVE state, fetched client-side). You only supply status; the live STATUS column is derived automatically from recipe + pr and needs no spec field — it shows open vs a ✓ (any not-open state), refreshing ~30s via the same-origin /pr/<recipe>/<n> proxy (cc-ci nix/modules/reports.nix); it degrades to a muted "?" if the proxy/repo is unreachable.
    • addendum[] — list of bullet strings (may be empty),
    • security[] — critical-CVE bulletin entries (may be empty),
    • changes[] — {recipe, body, links[]}, one per recipe that has a PR.

    Template — match this. .claude/skills/recipe-report/example-spec.json is the gold-standard example (operator-approved style/format/voice). Match its editorial voice, field shapes, and level of specificity, following the section order above. Read it before writing your spec.

  5. Render & publish — via the script only; never hand-edit the HTML. python3 .../recipe-report.py render /tmp/report-spec.json /tmp/week-<DATE>.html python3 .../recipe-report.py publish /tmp/week-<DATE>.html <DATE> (publish copies the page to cc-ci:/var/lib/cc-ci-reports/ and regenerates the index.) If render errors, fix the spec JSON (a malformed field) and re-run render — do NOT work around it by writing HTML by hand. The rendered file is an artifact of the spec, never edited directly.

  6. Verify & stop. curl -fsS https://report.ci.commoninternet.net/week-<DATE>.html renders and the index lists it. Print the report URL and RECIPE REPORT COMPLETE, then go idle (one-shot — do not loop).

Guardrails

  • Read-only. Never merge/edit/comment on PRs or touch recipes/tests. You only describe + link.
  • Public-safe. No secrets/tokens/internal hostnames/raw logs on the page. Summarize errors.
  • One page per run; always regenerate the index so all weeks are listed (newest first).
Reference in New Issue View Git Blame Copy Permalink