From 7e920c01da7235a54512e54e166da4d9feaa6546 Mon Sep 17 00:00:00 2001 From: notplants <@notplants> Date: Mon, 16 Nov 2026 17:56:02 +0000 Subject: [PATCH] fix: count CVEs from current open PR against current target --- .claude/skills/recipe-report/SKILL.md | 37 +++++++++++++++++---------- cc-ci-plan/recipe-report.py | 6 ++++- 2 files changed, 28 insertions(+), 15 deletions(-) diff --git a/.claude/skills/recipe-report/SKILL.md b/.claude/skills/recipe-report/SKILL.md index 6e4837c..d1095d4 100644 --- a/.claude/skills/recipe-report/SKILL.md +++ b/.claude/skills/recipe-report/SKILL.md @@ -39,14 +39,18 @@ keeps every weekly edition looking the same regardless of which model writes the - **Security analysis.** Scan the per-recipe `upgrade_notes_md` + the summary (and use your own knowledge of the version bumps) for upgrades that fix **CVEs / security issues**. For each recipe, **count the CVEs** the PR fixes — this drives both the table's `cve` column and the priority sort. - Before crediting a CVE to an open PR, compare that PR's head `compose.yml` with its **merge-base - commit on the target branch**. Run `advisory-scan.py --compose-from + Before crediting a CVE to an open PR, compare that PR's head `compose.yml` with the **current + target-branch tip**, both pinned by SHA at report time. Run + `advisory-scan.py --compose-from --compose-to ` when an upgrade log lacks that exact scan, or when its listed image windows disagree with the PR diff. Credit image CVEs only for image version changes in - that diff. A previous release or warm canonical is not the PR baseline: it can include fixes - already present in the target branch. For example, lasuite-docs PR #11 did not change Valkey, + that diff, and only if the PR head is the fixed version. An upgrade-time merge base may now be + stale if upstream advanced; a previous release or warm canonical can also include fixes already + present in the target branch. For example, lasuite-docs PR #11 did not change Valkey, so its 8.1.11→9.1.3 CVE must not be credited to that PR. Record a security-relevant config - fix separately with direct evidence. If the PR diff cannot be verified, use `?` and explain why. + fix separately with direct evidence that the target branch lacks it. If the current diff cannot + be verified, use `?` and explain why. If the PR head is behind the target branch or unmergeable, + say so in the same row; its old CI result is not proof that the current combined recipe works. - **ADDITIONALLY, and never instead:** each per-recipe log carries an `### Advisory scan (deterministic pre-step)` block (from `cc-ci-plan/advisory-scan.py` — GitHub Security Advisories + vendor security pages + OSV, with severities and fixed-in versions). Treat its @@ -56,10 +60,13 @@ keeps every weekly edition looking the same regardless of which model writes the lists **failed sources**, the count is **not** authoritative: render the cve cell as `?` (unknown), never `none` — a blank that reads as "clean" is exactly how two CVSS-9.8 gitea RCEs were reported as "none" on 2026-08-07. - - The table's **CVEs** column counts known CVEs addressed by the available recipe upgrade. - A zero for an up-to-date recipe means there is no upgrade window to credit; it does not - certify that the running images are free of vulnerabilities. If an upgrade exists but its - advisory scan is absent, a source failed, or an image window was unmatched, use `?`. + - The table's **CVEs** column counts known CVEs fixed by the **current open PR's diff against + current target branch**, even if `/upgrade-all` found no new image version this week. Keep the + existing one-row-per-recipe table and show that PR in the row. `SKIPPED — up-to-date` describes + this week's image survey, not the absence of a relevant open PR; use the PR's actual CI verdict + in TESTS when one exists. A zero means the current PR fixes no verified CVEs (or there is no + PR); it does not certify that deployed images are vulnerability-free. If the PR's advisory + scan is absent, a source failed, or an image window was unmatched, use `?`. If a confirmed security advisory is fixed but has no CVE ID, put `"YES"` in `cve` rather than `0`; describe its advisory ID and severity in Notes. If CVE IDs are also fixed, show the numeric CVE count and mention the additional non-CVE advisories in Notes. @@ -78,8 +85,10 @@ keeps every weekly edition looking the same regardless of which model writes the When a sidecar contributes a critical/high, name the image in the bulletin — CVE-2025-49844 is a redis flaw, not a discourse one, and an operator reading "discourse" needs to know that. (The scan headline itself now says `UNKNOWN` rather than a number in that case.) - In particular: a recipe with **no upgrade this run** (up-to-date/skipped) has nothing an - upgrade could have fixed — report `0`, not `?`. A recipe with a clean scan reports its number (including `0`). Benign notes in a scan + In particular: **no new image upgrade this run does not force `0`** if an open PR still has + an unmerged security change. Recheck its current diff and credit only fixes absent from the + current target branch. A recipe with no relevant PR reports `0`; a clean PR scan reports its + number (including `0`). Benign notes in a scan block (`no-advisories-published`, `skipped: template URL`) are NOT failures and must not trigger `?`. If you find yourself rendering `?` for many recipes, that is a bug to report in the Addendum, not a normal outcome. @@ -99,9 +108,9 @@ keeps every weekly edition looking the same regardless of which model writes the re-runs), drift between the summary and reality, a **misleading branch name**, a tooling snag that silently drops a recipe, leftover artifact PRs, or **something in the test run that could be improved**. **Only real issues — if everything looks clean, leave it empty; do NOT invent items.** - - **What changed — per recipe with a PR.** One short section per recipe that has a PR this week - describing what the upgrade actually changes (versions, notable deps, migrations/gotchas, reconcile - notes). Skip recipes with no PR. + - **What changed — per recipe with a relevant open PR.** Describe the PR's current diff (versions, + configuration, migrations/gotchas, reconcile notes), including an older PR still awaiting merge. + Say when it was last updated and whether it can merge cleanly. Skip recipes with no relevant PR. 4. **Write the spec** `/tmp/report-spec.json` (shape in the helper header): - `date`, `subtitle` ("Week of "), diff --git a/cc-ci-plan/recipe-report.py b/cc-ci-plan/recipe-report.py index 8075531..73cb72f 100755 --- a/cc-ci-plan/recipe-report.py +++ b/cc-ci-plan/recipe-report.py @@ -100,7 +100,11 @@ def survey(date): if s.get("context") == "cc-ci/testme": ci = {"state": s.get("state"), "url": s.get("target_url"), "desc": s.get("description")} rec["open_prs"].append({"number": p["number"], "title": p["title"], - "url": p["html_url"], "head": sha[:10], "ci": ci}) + "url": p["html_url"], "head": sha, + "base": p.get("base", {}).get("sha"), + "created_at": p.get("created_at"), + "updated_at": p.get("updated_at"), + "mergeable": p.get("mergeable"), "ci": ci}) # Per-recipe upgrade plan/summary (has the breaking-change + CVE/security analysis) — the agent # mines this to lead with critical-CVE upgrades. nf = os.path.join(LOGDIR, "upgrades", f"{r}-upgrade-{date}.md")