fix: count CVEs from current open PR against current target

This commit is contained in:
notplants
2026-11-16 17:56:02 +00:00
parent 5d8a86d96b
commit 7e920c01da
2 changed files with 28 additions and 15 deletions
+23 -14
View File
@@ -39,14 +39,18 @@ keeps every weekly edition looking the same regardless of which model writes the
- **Security analysis.** Scan the per-recipe `upgrade_notes_md` + the summary (and use your own - **Security analysis.** Scan the per-recipe `upgrade_notes_md` + the summary (and use your own
knowledge of the version bumps) for upgrades that fix **CVEs / security issues**. For each recipe, knowledge of the version bumps) for upgrades that fix **CVEs / security issues**. For each recipe,
**count the CVEs** the PR fixes — this drives both the table's `cve` column and the priority sort. **count the CVEs** the PR fixes — this drives both the table's `cve` column and the priority sort.
Before crediting a CVE to an open PR, compare that PR's head `compose.yml` with its **merge-base Before crediting a CVE to an open PR, compare that PR's head `compose.yml` with the **current
commit on the target branch**. Run `advisory-scan.py <recipe> --compose-from <base-sha-raw-url> target-branch tip**, both pinned by SHA at report time. Run
`advisory-scan.py <recipe> --compose-from <current-target-sha-raw-url>
--compose-to <head-sha-raw-url>` when an upgrade log lacks that exact scan, or when its listed --compose-to <head-sha-raw-url>` when an upgrade log lacks that exact scan, or when its listed
image windows disagree with the PR diff. Credit image CVEs only for image version changes in image windows disagree with the PR diff. Credit image CVEs only for image version changes in
that diff. A previous release or warm canonical is not the PR baseline: it can include fixes that diff, and only if the PR head is the fixed version. An upgrade-time merge base may now be
already present in the target branch. For example, lasuite-docs PR #11 did not change Valkey, stale if upstream advanced; a previous release or warm canonical can also include fixes already
present in the target branch. For example, lasuite-docs PR #11 did not change Valkey,
so its 8.1.11→9.1.3 CVE must not be credited to that PR. Record a security-relevant config so its 8.1.11→9.1.3 CVE must not be credited to that PR. Record a security-relevant config
fix separately with direct evidence. If the PR diff cannot be verified, use `?` and explain why. fix separately with direct evidence that the target branch lacks it. If the current diff cannot
be verified, use `?` and explain why. If the PR head is behind the target branch or unmergeable,
say so in the same row; its old CI result is not proof that the current combined recipe works.
- **ADDITIONALLY, and never instead:** each per-recipe log carries an `### Advisory scan - **ADDITIONALLY, and never instead:** each per-recipe log carries an `### Advisory scan
(deterministic pre-step)` block (from `cc-ci-plan/advisory-scan.py` — GitHub Security (deterministic pre-step)` block (from `cc-ci-plan/advisory-scan.py` — GitHub Security
Advisories + vendor security pages + OSV, with severities and fixed-in versions). Treat its Advisories + vendor security pages + OSV, with severities and fixed-in versions). Treat its
@@ -56,10 +60,13 @@ keeps every weekly edition looking the same regardless of which model writes the
lists **failed sources**, the count is **not** authoritative: render the cve cell as `?` lists **failed sources**, the count is **not** authoritative: render the cve cell as `?`
(unknown), never `none` — a blank that reads as "clean" is exactly how two CVSS-9.8 gitea (unknown), never `none` — a blank that reads as "clean" is exactly how two CVSS-9.8 gitea
RCEs were reported as "none" on 2026-08-07. RCEs were reported as "none" on 2026-08-07.
- The table's **CVEs** column counts known CVEs addressed by the available recipe upgrade. - The table's **CVEs** column counts known CVEs fixed by the **current open PR's diff against
A zero for an up-to-date recipe means there is no upgrade window to credit; it does not current target branch**, even if `/upgrade-all` found no new image version this week. Keep the
certify that the running images are free of vulnerabilities. If an upgrade exists but its existing one-row-per-recipe table and show that PR in the row. `SKIPPED — up-to-date` describes
advisory scan is absent, a source failed, or an image window was unmatched, use `?`. this week's image survey, not the absence of a relevant open PR; use the PR's actual CI verdict
in TESTS when one exists. A zero means the current PR fixes no verified CVEs (or there is no
PR); it does not certify that deployed images are vulnerability-free. If the PR's advisory
scan is absent, a source failed, or an image window was unmatched, use `?`.
If a confirmed security advisory is fixed but has no CVE ID, put `"YES"` in `cve` rather If a confirmed security advisory is fixed but has no CVE ID, put `"YES"` in `cve` rather
than `0`; describe its advisory ID and severity in Notes. If CVE IDs are also fixed, show than `0`; describe its advisory ID and severity in Notes. If CVE IDs are also fixed, show
the numeric CVE count and mention the additional non-CVE advisories in Notes. the numeric CVE count and mention the additional non-CVE advisories in Notes.
@@ -78,8 +85,10 @@ keeps every weekly edition looking the same regardless of which model writes the
When a sidecar contributes a critical/high, name the image in the bulletin — CVE-2025-49844 is When a sidecar contributes a critical/high, name the image in the bulletin — CVE-2025-49844 is
a redis flaw, not a discourse one, and an operator reading "discourse" needs to know that. a redis flaw, not a discourse one, and an operator reading "discourse" needs to know that.
(The scan headline itself now says `UNKNOWN` rather than a number in that case.) (The scan headline itself now says `UNKNOWN` rather than a number in that case.)
In particular: a recipe with **no upgrade this run** (up-to-date/skipped) has nothing an In particular: **no new image upgrade this run does not force `0`** if an open PR still has
upgrade could have fixed — report `0`, not `?`. A recipe with a clean scan reports its number (including `0`). Benign notes in a scan an unmerged security change. Recheck its current diff and credit only fixes absent from the
current target branch. A recipe with no relevant PR reports `0`; a clean PR scan reports its
number (including `0`). Benign notes in a scan
block (`no-advisories-published`, `skipped: template URL`) are NOT failures and must not block (`no-advisories-published`, `skipped: template URL`) are NOT failures and must not
trigger `?`. If you find yourself rendering `?` for many recipes, that is a bug to report in trigger `?`. If you find yourself rendering `?` for many recipes, that is a bug to report in
the Addendum, not a normal outcome. the Addendum, not a normal outcome.
@@ -99,9 +108,9 @@ keeps every weekly edition looking the same regardless of which model writes the
re-runs), drift between the summary and reality, a **misleading branch name**, a tooling snag that re-runs), drift between the summary and reality, a **misleading branch name**, a tooling snag that
silently drops a recipe, leftover artifact PRs, or **something in the test run that could be silently drops a recipe, leftover artifact PRs, or **something in the test run that could be
improved**. **Only real issues — if everything looks clean, leave it empty; do NOT invent items.** improved**. **Only real issues — if everything looks clean, leave it empty; do NOT invent items.**
- **What changed — per recipe with a PR.** One short section per recipe that has a PR this week - **What changed — per recipe with a relevant open PR.** Describe the PR's current diff (versions,
describing what the upgrade actually changes (versions, notable deps, migrations/gotchas, reconcile configuration, migrations/gotchas, reconcile notes), including an older PR still awaiting merge.
notes). Skip recipes with no PR. Say when it was last updated and whether it can merge cleanly. Skip recipes with no relevant PR.
4. **Write the spec** `/tmp/report-spec.json` (shape in the helper header): 4. **Write the spec** `/tmp/report-spec.json` (shape in the helper header):
- `date`, `subtitle` ("Week of <human date>"), - `date`, `subtitle` ("Week of <human date>"),
+5 -1
View File
@@ -100,7 +100,11 @@ def survey(date):
if s.get("context") == "cc-ci/testme": if s.get("context") == "cc-ci/testme":
ci = {"state": s.get("state"), "url": s.get("target_url"), "desc": s.get("description")} ci = {"state": s.get("state"), "url": s.get("target_url"), "desc": s.get("description")}
rec["open_prs"].append({"number": p["number"], "title": p["title"], rec["open_prs"].append({"number": p["number"], "title": p["title"],
"url": p["html_url"], "head": sha[:10], "ci": ci}) "url": p["html_url"], "head": sha,
"base": p.get("base", {}).get("sha"),
"created_at": p.get("created_at"),
"updated_at": p.get("updated_at"),
"mergeable": p.get("mergeable"), "ci": ci})
# Per-recipe upgrade plan/summary (has the breaking-change + CVE/security analysis) — the agent # Per-recipe upgrade plan/summary (has the breaking-change + CVE/security analysis) — the agent
# mines this to lead with critical-CVE upgrades. # mines this to lead with critical-CVE upgrades.
nf = os.path.join(LOGDIR, "upgrades", f"{r}-upgrade-{date}.md") nf = os.path.join(LOGDIR, "upgrades", f"{r}-upgrade-{date}.md")