recipe-maintainers/cc-ci-orchestrator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

advisory-scan: count sidecar CVEs via per-image windows

Browse Source 
A recipe upgrades several images, each through its own version range. The scan
previously classified only the app repo, so sidecar bumps contributed nothing — the
alternative to the earlier bug where sidecars were judged by the APP's window and
produced a false 133.

Now: --window KEY=FROM:TO (repeatable) gives any other source its own range; each
window is classified independently (one may use patched-version ranges while another
falls back to advisory dates) and the count is the UNION. An image with no window is
still not counted — the scan will not guess a range it was not given. If ANY requested
window cannot be ordered, the total is UNKNOWN rather than a partial number.

/recipe-upgrade now instructs passing a --window per bumped sidecar.

Verified on discourse app 3.5.3->2026.7.1 + redis 7.4->8.10: 128 = 123 (app, by
publish date) + 5 (redis, by version range). The redis five are genuine for that bump
(patched 7.4.1 / 7.4.6 / 8.2.3) and include CVE-2025-49844, CRITICAL — previously
invisible. Regressions clean: gitea still 2, discourse without the sidecar window
still 123.
This commit is contained in:
autonomic-bot
2026-08-10 22:01:28 +00:00
parent 78ae2be8ae
commit b5f8543a9b
3 changed files with 133 additions and 81 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude/skills/recipe-upgrade/SKILL.md
+8 -1
View File
@@ -163,9 +163,16 @@ Run the deterministic scanner for the exact upgrade window and **paste its markd
into the per-recipe log**:
```
python3 /srv/cc-ci/cc-ci-plan/advisory-scan.py <recipe> --from <old-app-version> --to <new-app-version>
python3 /srv/cc-ci/cc-ci-plan/advisory-scan.py <recipe> --from <old-app-version> --to <new-app-version> \
[--window <image-key>=<old>:<new> ...]
```
**Pass a `--window` for EVERY sidecar you bumped** (redis, postgres, nginx …), not just the app —
each image is judged by its own version range, and an image without a window is not counted at all.
e.g. discourse bumping app 3.5.3→2026.7.1 *and* redis 7.4→8.10:
`--from 3.5.3 --to 2026.7.1 --window redis=7.4:8.10` → 128 CVEs (123 app + 5 redis), where the redis
five include a **critical** (CVE-2025-49844) that is invisible if the sidecar is left out.
It queries, per recipe: the **GitHub Security Advisories API** for every source repo in
`cc-ci-plan/upstream/<recipe>.md` (CVE + GHSA + severity + vulnerable/patched ranges, so
"fixed by THIS upgrade" is computed, not guessed), every **vendor release/security URL** in that
cc-ci-plan/advisory-scan.SPEC.md
+16 -9
View File
@@ -26,6 +26,7 @@ advisory-scan.py <recipe> [--from <version>] [--to <version>] [--json] [--regist
|---|---|
| `<recipe>` | Recipe name; selects `cc-ci-plan/upstream/<recipe>.md` (the per-recipe URL registry) |
| `--from` / `--to` | The **primary app image's** version window being upgraded across |
| `--window KEY=FROM:TO` | A **sidecar's own** window (repeatable). `KEY` matches a source repo name, e.g. `--window redis=7.4:8.10`. Without it that image's advisories stay unclassified. |
| `--registry` | Registry dir; also `CCCI_UPSTREAM_REGISTRY` |
| `GITHUB_TOKEN` / `GITHUB_TOKEN_FILE` | Read-only token; **rate limit only** (60/hr anonymous → 5000/hr). Default file `/srv/cc-ci/.github-token`, mode 600. Public advisories need **no scopes**. |
@@ -100,12 +101,17 @@ published_at, context}`. A CVE seen by several sources keeps them all.
Two invariants govern this step, both learned from a wrong answer in production.
> **A. The window belongs to ONE image.** Only advisories from the **primary** source (the first
> `github-advisories:` source, i.e. the app repo the registry lists first) are classified. Sidecar
> advisories are recorded as unclassified so they stay visible without inflating the count.
> **A. Every image is judged by its OWN window.** The app repo uses `--from/--to`; each sidecar uses
> its own `--window KEY=FROM:TO`. An image with no window is **not** classified — its advisories are
> listed as unclassified so they stay visible without inflating the count. The reported count is the
> **union across windows**, and each window is classified independently (so one may use version
> ranges while another falls back to dates).
> *Why:* discourse once reported **133**, of which **34 were redis CVEs** — including
> `CVE-2021-21309`, patched in redis 6.0.11 in 2021 — counted purely because 6.0.11 sits numerically
> inside discourse's `3.5.3 → 2026.7.1` range.
> inside discourse's `3.5.3 → 2026.7.1` range. The fix is not to ignore sidecars but to give each one
> the window it actually moved through: with `--window redis=7.4:8.10`, discourse scores
> **128 = 123 (app, by date) + 5 (redis, by version range)** — and the redis five include
> `CVE-2025-49844`, **critical**, which was invisible while sidecars went uncounted.
>
> **B. Never emit a number you cannot justify.** If neither method below can order the window, the
> count is `null` / `UNKNOWN`, never `0`. A `0` in a security column asserts safety.
@@ -147,10 +153,11 @@ Markdown (default) for pasting into the per-recipe upgrade log, or `--json`.
| Field | Meaning |
|---|---|
| `cve_count_fixed` | Count, or **`null`** when undeterminable |
| `cve_count_fixed` | Union across all windows, or **`null`** if ANY requested window could not be ordered (a partial number would understate) |
| `count_known` | Distinguishes "counted zero" from "could not count" |
| `classified_by` | `patched version ranges` or `advisory publish date (version scheme changed)` |
| `date_window` | The resolved date window, when 4b was used |
| `windows` | Every source classified, with its from/to |
| `classified_by` | **Per source**: `patched version ranges` or `advisory publish date (version scheme changed)` |
| `date_window` | **Per source**, when 4b was used |
| `fixed_by_this_upgrade[]` | CVE ids, with severity / GHSA / fixed-in per id |
| `unclassified[]` | Seen but not attributable to this window (incl. other images) |
| `sources[]` | Every source with its own status |
@@ -175,8 +182,8 @@ in this tool or stale registry data.
## Known limits
1. **One window per scan.** Sidecar bumps (redis, postgres) are not counted; they appear as
unclassified. Per-image windows would be the natural extension.
1. **Windows must be supplied.** An image with no `--window` is not counted — the scan will not
guess a version range it was not told. `/recipe-upgrade` passes one per image it bumped.
2. **Date-based counts are temporal**, not exact — they assume publish-at-fix-time.
3. **Registry-bound.** Unlisted vendor security pages are invisible; the scan cannot know what it was
never pointed at.
cc-ci-plan/advisory-scan.py
+109 -71
View File
@@ -51,6 +51,8 @@ REGISTRY_DIR = os.environ.get("CCCI_UPSTREAM_REGISTRY", "/srv/cc-ci/cc-ci-plan/u
UA = "cc-ci-advisory-scan (+https://git.autonomic.zone/recipe-maintainers/cc-ci)"
TIMEOUT = int(os.environ.get("ADVISORY_SCAN_TIMEOUT", "45"))
CVE_RE = re.compile(r"CVE-\d{4}-\d{4,7}")
# Leading-version-component jump that means the scheme changed (semver → calver).
SCHEME_JUMP = 100
# Optional OSV mappings: recipe -> (ecosystem, package). Supplementary only (see module docstring).
OSV_PACKAGES: dict[str, tuple[str, str]] = {
@@ -281,7 +283,8 @@ def osv(recipe: str, version: str | None) -> dict | None:
return entry
def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str) -> dict:
def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
extra_windows: list[tuple[str, str, str]] | None = None) -> dict:
urls, reg_path = registry_urls(recipe, registry_dir)
report: dict = {
"recipe": recipe,
@@ -343,74 +346,92 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str) -
# meaningless: 2025.12.2 compares "newer" than 3.5.3 while shipping earlier. When the
# leading component jumps by more than SCHEME_JUMP we refuse to classify and say so,
# rather than emitting a confident wrong number.
kf, kt = _vkey(v_from), _vkey(v_to)
SCHEME_JUMP = 100
scheme_change = bool(kf and kt and abs(kt[0] - kf[0]) >= SCHEME_JUMP)
primary = None
if v_from or v_to:
# the app repo = first github source in the registry (registry lists the app service first)
primary = next((s["source"] for s in report["sources"]
if s["source"].startswith("github-advisories:")), None)
# ── Classification ────────────────────────────────────────────────────────────────────────
# A recipe upgrades SEVERAL images (app + redis/postgres/nginx sidecars), each with its OWN
# version window. Judging every advisory by the app's window is how discourse once reported a
# false 133 (34 of them redis CVEs, incl. one patched in redis 6.0.11 in 2021). So each source
# is classified against ITS OWN window, and the count is the union across windows.
#
# --from/--to → the PRIMARY app repo (first github source in the registry)
# --window K=F:T → any other source whose name contains K (repeatable), e.g. redis=7.4:8.10
#
# A source with no window is not classified: its advisories are listed as unclassified so they
# stay visible without inflating the count.
gh_sources = [x["source"] for x in report["sources"] if x["source"].startswith("github-advisories:")]
primary = gh_sources[0] if (gh_sources and (v_from or v_to)) else None
report["primary_source"] = primary
report["scheme_change"] = scheme_change
fixed, unknown = [], []
for cve, e in report["cves"].items():
# `patched_versions` is a RANGE EXPRESSION (">= 2.18.1"), not a bare version, and there may
# be several (one per patched release line, joined with ";"). Pull every version-looking
# token and treat the advisory as fixed-by-this-upgrade if ANY of them lands in (from, to].
cands = [_vkey(t) for t in re.findall(r"\d+(?:\.\d+)*", e.get("patched") or "")]
kp = next((c for c in cands if kf and kt and kf < c <= kt), None) or (cands[0] if cands else ())
from_primary = primary is not None and primary in e["sources"]
if scheme_change:
e["classification"] = "unclassified: version-scheme change, cannot order reliably"
unknown.append(cve)
elif not from_primary:
e["classification"] = "unclassified: different image than the given version window"
unknown.append(cve)
elif kf and kt and kp and kf < kp <= kt:
e["classification"] = "fixed-by-this-upgrade"
fixed.append(cve)
else:
e["classification"] = "unclassified" if not (kf and kt and kp) else "outside-window"
if e["classification"] == "unclassified":
unknown.append(cve)
# NEVER report 0 for something we could not determine. When classification was refused, the
# count is UNKNOWN (null) — a 0 would be read as "no CVEs", which is an assertion this scan
# cannot make. Consumers must distinguish "counted 0" from "could not count".
# DATE-BASED FALLBACK (phase datewin). Version strings cannot be ordered across a scheme change,
# but RELEASE DATES always can. Resolve both versions to their tag dates on the primary repo and
# count advisories PUBLISHED inside that window — the method a hand count used on 2026-08-10 to
# establish discourse 3.5.3 (2025-12-30) → 2026.7.1 (2026-07-31) = 123 CVEs, where version
# comparison had produced first a false 133 and then a refusal. Only used when the version path
# refuses; a successful version classification is always preferred (it is exact, not temporal).
date_window = None
if scheme_change and primary and primary.startswith("github-advisories:"):
owner_repo = primary.split(":", 1)[1]
owner, _, repo = owner_repo.partition("/")
d_from, d_to = _tag_date(owner, repo, v_from), _tag_date(owner, repo, v_to)
if d_from and d_to and d_from < d_to:
date_window = (d_from, d_to)
fixed, unknown = [], []
for cve, e in report["cves"].items():
pub = e.get("published_at")
if primary in e["sources"] and pub and d_from < pub <= d_to:
e["classification"] = "fixed-by-this-upgrade (by advisory publish date)"
fixed.append(cve)
else:
e["classification"] = ("outside-window (by date)" if primary in e["sources"]
else "unclassified: different image than the given window")
if primary not in e["sources"]:
unknown.append(cve)
scheme_change = False # resolved by date; a real count is available
report["classified_by"] = "advisory publish date (version scheme changed)"
report["date_window"] = {"from": d_from, "to": d_to}
if not date_window and not scheme_change:
report["classified_by"] = "patched version ranges"
report["fixed_by_this_upgrade"] = sorted(fixed)
windows = {} # source name -> (from, to)
if primary:
windows[primary] = (v_from, v_to)
for key, wf, wt in (extra_windows or []):
for src in gh_sources:
if key.lower() in src.lower() and src not in windows:
windows[src] = (wf, wt)
report["windows"] = {k: {"from": f, "to": t} for k, (f, t) in windows.items()}
def _classify_window(src, wf, wt):
"""Return (set_of_fixed_cves, method, date_window|None, unresolved:boolean) for one source."""
kf, kt = _vkey(wf), _vkey(wt)
# A version-SCHEME change (semver 3.5.3 → calver 2026.7.1) makes numeric ordering
# meaningless: 2025.12.2 compares "newer" than 3.5.3 while shipping earlier.
scheme = bool(kf and kt and abs(kt[0] - kf[0]) >= SCHEME_JUMP)
if not scheme:
got = set()
for cve, e in report["cves"].items():
if src not in e["sources"]:
continue
cands = [_vkey(t) for t in re.findall(r"\d+(?:\.\d+)*", e.get("patched") or "")]
if kf and kt and any(kf < c <= kt for c in cands):
got.add(cve)
return got, "patched version ranges", None, False
# DATE FALLBACK: release DATES always order, even across a scheme change. Resolve both
# versions to git tag dates and count advisories PUBLISHED in that window — the method a
# hand count used to establish discourse 3.5.3 (2025-12-30) → 2026.7.1 (2026-07-31) = 123.
owner, _, repo = src.split(":", 1)[1].partition("/")
d_from, d_to = _tag_date(owner, repo, wf), _tag_date(owner, repo, wt)
if d_from and d_to and d_from < d_to:
got = {cve for cve, e in report["cves"].items()
if src in e["sources"] and e.get("published_at")
and d_from < e["published_at"] <= d_to}
return got, "advisory publish date (version scheme changed)", (d_from, d_to), False
return set(), "unresolved", None, True
fixed_set, methods, date_windows, unresolved_any = set(), {}, {}, False
for src, (wf, wt) in windows.items():
got, method, dw, unresolved = _classify_window(src, wf, wt)
methods[src] = method
if dw:
date_windows[src] = {"from": dw[0], "to": dw[1]}
if unresolved:
unresolved_any = True
for cve in got:
report["cves"][cve]["classification"] = f"fixed-by-this-upgrade ({method}) via {src}"
fixed_set.add(cve)
unknown = []
for cve, e in report["cves"].items():
if cve in fixed_set:
continue
if not any(src in e["sources"] for src in windows):
e["classification"] = "unclassified: no version window given for this image"
unknown.append(cve)
else:
e.setdefault("classification", "outside-window")
if e["classification"] == "outside-window":
pass
else:
unknown.append(cve)
report["classified_by"] = methods
if date_windows:
report["date_window"] = date_windows
report["fixed_by_this_upgrade"] = sorted(fixed_set)
report["unclassified"] = sorted(unknown)
report["count_known"] = not scheme_change
report["cve_count_fixed"] = len(fixed) if not scheme_change else None
# NEVER report 0 for something we could not determine — a 0 asserts safety. If ANY requested
# window could not be ordered at all, the total is UNKNOWN rather than a partial number.
report["count_known"] = not unresolved_any
report["cve_count_fixed"] = len(fixed_set) if not unresolved_any else None
report["cve_count_total_seen"] = len(report["cves"])
# Only GENUINE failures make a count unreliable. "no-advisories-published" (404: the repo has
# no advisory feed) and "skipped: template URL" are benign and must not degrade the verdict.
@@ -448,10 +469,15 @@ def markdown(rep: dict) -> str:
return "\n".join(L)
if rep["fixed_by_this_upgrade"]:
L.append(f"\n**CVEs fixed by this upgrade: {rep['cve_count_fixed']}**\n")
if rep.get("date_window"):
L.append(f"_Counted by advisory PUBLISH DATE ({rep['date_window']['from'][:10]} → "
f"{rep['date_window']['to'][:10]}) because the version scheme changed across this "
f"jump; version strings cannot be ordered across it._\n")
cb = rep.get("classified_by") or {}
if isinstance(cb, dict) and cb:
for src, method in cb.items():
dw = (rep.get("date_window") or {}).get(src)
win = (rep.get("windows") or {}).get(src, {})
span = f"{win.get('from')} → {win.get('to')}"
extra = (f" (dates {dw['from'][:10]} → {dw['to'][:10]})" if dw else "")
L.append(f"_{src.split(':',1)[-1]}: {span} — counted by {method}{extra}._")
L.append("")
L.append("| CVE | severity | fixed in | advisory | source |")
L.append("|---|---|---|---|---|")
for cve in rep["fixed_by_this_upgrade"]:
@@ -479,8 +505,20 @@ def main() -> int:
ap.add_argument("--to", dest="v_to", default=None)
ap.add_argument("--json", action="store_true", help="emit raw JSON instead of markdown")
ap.add_argument("--registry", default=REGISTRY_DIR)
ap.add_argument("--window", action="append", default=[], metavar="KEY=FROM:TO",
help="extra image window, e.g. --window redis=7.4:8.10 (repeatable). "
"KEY matches a source repo name; its advisories are then counted "
"against ITS OWN bump instead of being left unclassified.")
a = ap.parse_args()
rep = scan(a.recipe, a.v_from, a.v_to, a.registry)
wins = []
for w in a.window:
key, _, rng = w.partition('=')
wf, _, wt = rng.partition(':')
if key and wf and wt:
wins.append((key, wf, wt))
else:
print(f'ignoring malformed --window {w!r} (expected KEY=FROM:TO)', file=sys.stderr)
rep = scan(a.recipe, a.v_from, a.v_to, a.registry, wins)
print(json.dumps(rep, indent=2) if a.json else markdown(rep))
return 0