advisory-scan: separate benign source absence from real failures; fix pgautoupgrade URLs

Two refinements found by running the scan across all 14 recipes of the 2026-08-07 run:

1. A repo with no advisory feed returns HTTP 404 on /security-advisories (e.g. the
   pgautoupgrade sidecar image). That is a BENIGN ABSENCE, not a failed check.
   Likewise registry entries that are TEMPLATE urls for humans
   (…/changelog/v<VERSION>/, …/<vX.Y.Z>/…) are documentation, not fetchable.
   Counting either as a failure pushed most recipes to '?', which would make the
   unknown-vs-clean distinction meaningless again — the exact signal the ? exists to
   preserve. Both are now recorded in sources_benign; only genuine errors (rate
   limit, network, 5xx, wrong URL) land in sources_failed.

2. upstream/*.md pointed at github.com/pgautoupgrade/pgautoupgrade, which 404s —
   the repo is pgautoupgrade/docker-pgautoupgrade. Corrected in n8n, lasuite-docs,
   lasuite-drive, lasuite-meet. A 404ing registry URL means we were not scanning a
   source we believed we were.

Effect on the 2026-08-07 data: recipes with genuine failed sources 5 -> 3 (the
remainder are really unreachable vendor pages). CVE counts unchanged where they
were already sound: discourse 130, gitea 2, plausible 1.
This commit is contained in:
autonomic-bot
2026-08-10 18:45:45 +00:00
parent cf26ef863a
commit 3307bdb0fe
5 changed files with 29 additions and 7 deletions
+25 -3
View File
@@ -138,7 +138,7 @@ def github_advisories(urls: list[str]) -> list[dict]:
hdrs["Authorization"] = f"Bearer {tok}"
entry = {"source": f"github-advisories:{owner}/{repo}", "status": "ok", "advisories": []}
try:
for a in json.loads(_fetch(api, hdrs)):
for a in json.loads(_fetch(api, hdrs)): # noqa: PERF401 — explicit for readability
vulns = a.get("vulnerabilities") or []
entry["advisories"].append(
{
@@ -151,7 +151,12 @@ def github_advisories(urls: list[str]) -> list[dict]:
"url": a.get("html_url"),
}
)
except Exception as e: # noqa: BLE001 — a dead source must be REPORTED, never silent
except urllib.error.HTTPError as e:
# 404 = this repo simply publishes no security advisories (e.g. sidecar images like
# pgautoupgrade). That is a BENIGN ABSENCE, not a failed check — conflating the two
# would push nearly every recipe to "unknown" and make the ? signal meaningless again.
entry["status"] = "no-advisories-published" if e.code == 404 else f"error: HTTP {e.code}"
except Exception as e: # noqa: BLE001 — a genuinely dead source must be REPORTED, never silent
entry["status"] = f"error: {type(e).__name__}: {e}"
results.append(entry)
return results
@@ -163,6 +168,12 @@ def vendor_pages(urls: list[str]) -> list[dict]:
for u in urls:
if u.startswith("https://api.github.com"):
continue
if re.search(r"[<>{}]|\bVERSION\b|\bvX\.Y\.Z\b", u):
# Registry entries sometimes carry TEMPLATE urls for humans
# (…/changelog/v<VERSION>/). They are documentation, not fetchable — skipping them is
# correct; counting them as failures would wrongly mark the recipe's count unreliable.
out.append({"source": u, "status": "skipped: template URL (not fetchable)", "cves": [], "context": {}})
continue
entry = {"source": u, "status": "ok", "cves": [], "context": {}}
try:
text = _fetch(u)
@@ -267,7 +278,18 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str) -
report["unclassified"] = sorted(unknown)
report["cve_count_fixed"] = len(fixed)
report["cve_count_total_seen"] = len(report["cves"])
report["sources_failed"] = [s["source"] for s in report["sources"] if s["status"] != "ok"]
# Only GENUINE failures make a count unreliable. "no-advisories-published" (404: the repo has
# no advisory feed) and "skipped: template URL" are benign and must not degrade the verdict.
report["sources_failed"] = [
s["source"]
for s in report["sources"]
if not (s["status"] == "ok" or s["status"].startswith(("no-advisories-published", "skipped:")))
]
report["sources_benign"] = [
s["source"]
for s in report["sources"]
if s["status"].startswith(("no-advisories-published", "skipped:"))
]
return report