audit-sources: check we are still looking where releases actually happen

A recipe tracks an image repo and a set of registry URLs. When upstream moves,
nothing errors — the old repo just stops receiving tags and the recipe looks
'up to date' forever. plausible is the case: it tracked plausible/analytics on
Docker Hub while upstream moved to ghcr.io/plausible/community-edition. Every
survey said 'no upgrades available' while v3 shipped elsewhere.

audit-sources.py reports the signals that catch it, per image and per registry
URL: image gone quiet (newest tag older than --quiet-days), deprecation wording
in the registry description, and GitHub repos that are archived, renamed or
gone. Signals, not verdicts — a stable image can be quiet for good reason — so
each finding says what was measured.

First run over 22 recipes, 11 findings, 4 alerts. It independently re-derived
the plausible case (analytics quiet 1126 days), and found:
  - drone: harness/drone now answers as harness/harness (the image is fine)
  - lasuite-docs, lasuite-drive: minio/minio is ARCHIVED on GitHub
  - lasuite-docs: docspecio/api is ARCHIVED
  - matrix-synapse: halfshot/matrix-appservice-discord image quiet 2078 days
  - mumble: NO cc-ci-plan/upstream/mumble.md at all

That last one exposed a scanner bug. With no registry file there is no source to
query, yet the scan still printed '0 identified by the deterministic scan' — and
that 0 was published as a clean count in the 2026-08-11 CVE check. A scan with no
usable source has measured nothing and must not report a number, least of all 0.
It now returns UNKNOWN and says the registry file is missing.

upstream/mumble.md added; mumble now scans 6 sources for a genuine 0.
This commit is contained in:
autonomic-bot
2026-08-11 15:02:06 +00:00
parent 6c91373357
commit e89da2d842
3 changed files with 282 additions and 6 deletions
+23 -6
View File
@@ -797,8 +797,19 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
report["unclassified"] = sorted(unknown)
# NEVER report 0 for something we could not determine — a 0 asserts safety. If ANY requested
# window could not be ordered at all, the total is UNKNOWN rather than a partial number.
report["count_known"] = not unresolved_any
report["cve_count_fixed"] = len(fixed_set) if not unresolved_any else None
# A scan with NO usable source has not measured anything, so it must not report a number —
# least of all 0, which asserts safety. mumble had no cc-ci-plan/upstream/mumble.md at all and
# still produced "0 identified", which was then published as a clean 0 in a CVE report.
usable_sources = [
s for s in report["sources"]
if s["status"] == "ok" or s["status"].startswith("no-advisories-published")
]
no_sources = not usable_sources
if no_sources:
report["no_usable_sources"] = True
report["count_known"] = not unresolved_any and not no_sources
report["cve_count_fixed"] = (len(fixed_set)
if (not unresolved_any and not no_sources) else None)
report["cve_count_total_seen"] = len(report["cves"])
# Only GENUINE failures make a count unreliable. "no-advisories-published" (404: the repo has
# no advisory feed) and "skipped: template URL" are benign and must not degrade the verdict.
@@ -821,10 +832,16 @@ def markdown(rep: dict) -> str:
f"{rep.get('from') or '?'}{rep.get('to') or '?'}"]
if not rep.get("count_known", True):
L.append("\n**CVEs fixed by this upgrade: UNKNOWN — the scan could NOT determine a count.**")
L.append("\n⚠ This is NOT zero. A version-scheme change (e.g. semver → calver) makes numeric "
"ordering meaningless across this jump, so no advisory could be classified. Render "
"this recipe's cve cell as `?`, never `0`. Read the vendor's release notes for the "
"jump and count by hand.")
if rep.get("no_usable_sources"):
L.append("\n⚠ This is NOT zero. **No usable source was checked at all** — the registry "
"file `cc-ci-plan/upstream/<recipe>.md` is missing or every source failed, so "
"nothing was measured. Render this recipe's cve cell as `?`, never `0`, and add "
"the registry file.")
else:
L.append("\n⚠ This is NOT zero. A version-scheme change (e.g. semver → calver) makes "
"numeric ordering meaningless across this jump, so no advisory could be "
"classified. Render this recipe's cve cell as `?`, never `0`. Read the vendor's "
"release notes for the jump and count by hand.")
if rep["unclassified"]:
L.append(f"\nAdvisories seen but unclassifiable ({len(rep['unclassified'])}) — includes "
f"other images in this recipe: " + ", ".join(rep["unclassified"][:12]))