audit-sources --security-sources: find the recipes that cannot see CVEs at all

Follow-up to the nginx blind spot. Sweeping all 22 recipes for sources whose CVEs
are USABLE (structured advisory feed, or a changelog attributable to releases)
rather than merely visible.

Before the changelog-attribution fix: 20 unusable sources. After: 5, and all five
are redundant - the same project also publishes an advisory feed (redis, gitea,
minio, clickhouse), so nothing is actually lost.

One real find, same shape as nginx: ONLYOFFICE/DocumentServer publishes NO GitHub
advisories, and the registry pointed its CHANGELOG.md at the GitHub *blob* page -
636KB of markup in which the release headings do not survive HTML-stripping, so 24
CVEs were visible and NONE attributable. The raw URL attributes all 24. Rather than
fix one registry line, advisory-scan now normalises github.com/../blob/.. to
raw.githubusercontent.com, which fixes every entry present and future.
lasuite-drive bumps documentserver, so this was live.

Genuinely blind after all that: mattermost-lts and mumble - no advisory feed, no
attributable changelog, no CVE data anywhere the registry points. mattermost is the
notable one: its bulletins are client-side rendered, so a regex sweep sees nothing.
Their scans can report 0 while nothing was measured, so /cve-check now renders those
recipes as ? and says why.

The audit output distinguishes a blind RECIPE from an unparseable PAGE, because
conflating them made 5 harmless redundancies look like 5 gaps.
This commit is contained in:
autonomic-bot
2026-08-11 20:02:06 +00:00
parent dab3edf3c2
commit 46ace30b4d
3 changed files with 94 additions and 1 deletions
+15 -1
View File
@@ -303,6 +303,20 @@ def _changelog_versions(text: str) -> dict:
return out
_BLOB_RE = re.compile(r"^https://github\.com/([^/]+)/([^/]+)/blob/(.+)$")
def _raw_if_blob(url: str) -> str:
"""A GitHub *blob* URL is an HTML viewer, not the file.
The registry pointed ONLYOFFICE's CHANGELOG.md at its blob page. Fetching that returns 636KB of
markup in which the release headings do not survive HTML-stripping, so 24 CVEs were visible and
NONE attributable to a release — the same shape of blind spot as nginx. The raw URL attributes
all 24. Normalising here fixes every registry entry at once, present and future."""
m = _BLOB_RE.match(url)
return f"https://raw.githubusercontent.com/{m.group(1)}/{m.group(2)}/{m.group(3)}" if m else url
def vendor_pages(urls: list[str]) -> list[dict]:
"""Fetch each registry URL and regex out CVE ids, with a little surrounding context."""
out = []
@@ -317,7 +331,7 @@ def vendor_pages(urls: list[str]) -> list[dict]:
continue
entry = {"source": u, "status": "ok", "cves": [], "context": {}, "fixed_in": {}}
try:
text = _fetch(u)
text = _fetch(_raw_if_blob(u))
plain = re.sub(r"<[^>]+>", " ", text)
for cve in sorted(set(CVE_RE.findall(plain))):
entry["cves"].append(cve)