audit-sources --security-sources: find the recipes that cannot see CVEs at all
Follow-up to the nginx blind spot. Sweeping all 22 recipes for sources whose CVEs are USABLE (structured advisory feed, or a changelog attributable to releases) rather than merely visible. Before the changelog-attribution fix: 20 unusable sources. After: 5, and all five are redundant - the same project also publishes an advisory feed (redis, gitea, minio, clickhouse), so nothing is actually lost. One real find, same shape as nginx: ONLYOFFICE/DocumentServer publishes NO GitHub advisories, and the registry pointed its CHANGELOG.md at the GitHub *blob* page - 636KB of markup in which the release headings do not survive HTML-stripping, so 24 CVEs were visible and NONE attributable. The raw URL attributes all 24. Rather than fix one registry line, advisory-scan now normalises github.com/../blob/.. to raw.githubusercontent.com, which fixes every entry present and future. lasuite-drive bumps documentserver, so this was live. Genuinely blind after all that: mattermost-lts and mumble - no advisory feed, no attributable changelog, no CVE data anywhere the registry points. mattermost is the notable one: its bulletins are client-side rendered, so a regex sweep sees nothing. Their scans can report 0 while nothing was measured, so /cve-check now renders those recipes as ? and says why. The audit output distinguishes a blind RECIPE from an unparseable PAGE, because conflating them made 5 harmless redundancies look like 5 gaps.
This commit is contained in:
@@ -303,6 +303,20 @@ def _changelog_versions(text: str) -> dict:
|
||||
return out
|
||||
|
||||
|
||||
_BLOB_RE = re.compile(r"^https://github\.com/([^/]+)/([^/]+)/blob/(.+)$")
|
||||
|
||||
|
||||
def _raw_if_blob(url: str) -> str:
|
||||
"""A GitHub *blob* URL is an HTML viewer, not the file.
|
||||
|
||||
The registry pointed ONLYOFFICE's CHANGELOG.md at its blob page. Fetching that returns 636KB of
|
||||
markup in which the release headings do not survive HTML-stripping, so 24 CVEs were visible and
|
||||
NONE attributable to a release — the same shape of blind spot as nginx. The raw URL attributes
|
||||
all 24. Normalising here fixes every registry entry at once, present and future."""
|
||||
m = _BLOB_RE.match(url)
|
||||
return f"https://raw.githubusercontent.com/{m.group(1)}/{m.group(2)}/{m.group(3)}" if m else url
|
||||
|
||||
|
||||
def vendor_pages(urls: list[str]) -> list[dict]:
|
||||
"""Fetch each registry URL and regex out CVE ids, with a little surrounding context."""
|
||||
out = []
|
||||
@@ -317,7 +331,7 @@ def vendor_pages(urls: list[str]) -> list[dict]:
|
||||
continue
|
||||
entry = {"source": u, "status": "ok", "cves": [], "context": {}, "fixed_in": {}}
|
||||
try:
|
||||
text = _fetch(u)
|
||||
text = _fetch(_raw_if_blob(u))
|
||||
plain = re.sub(r"<[^>]+>", " ", text)
|
||||
for cve in sorted(set(CVE_RE.findall(plain))):
|
||||
entry["cves"].append(cve)
|
||||
|
||||
Reference in New Issue
Block a user