audit-sources: check we are still looking where releases actually happen

A recipe tracks an image repo and a set of registry URLs. When upstream moves, nothing errors — the old repo just stops receiving tags and the recipe looks 'up to date' forever. plausible is the case: it tracked plausible/analytics on Docker Hub while upstream moved to ghcr.io/plausible/community-edition. Every survey said 'no upgrades available' while v3 shipped elsewhere. audit-sources.py reports the signals that catch it, per image and per registry URL: image gone quiet (newest tag older than --quiet-days), deprecation wording in the registry description, and GitHub repos that are archived, renamed or gone. Signals, not verdicts — a stable image can be quiet for good reason — so each finding says what was measured. First run over 22 recipes, 11 findings, 4 alerts. It independently re-derived the plausible case (analytics quiet 1126 days), and found: - drone: harness/drone now answers as harness/harness (the image is fine) - lasuite-docs, lasuite-drive: minio/minio is ARCHIVED on GitHub - lasuite-docs: docspecio/api is ARCHIVED - matrix-synapse: halfshot/matrix-appservice-discord image quiet 2078 days - mumble: NO cc-ci-plan/upstream/mumble.md at all That last one exposed a scanner bug. With no registry file there is no source to query, yet the scan still printed '0 identified by the deterministic scan' — and that 0 was published as a clean count in the 2026-08-11 CVE check. A scan with no usable source has measured nothing and must not report a number, least of all 0. It now returns UNKNOWN and says the registry file is missing. upstream/mumble.md added; mumble now scans 6 sources for a genuine 0.
2026-08-11 15:02:06 +00:00
parent 6c91373357
commit e89da2d842
3 changed files with 282 additions and 6 deletions
@@ -797,8 +797,19 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
    report["unclassified"] = sorted(unknown)
    # NEVER report 0 for something we could not determine — a 0 asserts safety. If ANY requested
    # window could not be ordered at all, the total is UNKNOWN rather than a partial number.
-    report["count_known"] = not unresolved_any
-    report["cve_count_fixed"] = len(fixed_set) if not unresolved_any else None
+    # A scan with NO usable source has not measured anything, so it must not report a number —
+    # least of all 0, which asserts safety. mumble had no cc-ci-plan/upstream/mumble.md at all and
+    # still produced "0 identified", which was then published as a clean 0 in a CVE report.
+    usable_sources = [
+        s for s in report["sources"]
+        if s["status"] == "ok" or s["status"].startswith("no-advisories-published")
+    ]
+    no_sources = not usable_sources
+    if no_sources:
+        report["no_usable_sources"] = True
+    report["count_known"] = not unresolved_any and not no_sources
+    report["cve_count_fixed"] = (len(fixed_set)
+                                 if (not unresolved_any and not no_sources) else None)
    report["cve_count_total_seen"] = len(report["cves"])
    # Only GENUINE failures make a count unreliable. "no-advisories-published" (404: the repo has
    # no advisory feed) and "skipped: template URL" are benign and must not degrade the verdict.
@@ -821,10 +832,16 @@ def markdown(rep: dict) -> str:
         f"{rep.get('from') or '?'} → {rep.get('to') or '?'}"]
    if not rep.get("count_known", True):
        L.append("\n**CVEs fixed by this upgrade: UNKNOWN — the scan could NOT determine a count.**")
-        L.append("\n⚠ This is NOT zero. A version-scheme change (e.g. semver → calver) makes numeric "
-                 "ordering meaningless across this jump, so no advisory could be classified. Render "
-                 "this recipe's cve cell as `?`, never `0`. Read the vendor's release notes for the "
-                 "jump and count by hand.")
+        if rep.get("no_usable_sources"):
+            L.append("\n⚠ This is NOT zero. **No usable source was checked at all** — the registry "
+                     "file `cc-ci-plan/upstream/<recipe>.md` is missing or every source failed, so "
+                     "nothing was measured. Render this recipe's cve cell as `?`, never `0`, and add "
+                     "the registry file.")
+        else:
+            L.append("\n⚠ This is NOT zero. A version-scheme change (e.g. semver → calver) makes "
+                     "numeric ordering meaningless across this jump, so no advisory could be "
+                     "classified. Render this recipe's cve cell as `?`, never `0`. Read the vendor's "
+                     "release notes for the jump and count by hand.")
        if rep["unclassified"]:
            L.append(f"\nAdvisories seen but unclassifiable ({len(rep['unclassified'])}) — includes "
                     f"other images in this recipe: " + ", ".join(rep["unclassified"][:12]))
@@ -0,0 +1,238 @@
+#!/usr/bin/env python3
+"""audit-sources — are we still looking in the right place for each recipe's updates?
+
+A recipe tracks an image repo and a set of registry URLs. Upstreams move: they rename the image,
+switch registry, archive the GitHub repo, or split a community edition out of the original. When that
+happens nothing errors — the old repo simply stops receiving tags, and the recipe looks "up to date"
+forever while real releases happen somewhere else.
+
+plausible is the worked example. It tracked `plausible/analytics` on Docker Hub; upstream moved to
+`ghcr.io/plausible/community-edition`. The old repo still exists and still serves v2.0.0, so every
+survey said "no upgrades available" while v3 shipped elsewhere.
+
+This reports the signals that catch that, per image and per registry URL:
+
+  * IMAGE GONE QUIET     — newest tag is older than --quiet-days (default 365). The single strongest
+                           signal that releases moved somewhere else.
+  * DEPRECATION WORDING  — the registry description says deprecated / moved / no longer maintained.
+  * GITHUB REPO ARCHIVED — upstream archived it.
+  * GITHUB REPO RENAMED  — the API redirects to a different owner/name than we ask for.
+  * GITHUB REPO GONE     — 404.
+
+Everything is a SIGNAL, not a verdict: a genuinely stable image (mumble, custom-html) can be quiet
+for good reason. The output is for a human to judge, so each finding says what was measured.
+
+    audit-sources.py [recipe ...] [--ssh HOST] [--quiet-days N] [--json]
+"""
+
+from __future__ import annotations
+
+import argparse
+import importlib.util
+import json
+import os
+import re
+import sys
+import urllib.error
+import urllib.request
+from datetime import datetime, timezone
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+_spec = importlib.util.spec_from_file_location("resolve_images", os.path.join(HERE, "resolve-images.py"))
+RI = importlib.util.module_from_spec(_spec)
+_spec.loader.exec_module(RI)
+
+REGISTRY_DIR = os.environ.get("CCCI_UPSTREAM_REGISTRY", os.path.join(HERE, "upstream"))
+USED_RECIPES = os.path.join(HERE, "used-recipes.md")
+DEPRECATION_RE = re.compile(
+    r"\b(deprecat|no longer maintain|unmaintained|superseded|moved to|migrated to|"
+    r"has moved|discontinued|end.of.life|archived)\b", re.I)
+
+
+def _days_since(iso: str | None) -> int | None:
+    if not iso:
+        return None
+    try:
+        d = datetime.fromisoformat(iso.replace("Z", "+00:00"))
+    except ValueError:
+        return None
+    return (datetime.now(timezone.utc) - d).days
+
+
+def hub_repo_meta(repo: str) -> dict:
+    """Docker Hub repo metadata: when it was last pushed to, and how it describes itself."""
+    try:
+        d = RI._json(f"https://hub.docker.com/v2/repositories/{repo}", RI._hub_auth())
+    except urllib.error.HTTPError as e:
+        return {"status": f"HTTP {e.code}"}
+    except Exception as e:  # noqa: BLE001
+        return {"status": f"{type(e).__name__}"}
+    text = f"{d.get('description') or ''}\n{d.get('full_description') or ''}"
+    m = DEPRECATION_RE.search(text)
+    return {"status": "ok", "last_updated": d.get("last_updated"),
+            "deprecation_hint": (m.group(0) if m else None),
+            "archived": bool(d.get("is_archived") or d.get("status") == "inactive")}
+
+
+def github_repo_meta(owner: str, repo: str) -> dict:
+    """GitHub repo state — archived, renamed (the API answers with the CURRENT full_name), or gone."""
+    hdrs = {"Accept": "application/vnd.github+json"}
+    tok = RI._gh_token()
+    if tok:
+        hdrs["Authorization"] = f"Bearer {tok}"
+    try:
+        d = RI._json(f"https://api.github.com/repos/{owner}/{repo}", hdrs)
+    except urllib.error.HTTPError as e:
+        return {"status": f"HTTP {e.code}"}
+    except Exception as e:  # noqa: BLE001
+        return {"status": f"{type(e).__name__}"}
+    asked, got = f"{owner}/{repo}".lower(), (d.get("full_name") or "").lower()
+    return {"status": "ok", "archived": bool(d.get("archived")), "pushed_at": d.get("pushed_at"),
+            "renamed_to": (d.get("full_name") if got and got != asked else None),
+            "description": d.get("description") or ""}
+
+
+def newest_tag_date(registry: str, repo: str, tag: str) -> str | None:
+    """When was the repo's newest same-shape tag pushed? Docker Hub only (it dates its tags)."""
+    if registry not in ("docker.io", "registry-1.docker.io"):
+        return None
+    try:
+        d = RI._json(f"https://hub.docker.com/v2/repositories/{repo}/tags"
+                     f"?page_size=100&ordering=last_updated", RI._hub_auth())
+    except Exception:  # noqa: BLE001
+        return None
+    want = RI.shape(tag)
+    for row in d.get("results", []):
+        if RI.shape(row.get("name") or "") == want:
+            return row.get("last_updated")
+    return (d.get("results") or [{}])[0].get("last_updated")
+
+
+def audit_recipe(recipe: str, ssh: str | None, quiet_days: int) -> dict:
+    out = {"recipe": recipe, "findings": [], "images": [], "sources": []}
+    try:
+        refs = (RI.compose_images_ssh(recipe, ssh, "~/.abra/recipes") if ssh
+                else RI.compose_images(recipe, RI.RECIPE_DIR))
+    except Exception as e:  # noqa: BLE001
+        out["findings"].append({"level": "error", "what": f"could not read compose: {e}"})
+        return out
+
+    for ref in refs:
+        if "${" in ref:
+            continue
+        info = RI.parse_ref(ref)
+        row = {"ref": ref, "registry": info["registry"], "repo": info["repo"], "tag": info["tag"]}
+        if info["registry"] in ("docker.io", "registry-1.docker.io"):
+            meta = hub_repo_meta(info["repo"])
+            row.update(meta)
+            newest = newest_tag_date(info["registry"], info["repo"], info["tag"])
+            row["newest_tag_pushed"] = newest
+            age = _days_since(newest)
+            row["newest_tag_age_days"] = age
+            if age is not None and age > quiet_days:
+                out["findings"].append({
+                    "level": "warn", "what": "image has gone quiet",
+                    "detail": f"{info['repo']}: newest {RI.shape(info['tag'])}-shaped tag pushed "
+                              f"{age} days ago — releases may have moved elsewhere"})
+            if meta.get("deprecation_hint"):
+                out["findings"].append({
+                    "level": "warn", "what": "registry text suggests deprecation",
+                    "detail": f"{info['repo']}: says {meta['deprecation_hint']!r}"})
+            if meta.get("archived"):
+                out["findings"].append({"level": "warn", "what": "registry repo archived/inactive",
+                                        "detail": info["repo"]})
+        out["images"].append(row)
+
+    urls, reg_path = ([], None)
+    try:
+        urls, reg_path = _registry_urls(recipe)
+    except Exception:  # noqa: BLE001
+        pass
+    if reg_path is None:
+        out["findings"].append({"level": "warn", "what": "no upstream registry file",
+                                "detail": f"cc-ci-plan/upstream/{recipe}.md is missing — the advisory "
+                                          f"scan has nowhere to look"})
+    seen = set()
+    for u in urls:
+        m = re.match(r"https?://github\.com/([^/]+)/([^/#?]+)", u)
+        if not m:
+            continue
+        owner, repo = m.group(1), m.group(2).removesuffix(".git")
+        if (owner, repo) in seen:
+            continue
+        seen.add((owner, repo))
+        meta = github_repo_meta(owner, repo)
+        row = {"repo": f"{owner}/{repo}", **meta}
+        age = _days_since(meta.get("pushed_at"))
+        row["pushed_age_days"] = age
+        out["sources"].append(row)
+        if meta.get("status") != "ok":
+            out["findings"].append({"level": "warn", "what": "registry source unreachable",
+                                    "detail": f"{owner}/{repo}: {meta['status']}"})
+            continue
+        if meta.get("renamed_to"):
+            out["findings"].append({"level": "alert", "what": "GitHub repo has MOVED",
+                                    "detail": f"{owner}/{repo} now answers as {meta['renamed_to']}"})
+        if meta.get("archived"):
+            out["findings"].append({"level": "alert", "what": "GitHub repo is ARCHIVED",
+                                    "detail": f"{owner}/{repo} — upstream development has stopped here"})
+        if age is not None and age > quiet_days:
+            out["findings"].append({"level": "warn", "what": "GitHub repo quiet",
+                                    "detail": f"{owner}/{repo}: last push {age} days ago"})
+    return out
+
+
+def _registry_urls(recipe: str):
+    path = os.path.join(REGISTRY_DIR, f"{recipe}.md")
+    if not os.path.exists(path):
+        return [], None
+    text = open(path).read()
+    urls = []
+    for u in re.findall(r"https?://[^\s)|\]]+", text):
+        u = u.rstrip("`'\"*.,;:>)")
+        if u and u not in urls:
+            urls.append(u)
+    return urls, path
+
+
+def all_recipes() -> list[str]:
+    out = []
+    for ln in open(USED_RECIPES):
+        ln = ln.strip()
+        if not ln or ln.startswith("#") or ln.startswith("`"):
+            continue
+        parts = ln.split()
+        if len(parts) >= 2 and parts[1] in ("weekly", "external"):
+            out.append(parts[0])
+    return out
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__,
+                                 formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("recipes", nargs="*")
+    ap.add_argument("--ssh", default=None)
+    ap.add_argument("--quiet-days", type=int, default=365)
+    ap.add_argument("--json", action="store_true")
+    a = ap.parse_args()
+
+    recipes = a.recipes or all_recipes()
+    reports = [audit_recipe(r, a.ssh, a.quiet_days) for r in recipes]
+    if a.json:
+        print(json.dumps(reports, indent=2))
+        return 0
+    alerts = 0
+    for rep in reports:
+        fs = rep["findings"]
+        mark = "OK " if not fs else ("!! " if any(f["level"] == "alert" for f in fs) else " ? ")
+        print(f"{mark} {rep['recipe']}")
+        for f in fs:
+            alerts += f["level"] == "alert"
+            print(f"      [{f['level']}] {f['what']}: {f.get('detail','')}")
+    print(f"\n{len(reports)} recipes audited · "
+          f"{sum(len(r['findings']) for r in reports)} findings · {alerts} alerts")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
@@ -0,0 +1,21 @@
+# Upstream sources — mumble
+
+| service | image | source repo | releases / changelog |
+|---------|-------|-------------|----------------------|
+| app | mumblevoip/mumble-server | https://github.com/mumble-voip/mumble | https://github.com/mumble-voip/mumble/releases |
+| web | rankenstein/mumble-web | https://github.com/rankenstein/mumble-web | https://github.com/rankenstein/mumble-web/releases |
+
+## Standing notes
+- This file was **missing entirely** until 2026-08-11. Without it the advisory scan had no source to
+  query, and still printed "0 identified by the deterministic scan" — which was then published as a
+  clean `0` in the 2026-08-11 CVE check. The scan now refuses to emit a count when it has no usable
+  source (it reports UNKNOWN), and `audit-sources.py` flags a missing registry file directly.
+- `mumblevoip/mumble-server` tracks the upstream server releases and DOES publish GitHub security
+  advisories, so it is the recipe's primary CVE source.
+- `rankenstein/mumble-web` is a **fork** of the original `Johni0702/mumble-web`, which has been
+  dormant since 2023-05. The fork itself last pushed 2023-07 and its Docker tag `0.5` was last built
+  well over five years ago. Neither is archived, but treat the web client as effectively unmaintained:
+  if a CVE lands there, expect no upstream fix and plan a replacement rather than an upgrade.
+- The server image tag is `v<version>-<build>` (e.g. `v1.6.870-4`); the trailing number is the image
+  build, not an app version, and moves independently of upstream releases — `abra recipe upgrade`
+  reports "no new versions" for it, so use `resolve-images.py` to see those bumps.