The 2026-08-07 regeneration rendered '?' for 5 of 21 recipes. '?' is meant to be a
rare 'we tried and could not tell'; at that rate it is indistinguishable from noise
and hides the real unknowns. Three causes, none of them genuine uncertainty:
1. URL EXTRACTION BUG (mine). The registry is markdown, so urls appear inside
`backticks` and 'quotes'. The extractor captured the trailing punctuation, so
it fetched https://docs.n8n.io/release-notes/` and https://git.autonomic.zone'`
— both 404 on the malformed url, both 200 when clean. Trailing markdown
punctuation is now stripped. Fixed immich + n8n.
2. STALE REGISTRY URL. mattermost-lts pointed at
docs.mattermost.com/about/mattermost-changelog.html, which 404s; the page moved
to /deploy/. Corrected (same class as the pgautoupgrade fix).
3. WRONG SEMANTICS FOR 'NO UPGRADE'. lasuite-docs and custom-html-tiny were
up-to-date this run, so no scan block existed and the report fell back to '?'.
But a recipe with no upgrade has nothing an upgrade could have fixed — that is
0, not unknown. The report skill now says so explicitly, restricts '?' to scans
that RAN and reported genuinely failed sources, states that benign notes
(no-advisories-published / template URL) never trigger '?', and instructs that
many '?' is itself a bug to raise in the Addendum.
Result across all 16 scanned recipes of that run: 0 failed sources (was 5).
Counts also improved with the classifier fix: discourse 130->133, keycloak ->7.
Exposed by asking whether the scan catches the n8n CVEs (CVE-2026-42231/42232). It
did not — the advisories were fetched correctly but both misclassified as
out-of-window. Two bugs:
1. Only vulnerabilities[0] was read. An advisory carries ONE ENTRY PER PATCHED
RELEASE LINE: n8n patches three (1.123.32, 2.17.4, 2.18.1), so whichever line
the deployment is actually on was silently dropped. gitea passed only because it
patches a single line. Now all entries are kept.
2. patched_versions is a RANGE EXPRESSION ('>= 2.18.1'), not a bare version. Naive
parsing produced (18,1) instead of (2,18,1), so no comparison could ever match.
Version tokens are now extracted with a regex and the advisory counts as
fixed-by-this-upgrade if ANY patched line falls in (from, to].
Verified: n8n 2.17.0 -> 2.18.1 now reports 12 CVEs including both criticals
(CVE-2026-42231 GHSA-q5f4-99jv-pgg5, CVE-2026-42232); gitea 1.27.0 -> 1.27.1 still
reports exactly 2. Note our deployed n8n (2.27.2+) is already past all three patched
lines, so these were never outstanding for us — the bug was in detection, not
exposure.
Two refinements found by running the scan across all 14 recipes of the 2026-08-07 run:
1. A repo with no advisory feed returns HTTP 404 on /security-advisories (e.g. the
pgautoupgrade sidecar image). That is a BENIGN ABSENCE, not a failed check.
Likewise registry entries that are TEMPLATE urls for humans
(…/changelog/v<VERSION>/, …/<vX.Y.Z>/…) are documentation, not fetchable.
Counting either as a failure pushed most recipes to '?', which would make the
unknown-vs-clean distinction meaningless again — the exact signal the ? exists to
preserve. Both are now recorded in sources_benign; only genuine errors (rate
limit, network, 5xx, wrong URL) land in sources_failed.
2. upstream/*.md pointed at github.com/pgautoupgrade/pgautoupgrade, which 404s —
the repo is pgautoupgrade/docker-pgautoupgrade. Corrected in n8n, lasuite-docs,
lasuite-drive, lasuite-meet. A 404ing registry URL means we were not scanning a
source we believed we were.
Effect on the 2026-08-07 data: recipes with genuine failed sources 5 -> 3 (the
remainder are really unreachable vendor pages). CVE counts unchanged where they
were already sound: discourse 130, gitea 2, plausible 1.
Anonymous GitHub API is 60 req/hr — a full weekly sweep across ~20 recipes exhausts
it and the scan then reports sources as failed (visible, but degraded coverage). A
token lifts it to 5000/hr.
_github_token(): GITHUB_TOKEN env wins, else GITHUB_TOKEN_FILE (default
/srv/cc-ci/.github-token, 0600, gitignored). Reading PUBLIC advisories needs NO
scopes — a classic PAT with nothing ticked, or fine-grained limited to 'Public
repositories: read'. The tool only ever GETs advisories; do not grant write scopes.
A missing token is not an error: the scan runs anonymously and surfaces failures.
Also gitignores .github-token and .hcloud-token.
Why: gitea 1.27.1 fixed CVE-2026-60004 + CVE-2026-59774 (both CVSS 9.8). The
2026-08-03 report printed gitea's CVE count as '1', the 2026-08-07 report as
'none'. Cause chain: the upgrade subagent read the GitHub release notes, which
name NEITHER cve (they are announced only in the vendor blog's security section),
so it recorded one unrelated minor item; the report then derived security content
from those notes plus model knowledge, and the model's training predates the CVEs.
Nothing in the pipeline ever queried an advisory source.
cc-ci-plan/advisory-scan.py — deterministic, per recipe, per upgrade window:
1. GitHub Security Advisories API for every source repo in the upstream registry.
PRIMARY: CVE + GHSA + severity + vulnerable/patched ranges, so 'fixed by THIS
upgrade' is computed. Needs no new per-recipe config (134 registry URLs are
already github.com).
2. Vendor release/security pages — every registry URL, fetched + regex-scanned.
This is the source that actually had the gitea CVEs.
3. OSV where a package mapping exists — supplementary.
Each source reports its own status so 'checked, none found' is never confused with
'not checked'. Source selection was measured, not assumed: for these two CVEs OSV
404'd and NVD's API had them by neither CPE, id, nor keyword — advisory DBs lag the
vendor, hence 1+2 lead.
Wiring is strictly ADDITIVE:
- /recipe-upgrade gains step 2a: run the scan, paste the block into the per-recipe
log, and report the UNION of it and the existing release-note reading. The scan
may never lower a count established by reading.
- /recipe-report treats the block as a FURTHER source, prefers its advisory ids /
severities / fixed-in versions for citation, and must render '?' (not 'none')
when a scan is absent or has failed sources — the false-clean 'none' is exactly
what happened on 2026-08-07.
- upstream/gitea.md records blog.gitea.com as the security-announcement URL.
Verified on the real regression: 1.27.0 -> 1.27.1 now yields exactly the 2 missed
criticals with their GHSA ids; the wider 1.26.2 -> 1.27.1 window yields 62.
autonomic-bot