Operator: 'the scanner should not say 0 when it was not able to scan.' Correct — the
previous patch still led with '0 identified' and relegated the caveat to a footnote,
so the headline number was wrong even though the prose was right. A 0 in a security
column is an assertion of safety; it must never be emitted for an undetermined result.
Now: cve_count_fixed is null (not 0) in JSON, a count_known flag distinguishes
'counted zero' from 'could not count', and the markdown headline reads
'CVEs fixed by this upgrade: UNKNOWN — the scan could NOT determine a count' with an
explicit 'This is NOT zero' and instructions to render '?'.
Verified: discourse 3.5.3 -> 2026.7.1 (semver->calver) now reports UNKNOWN; gitea
1.27.0 -> 1.27.1 still reports 2.
Operator disbelieved discourse's '133 CVEs fixed' — correctly. Two defects made it
confidently wrong:
1. ONE WINDOW APPLIED TO EVERY IMAGE. The scan queries all source repos in the
recipe's registry (app + redis/postgres/nginx sidecars) but judged them all with
the APP's version window. 34 of the 133 were redis advisories, including
CVE-2021-21309 — patched in redis 6.0.11 back in 2021 — scored as 'fixed by this
upgrade' purely because 6.0.11 sits numerically inside discourse's 3.5.3 ->
2026.7.1 range. Only the PRIMARY app repo is now classified; other sources are
reported as unclassified so they stay visible without inflating the count.
2. VERSION-SCHEME CHANGES BREAK ORDERING. discourse moved semver -> calver
(3.5.3 -> 2026.7.1), so 2025.12.2 compares 'newer' than 3.5.3 while shipping
earlier. Numeric comparison cannot order that. The scan now detects a leading-
component jump >= 100, refuses to classify, and says so in the block: the count
is '0 by refusal, not by evidence — read the vendor's release notes'.
Refusing to answer beats answering wrongly: a fabricated 133 in a public security
report is worse than an explicit 'cannot determine'.
Verified after the fix: discourse 133 -> 0 (with the refusal caveat), gitea still
exactly 2 (both criticals, patched 1.27.1), keycloak 7 all genuinely from
keycloak/keycloak patched in 26.7.1, plausible 1. No other count changed.
The 2026-08-07 regeneration rendered '?' for 5 of 21 recipes. '?' is meant to be a
rare 'we tried and could not tell'; at that rate it is indistinguishable from noise
and hides the real unknowns. Three causes, none of them genuine uncertainty:
1. URL EXTRACTION BUG (mine). The registry is markdown, so urls appear inside
`backticks` and 'quotes'. The extractor captured the trailing punctuation, so
it fetched https://docs.n8n.io/release-notes/` and https://git.autonomic.zone'`
— both 404 on the malformed url, both 200 when clean. Trailing markdown
punctuation is now stripped. Fixed immich + n8n.
2. STALE REGISTRY URL. mattermost-lts pointed at
docs.mattermost.com/about/mattermost-changelog.html, which 404s; the page moved
to /deploy/. Corrected (same class as the pgautoupgrade fix).
3. WRONG SEMANTICS FOR 'NO UPGRADE'. lasuite-docs and custom-html-tiny were
up-to-date this run, so no scan block existed and the report fell back to '?'.
But a recipe with no upgrade has nothing an upgrade could have fixed — that is
0, not unknown. The report skill now says so explicitly, restricts '?' to scans
that RAN and reported genuinely failed sources, states that benign notes
(no-advisories-published / template URL) never trigger '?', and instructs that
many '?' is itself a bug to raise in the Addendum.
Result across all 16 scanned recipes of that run: 0 failed sources (was 5).
Counts also improved with the classifier fix: discourse 130->133, keycloak ->7.
Exposed by asking whether the scan catches the n8n CVEs (CVE-2026-42231/42232). It
did not — the advisories were fetched correctly but both misclassified as
out-of-window. Two bugs:
1. Only vulnerabilities[0] was read. An advisory carries ONE ENTRY PER PATCHED
RELEASE LINE: n8n patches three (1.123.32, 2.17.4, 2.18.1), so whichever line
the deployment is actually on was silently dropped. gitea passed only because it
patches a single line. Now all entries are kept.
2. patched_versions is a RANGE EXPRESSION ('>= 2.18.1'), not a bare version. Naive
parsing produced (18,1) instead of (2,18,1), so no comparison could ever match.
Version tokens are now extracted with a regex and the advisory counts as
fixed-by-this-upgrade if ANY patched line falls in (from, to].
Verified: n8n 2.17.0 -> 2.18.1 now reports 12 CVEs including both criticals
(CVE-2026-42231 GHSA-q5f4-99jv-pgg5, CVE-2026-42232); gitea 1.27.0 -> 1.27.1 still
reports exactly 2. Note our deployed n8n (2.27.2+) is already past all three patched
lines, so these were never outstanding for us — the bug was in detection, not
exposure.
Two refinements found by running the scan across all 14 recipes of the 2026-08-07 run:
1. A repo with no advisory feed returns HTTP 404 on /security-advisories (e.g. the
pgautoupgrade sidecar image). That is a BENIGN ABSENCE, not a failed check.
Likewise registry entries that are TEMPLATE urls for humans
(…/changelog/v<VERSION>/, …/<vX.Y.Z>/…) are documentation, not fetchable.
Counting either as a failure pushed most recipes to '?', which would make the
unknown-vs-clean distinction meaningless again — the exact signal the ? exists to
preserve. Both are now recorded in sources_benign; only genuine errors (rate
limit, network, 5xx, wrong URL) land in sources_failed.
2. upstream/*.md pointed at github.com/pgautoupgrade/pgautoupgrade, which 404s —
the repo is pgautoupgrade/docker-pgautoupgrade. Corrected in n8n, lasuite-docs,
lasuite-drive, lasuite-meet. A 404ing registry URL means we were not scanning a
source we believed we were.
Effect on the 2026-08-07 data: recipes with genuine failed sources 5 -> 3 (the
remainder are really unreachable vendor pages). CVE counts unchanged where they
were already sound: discourse 130, gitea 2, plausible 1.
Anonymous GitHub API is 60 req/hr — a full weekly sweep across ~20 recipes exhausts
it and the scan then reports sources as failed (visible, but degraded coverage). A
token lifts it to 5000/hr.
_github_token(): GITHUB_TOKEN env wins, else GITHUB_TOKEN_FILE (default
/srv/cc-ci/.github-token, 0600, gitignored). Reading PUBLIC advisories needs NO
scopes — a classic PAT with nothing ticked, or fine-grained limited to 'Public
repositories: read'. The tool only ever GETs advisories; do not grant write scopes.
A missing token is not an error: the scan runs anonymously and surfaces failures.
Also gitignores .github-token and .hcloud-token.
Why: gitea 1.27.1 fixed CVE-2026-60004 + CVE-2026-59774 (both CVSS 9.8). The
2026-08-03 report printed gitea's CVE count as '1', the 2026-08-07 report as
'none'. Cause chain: the upgrade subagent read the GitHub release notes, which
name NEITHER cve (they are announced only in the vendor blog's security section),
so it recorded one unrelated minor item; the report then derived security content
from those notes plus model knowledge, and the model's training predates the CVEs.
Nothing in the pipeline ever queried an advisory source.
cc-ci-plan/advisory-scan.py — deterministic, per recipe, per upgrade window:
1. GitHub Security Advisories API for every source repo in the upstream registry.
PRIMARY: CVE + GHSA + severity + vulnerable/patched ranges, so 'fixed by THIS
upgrade' is computed. Needs no new per-recipe config (134 registry URLs are
already github.com).
2. Vendor release/security pages — every registry URL, fetched + regex-scanned.
This is the source that actually had the gitea CVEs.
3. OSV where a package mapping exists — supplementary.
Each source reports its own status so 'checked, none found' is never confused with
'not checked'. Source selection was measured, not assumed: for these two CVEs OSV
404'd and NVD's API had them by neither CPE, id, nor keyword — advisory DBs lag the
vendor, hence 1+2 lead.
Wiring is strictly ADDITIVE:
- /recipe-upgrade gains step 2a: run the scan, paste the block into the per-recipe
log, and report the UNION of it and the existing release-note reading. The scan
may never lower a count established by reading.
- /recipe-report treats the block as a FURTHER source, prefers its advisory ids /
severities / fixed-in versions for citation, and must render '?' (not 'none')
when a scan is absent or has failed sources — the false-clean 'none' is exactly
what happened on 2026-08-07.
- upstream/gitea.md records blog.gitea.com as the security-announcement URL.
Verified on the real regression: 1.27.0 -> 1.27.1 now yields exactly the 2 missed
criticals with their GHSA ids; the wider 1.26.2 -> 1.27.1 window yields 62.
autonomic-bot