Commit Graph
1 Commits
Author SHA1 Message Date
autonomic-bot 44cb9b6704 advisory-scan: tests, audit, and two real undercounts they found
Adds test-advisory-scan.py (58 offline tests on fixtures + 6 live regressions
against the week-2026-08-07 report) and audit-advisory-scan.py, which re-derives
every count with a SEPARATE semver implementation and its own release fetch and
diffs against the scanner. Both found real defects:

1. Window membership was compared on ragged tuples, so (18,) < (18,0) — a CVE
   patched in 18.0 fell OUTSIDE a window ending at 18. Bare major tags are the
   norm for sidecars (postgres:18, redis:8-alpine). Now zero-padded, which also
   keeps the upper bound conservative (18.5 stays out of a window ending at 18).

2. Advisories with no knowable fix version were silently counted as 'not fixed'.
   Twelve redis advisories say patched_versions 'TBD' or '7.4.X' with an
   open-ended range — six of them high severity. They are now INDETERMINATE:
   not counted, not dismissed, and surfaced in the output.

   All twelve turned out to be genuinely fixed: redis names each in the release
   notes of every branch that got the fix (CVE-2025-32023 -> 6.2.19, 7.2.10,
   7.4.5, 8.0.3, 8.2.0). So a third deterministic method resolves them from
   release notes, with the naming tags recorded as the citation. discourse's
   redis contribution goes 5 -> 17, and its total 128 -> 140.

Pass 2 (--adjudicate) is the model-judged stage for what arithmetic cannot
settle: it hands over each open case's full evidence, plus every verdict pass 1
reached, and takes FIXED/NOT-FIXED/STILL-UNKNOWN with a reason citing that
evidence. It may only raise a count. Vendor-page-only CVEs — the shape of both
gitea CVSS-9.8 RCEs — now reach it instead of being dropped.

Tests cover pass 1 only, by design; pass 2's judgement is a model's. What is
tested there is deterministic: which cases it selects, and that truncation is
announced rather than silent.

SPEC.md rewritten around the two passes.
2026-08-11 01:29:11 +00:00