autonomic-bot
65bf3c095b
The image name was packed into the value, so the flag needed a hand-rolled KEY=FROM:TO parser with its own malformed-input branch, and 'window' named the wrong thing — the tool has two kinds of window (version ranges and, on the date fallback, real date windows) and the flag meant only the first. Now each part is its own argument: --image redis 7.4 8.10, repeatable, all in one call. argparse enforces the arity, so the string parsing and its error path are deleted. 'windows' survives internally as the computed-range concept. Counts unchanged: discourse 128 with redis / 123 without, gitea 2.
197 lines
11 KiB
Markdown
197 lines
11 KiB
Markdown
# Advisory scan — specification
|
|
|
|
What `cc-ci-plan/advisory-scan.py` does, step by step, and why each step exists. This documents the
|
|
implementation as it stands (2026-08-10); if you change the code, change this file in the same commit.
|
|
|
|
**Role.** A deterministic, per-recipe CVE detector run as a **pre-step of `/recipe-upgrade`** (step 2a).
|
|
It is **strictly additive**: it never replaces the release-note reading the upgrade agent already does.
|
|
The CVE count reported for a recipe is the **union** of what the agent read and what this scan found;
|
|
the scan may never *lower* a count established by reading.
|
|
|
|
**Why it exists.** gitea 1.27.1 fixed CVE-2026-60004 and CVE-2026-59774 (both CVSS 9.8). The weekly
|
|
report printed gitea's CVE count as `1`, then `none`. The upgrade agent had read the GitHub *release
|
|
notes*, which name neither — both were announced only in the vendor's blog security section — and the
|
|
report then derived security content from those notes plus model knowledge, which predates the CVEs.
|
|
Nothing in the pipeline queried an advisory source. This scan closes that hole.
|
|
|
|
---
|
|
|
|
## Inputs
|
|
|
|
```
|
|
advisory-scan.py <recipe> [--from <version>] [--to <version>]
|
|
[--image <name> <from> <to>]... [--json] [--registry DIR]
|
|
```
|
|
|
|
| Input | Meaning |
|
|
|---|---|
|
|
| `<recipe>` | Recipe name; selects `cc-ci-plan/upstream/<recipe>.md` (the per-recipe URL registry) |
|
|
| `--from` / `--to` | The **primary app image's** version window being upgraded across |
|
|
| `--image NAME FROM TO` | A **sidecar image and the versions it moved between** (repeatable). `NAME` is matched as a substring against source repo names, e.g. `--image redis 7.4 8.10`. Without it that image's advisories stay unclassified. |
|
|
| `--registry` | Registry dir; also `CCCI_UPSTREAM_REGISTRY` |
|
|
| `GITHUB_TOKEN` / `GITHUB_TOKEN_FILE` | Read-only token; **rate limit only** (60/hr anonymous → 5000/hr). Default file `/srv/cc-ci/.github-token`, mode 600. Public advisories need **no scopes**. |
|
|
|
|
Exit code is always 0 — this is informational. Failures are *reported*, never raised.
|
|
|
|
---
|
|
|
|
## Step 1 — Collect source URLs from the registry
|
|
|
|
Read `cc-ci-plan/upstream/<recipe>.md` and extract every `http(s)://…` URL.
|
|
|
|
**Trailing markdown punctuation is stripped** (`` ` `` `'` `"` `*` `.` `,` `;` `:` `>` `)`). The registry
|
|
is markdown, so URLs appear inside backticks and quotes; capturing the punctuation produced fetches of
|
|
`https://docs.n8n.io/release-notes/\`` which 404, and made immich and n8n render `?` for no real reason.
|
|
|
|
> **Registry hygiene matters.** The scan can only look where the registry points. Two classes of defect
|
|
> have been found and fixed by running it: a **wrong URL** (`pgautoupgrade/pgautoupgrade`, which 404s —
|
|
> the repo is `pgautoupgrade/docker-pgautoupgrade`) and a **missing** one (gitea's CVEs are announced at
|
|
> `blog.gitea.com`, which the registry didn't list). When a vendor publishes security notes somewhere
|
|
> the registry lacks, add it.
|
|
|
|
## Step 2 — Query the sources
|
|
|
|
Three source classes, each recording **its own status** so *"checked, none found"* is never confused
|
|
with *"not checked"*.
|
|
|
|
### 2a. GitHub Security Advisories — PRIMARY
|
|
|
|
For every `github.com/<owner>/<repo>` URL in the registry:
|
|
`GET /repos/<owner>/<repo>/security-advisories`.
|
|
|
|
Captured per advisory: `cve_id`, `ghsa_id`, `severity`, `summary`, `published_at`, and **all**
|
|
`vulnerabilities[]` entries' `vulnerable_version_range` + `patched_versions` (joined with `;`).
|
|
|
|
- **All entries, not just the first.** An advisory carries one entry **per patched release line** —
|
|
n8n patches three (1.123.32, 2.17.4, 2.18.1). Reading only `vulnerabilities[0]` silently dropped the
|
|
line a deployment was actually on, and misclassified CVE-2026-42231/42232 as out-of-window.
|
|
- **Pagination via the `Link rel="next"` cursor**, to exhaustion (cap 20 hops). This endpoint returns
|
|
at most 100 rows **and ignores `?page=`** — it re-returns the same rows, which silently truncates busy
|
|
projects. discourse has 286 advisories; a single page cannot even cover one upgrade window.
|
|
- **HTTP 404 ⇒ `no-advisories-published`** — a benign absence (many sidecar images publish none), **not**
|
|
a failure. Conflating the two pushed nearly every recipe to `?` and destroyed the signal.
|
|
|
|
This source is primary because it carries **severity and version ranges**, making "fixed by *this*
|
|
upgrade" computable rather than guessed.
|
|
|
|
### 2b. Vendor release / security pages
|
|
|
|
Every other registry URL is fetched, HTML-stripped, and scanned for `CVE-\d{4}-\d{4,7}`, keeping ±160
|
|
characters of context per hit.
|
|
|
|
URLs containing `<`, `>`, `{`, `}`, `VERSION`, or `vX.Y.Z` are **skipped as templates** — they are
|
|
human documentation (`…/changelog/v<VERSION>/`), not fetchable, and counting them as failures is wrong.
|
|
|
|
This is the source that would have caught gitea: the vendor blog names both CVEs, the GitHub release
|
|
page names neither.
|
|
|
|
### 2c. OSV.dev — supplementary
|
|
|
|
Only when the recipe has an entry in `OSV_PACKAGES` (ecosystem + package) and a version is given.
|
|
|
|
> **Measured, not assumed.** For the two gitea CVEs, OSV **404'd on both** and returned only Go
|
|
> *dependency* advisories for the package; NVD's API had them by neither CPE, CVE id, nor keyword.
|
|
> **Advisory databases lag the vendor**, which is why 2a and 2b lead and this is supplementary.
|
|
|
|
## Step 3 — Union
|
|
|
|
All findings merge into one CVE map: id → `{sources[], severity, ghsa, vulnerable_range, patched,
|
|
published_at, context}`. A CVE seen by several sources keeps them all.
|
|
|
|
## Step 4 — Classify against the upgrade window
|
|
|
|
Two invariants govern this step, both learned from a wrong answer in production.
|
|
|
|
> **A. Every image is judged by its OWN versions.** The app repo uses `--from/--to`; each sidecar uses
|
|
> its own `--image NAME FROM TO`. **Pass them all in ONE invocation** — the count is a union across
|
|
> images, and the UNKNOWN guarantee in B only holds when a single run sees every one. An image with no window is **not** classified — its advisories are
|
|
> listed as unclassified so they stay visible without inflating the count. The reported count is the
|
|
> **union across windows**, and each window is classified independently (so one may use version
|
|
> ranges while another falls back to dates).
|
|
> *Why:* discourse once reported **133**, of which **34 were redis CVEs** — including
|
|
> `CVE-2021-21309`, patched in redis 6.0.11 in 2021 — counted purely because 6.0.11 sits numerically
|
|
> inside discourse's `3.5.3 → 2026.7.1` range. The fix is not to ignore sidecars but to give each one
|
|
> the versions it actually moved through: with `--image redis 7.4 8.10`, discourse scores
|
|
> **128 = 123 (app, by date) + 5 (redis, by version range)** — and the redis five include
|
|
> `CVE-2025-49844`, **critical**, which was invisible while sidecars went uncounted.
|
|
>
|
|
> **B. Never emit a number you cannot justify.** If neither method below can order the window, the
|
|
> count is `null` / `UNKNOWN`, never `0`. A `0` in a security column asserts safety.
|
|
|
|
### 4a. By patched version (preferred — exact)
|
|
|
|
`patched_versions` is a **range expression** (`">= 2.18.1"`), possibly several joined by `;`. Extract
|
|
every version-looking token; the advisory is **fixed-by-this-upgrade** if **any** patched version `p`
|
|
satisfies `from < p <= to`, using a loose numeric key (leading integers per dot-part).
|
|
|
|
### 4b. By advisory publish date (fallback — temporal)
|
|
|
|
Used **only** when 4a cannot be trusted: a **version-scheme change**, detected as the leading version
|
|
component jumping by ≥ `SCHEME_JUMP` (100) — e.g. semver `3.5.3` → calver `2026.7.1`.
|
|
|
|
Version strings are unorderable across such a jump (`2025.12.2` compares "newer" than `3.5.3` while
|
|
shipping earlier), but **release dates always order**. So:
|
|
|
|
1. Resolve `--from` and `--to` to **git tag dates** on the primary repo (tries `v<version>` then
|
|
`<version>`; annotated tag → tagger date, else commit date).
|
|
2. An advisory counts as fixed when `date_from < published_at <= date_to`.
|
|
|
|
Both the method and the resolved window appear in the output. This reproduces, automatically, the hand
|
|
count that established discourse `3.5.3` (2025-12-30) → `2026.7.1` (2026-07-31) = **123 CVEs**.
|
|
|
|
*Assumption:* the vendor publishes advisories at fix time (true for discourse). The count includes
|
|
**first-party plugin advisories** where the vendor files them on the same repo — which is why a
|
|
plugin-rich project scores far higher than a monolith, not a statement about relative security.
|
|
|
|
### 4c. Otherwise
|
|
|
|
`count_known = false`, `cve_count_fixed = null`, and the markdown headline reads
|
|
**"CVEs fixed by this upgrade: UNKNOWN — the scan could NOT determine a count"** with an explicit
|
|
*"This is NOT zero"*.
|
|
|
|
## Step 5 — Output
|
|
|
|
Markdown (default) for pasting into the per-recipe upgrade log, or `--json`.
|
|
|
|
| Field | Meaning |
|
|
|---|---|
|
|
| `cve_count_fixed` | Union across all windows, or **`null`** if ANY requested window could not be ordered (a partial number would understate) |
|
|
| `count_known` | Distinguishes "counted zero" from "could not count" |
|
|
| `windows` | Every source classified, with its from/to (the internal computed ranges) |
|
|
| `classified_by` | **Per source**: `patched version ranges` or `advisory publish date (version scheme changed)` |
|
|
| `date_window` | **Per source**, when 4b was used |
|
|
| `fixed_by_this_upgrade[]` | CVE ids, with severity / GHSA / fixed-in per id |
|
|
| `unclassified[]` | Seen but not attributable to this window (incl. other images) |
|
|
| `sources[]` | Every source with its own status |
|
|
| `sources_failed[]` | **Genuine** failures only |
|
|
| `sources_benign[]` | `no-advisories-published`, `skipped: template URL` |
|
|
|
|
---
|
|
|
|
## How consumers must read it
|
|
|
|
`/recipe-report` renders the `cve` column from the **union** of this scan and the agent's own reading:
|
|
|
|
- a clean scan → its number, **including `0`**;
|
|
- **failed sources**, or `UNKNOWN` → **`?`**, never `none` — a blank reads as "clean", which is exactly
|
|
how two CVSS-9.8 gitea RCEs were published as `none`;
|
|
- **no upgrade this run** → `0`, not `?` — nothing an upgrade could have fixed;
|
|
- benign notes → never `?`.
|
|
|
|
`?` must stay **rare**: it means *we tried and could not tell*, not *we did not look*. A rash of `?` is
|
|
a bug to raise in the report's Addendum, not a normal outcome — every instance so far traced to a defect
|
|
in this tool or stale registry data.
|
|
|
|
## Known limits
|
|
|
|
1. **Versions must be supplied per image.** An image with no `--image` is not counted — the scan will not
|
|
guess a version range it was not told. `/recipe-upgrade` passes one per image it bumped.
|
|
2. **Date-based counts are temporal**, not exact — they assume publish-at-fix-time.
|
|
3. **Registry-bound.** Unlisted vendor security pages are invisible; the scan cannot know what it was
|
|
never pointed at.
|
|
4. **`NAME` matching is substring-against-source-name**, so a short or generic name can attach to more
|
|
than one repo (`postgres` matches `discourse/discourse-postgres`). The primary source is claimed
|
|
first and cannot be stolen; be specific enough to hit one repo.
|
|
5. **Rate limit** without a token is 60/hr — a full weekly sweep will exhaust it and degrade to failed
|
|
sources (visibly, but degraded).
|