Compare commits

..
Author SHA1 Message Date
autonomic-bot 69d1840ea5 upstream(lasuite-docs): note minio Docker images stopped at 2025-09-07 2026-08-14 03:06:49 +00:00
autonomic-bot 74117c2260 cve-check: record the remedy for a blind recipe, not just the symptom
The skill said to render a sourceless recipe as '?'. It now says how to stop it
being sourceless: declare an NVD CPE in the registry. That is what took the fleet
from two blind recipes to zero, and it is the first thing to try when the sweep
flags another.
2026-08-11 22:16:52 +00:00
autonomic-bot 985dc06e47 advisory-scan: NVD by CPE, so mattermost and mumble stop scanning as '?'
Two recipes could not see CVEs at all. mattermost-lts has an empty GitHub advisory
feed and renders its security bulletins client-side, so a text sweep finds nothing;
mumble publishes nothing anywhere the registry points. Both returned '?' - nothing
measured - which is honest but useless.

NVD is CPE-indexed and carries structured version ranges, so it answers where the
vendor does not. Declared per recipe as 'nvd-cpe: <image> = <cpe:2.3:...>'.

  mattermost-lts 10.5.0  -> 10.12.4   165 CVEs
  mattermost-lts 10.11.22 -> 10.12.4    0 CVEs  (measured, not unknown)
  mumble         1.3.0   -> 1.6.870      2 CVEs

Both NVD range forms are used: versionEndExcluding is a patched version;
versionEndIncluding means the fix version is unpublished but the upgrade delivers
it whenever it crosses X.

That 0 for the actual mattermost upgrade is the interesting one, and it needed a
new rule to be correct: a fix on the line you upgrade FROM was already yours.
mattermost patches every maintained line at once, so 10.11.22 -> 10.12.4 crosses
10.12.1 while 10.11.22 already had the 10.11.4 backport. Without the rule the scan
claimed 12 CVEs the upgrade did not deliver.

The rule is skipped for placeholders: '7.4.X' parses to a bare 7.4 and would read
as 'already fixed at 7.4', which silently dropped redis CVE-2024-46981 and took
discourse 140 -> 139 before I caught it.

79 tests. discourse 140 / gitea 2 / mailu 2 / keycloak 12 / plausible 6 unchanged.
Fleet sweep: 0 recipes with no usable CVE source, down from 2.
2026-08-11 22:16:37 +00:00
autonomic-bot 4b9978ac02 Merge pull request 'audit-sources --security-sources: find the recipes that cannot see CVEs at all' (#10) from feat/audit-security-sources into main 2026-08-11 20:02:24 +00:00
autonomic-bot 46ace30b4d audit-sources --security-sources: find the recipes that cannot see CVEs at all
Follow-up to the nginx blind spot. Sweeping all 22 recipes for sources whose CVEs
are USABLE (structured advisory feed, or a changelog attributable to releases)
rather than merely visible.

Before the changelog-attribution fix: 20 unusable sources. After: 5, and all five
are redundant - the same project also publishes an advisory feed (redis, gitea,
minio, clickhouse), so nothing is actually lost.

One real find, same shape as nginx: ONLYOFFICE/DocumentServer publishes NO GitHub
advisories, and the registry pointed its CHANGELOG.md at the GitHub *blob* page -
636KB of markup in which the release headings do not survive HTML-stripping, so 24
CVEs were visible and NONE attributable. The raw URL attributes all 24. Rather than
fix one registry line, advisory-scan now normalises github.com/../blob/.. to
raw.githubusercontent.com, which fixes every entry present and future.
lasuite-drive bumps documentserver, so this was live.

Genuinely blind after all that: mattermost-lts and mumble - no advisory feed, no
attributable changelog, no CVE data anywhere the registry points. mattermost is the
notable one: its bulletins are client-side rendered, so a regex sweep sees nothing.
Their scans can report 0 while nothing was measured, so /cve-check now renders those
recipes as ? and says why.

The audit output distinguishes a blind RECIPE from an unparseable PAGE, because
conflating them made 5 harmless redundancies look like 5 gaps.
2026-08-11 20:02:06 +00:00
autonomic-bot dab3edf3c2 Merge pull request 'advisory-scan: attribute vendor-changelog CVEs to the release that fixed them' (#9) from feat/changelog-version-attribution into main 2026-08-11 19:50:00 +00:00
autonomic-bot db37f1618b advisory-scan: attribute vendor-changelog CVEs to the release that fixed them
nginx publishes NO GitHub security advisories. Every nginx CVE we can see comes
from nginx.org/en/CHANGES, and the scan scraped ids out of it without attributing
them to a release - so they had no patched version, could never be classified, and
every nginx bump in the fleet reported 0 CVEs. nginx is a sidecar in most recipes,
so this was a fleet-wide blind spot.

Measured on the two PRs that prompted the question:
  lasuite-docs#7  nginx 1.31.1 -> 1.31.3   0 -> 6 CVEs
  lasuite-drive#6 nginx 1.31.2 -> 1.31.3   0 -> 3 CVEs
matching a hand count of the changelog exactly (three fixed in 1.31.2, three in
1.31.3; the narrower window correctly counts only the latter).

How: when a vendor page is organised by release, each CVE is attributed to the
nearest preceding release heading ('Changes with nginx 1.31.3', '## v1.31.3'),
and that becomes its fixed-in version. The CVE is tied to a window by the image
name appearing in the page URL (window 'nginx' <-> nginx.org/...). A changelog
lists the project's whole history, so only releases the window actually crosses
count - asserted by a test that the 2013 entries stay out.

76 tests. discourse 140 / gitea 2 / mailu 2 / keycloak 12 unchanged.
2026-08-11 19:49:43 +00:00
autonomic-bot 4bad1ea6db Merge pull request 'advisory-scan: derive the scan windows from a compose diff (--compose-to)' (#8) from feat/advisory-scan-compose into main 2026-08-11 19:24:46 +00:00
autonomic-bot ef58e33102 advisory-scan: derive windows from a compose diff (--compose-to)
Typing --from/--to/--image by hand means someone has to remember the recipe also
bumped its redis. That is how sidecar CVEs went uncounted for months. Point this
at a PR's compose.yml and it reads the windows off the diff instead.

  advisory-scan.py plausible --compose-to <.../branch/<pr>/compose.yml>
    -> community-edition: v2.0.0 -> v3.2.1
    -> clickhouse-server: 23.4.2.11-alpine -> 24.12-alpine
    -> 6 CVEs, identical to the hand-specified args

Details that mattered:

- keyed by SERVICE, not image repo. plausible moved plausible/analytics ->
  ghcr.io/plausible/community-edition; keyed by repo that reads as one image
  vanishing and an unrelated one appearing, and the app window - the one carrying
  the critical - is lost entirely.
- the baseline is the repo's DEFAULT BRANCH resolved from the API, never assumed
  to be main, because several recipes keep a stale main beside a live master.
- image names are matched against advisory sources BOTH ways: an image name is
  often longer than its source repo (clickhouse/clickhouse-server vs
  ClickHouse/ClickHouse) and sometimes shorter (redis vs redis/redis). One
  direction silently dropped the clickhouse window.
- credentials go in an Authorization header, never the URL: in-URL creds leak
  into shell history and process lists, and urllib mis-parses a password
  containing a colon.

--from/--to/--image remain for finer-grained checks (scanning a window that is
not a literal compose diff). 71 tests; discourse 140 / gitea 2 / mailu 2
unchanged.
2026-08-11 19:24:24 +00:00
autonomic-bot c352ea9058 Merge pull request 'add /cc-ci-cleanup — reconcile, close dead PRs, report what blocks the rest' (#7) from skill/cc-ci-cleanup into main 2026-08-11 19:17:19 +00:00
autonomic-bot 6ebc35bc18 Merge pull request 'AGENTS.md: ship infra work as PRs, self-merge, operator reviews retrospectively' (#6) from policy/pr-then-merge into main 2026-08-11 19:08:55 +00:00
autonomic-bot 96c536f543 AGENTS.md: ship infra work as PRs, self-merge, operator reviews retrospectively
Operator policy (2026-08-11). For cc-ci-orchestrator and cc-ci: branch, open a PR
whose description is written to be read AFTER the fact, merge it yourself once
verified, and let the operator review retrospectively. The PR is not a gate — it
is how the work stays legible — so a description that says 'fix scanner' has
failed at its only job.

Explicitly does NOT extend to recipe repos: those are created and verified but
never agent-merged, because they change what deploys on other people's
infrastructure.

Also records what to do when work has already landed on main without a PR: pin a
branch at the pre-work commit and PR against that, rather than rewriting
published history.
2026-08-11 19:08:39 +00:00
9 changed files with 730 additions and 9 deletions
+21
View File
@@ -104,6 +104,27 @@ CRITICAL came from, and an image with no window is not counted at all.
distinction was the difference between two false zeros and the truth (both recipes turned out fine,
but nothing in the survey said so).
### 2c. Know which recipes CANNOT see CVEs at all
```
python3 cc-ci-plan/audit-sources.py --security-sources
```
A recipe whose sources yield **no CVE data at all** cannot produce a meaningful `0` — nothing was
measured, the same way a missing registry file cannot. Render those as **`?`**, not `0`.
**The fleet is currently at zero such recipes.** The last two — `mattermost-lts` (empty advisory
feed, client-side-rendered bulletins) and `mumble` (nothing published anywhere) — were fixed by
declaring an NVD CPE in their registry:
```
- nvd-cpe: mattermost-team-edition = cpe:2.3:a:mattermost:mattermost_server:*:*:*:*:*:*:*:*
```
**If this sweep ever reports a blind recipe again, that is the fix**: find the product's CPE at
nvd.nist.gov and add the line. Prefer a real advisory feed or an attributable changelog when one
exists — NVD lags the vendor — but a lagging source beats no source, and it turns a `?` into a
number.
An *unparseable page* is NOT the same thing: it is harmless when the same project also publishes an
advisory feed (redis, gitea, minio, clickhouse all do). Only "no usable source for this image" counts.
### 3. Run the advisory scan over that window
```
python3 /srv/cc-ci/cc-ci-plan/advisory-scan.py <recipe> --from <old-app> --to <new-app> \
+26
View File
@@ -115,3 +115,29 @@ When the orchestrator, Builder, or assistant makes intentional repository change
promptly and push them to `git.autonomic.zone` in append-only fashion (never force-push). Match the
existing commit author and message style in this repo. Do not bundle unrelated worktree changes you
did not make; stage only the intended files.
## Ship as PRs, merge them yourself, operator reviews retrospectively
**This applies to the two INFRASTRUCTURE repos — `recipe-maintainers/cc-ci-orchestrator` (here) and
`recipe-maintainers/cc-ci` (the CI product).** For work in either:
1. Branch, don't commit straight to `main`.
2. Open a PR with a description written to be read **after** the fact: what changed, why, and what
evidence says it works (test output, a verified run, a before/after number). The PR *is* the
review artifact and the historical record.
3. **Merge it yourself once it is verified** — do not wait for review. The invocation is the
authorization; blocking on review would stall the pipeline these repos exist to run.
4. The operator reviews **retrospectively**, from the PR.
So the PR is not a gate — it is how the work stays legible. A PR that merely says "fix scanner" has
failed at its only job.
> ### This does NOT extend to RECIPE repos
> Recipe PRs — any `coop-cloud/<recipe>` or its `recipe-maintainers/<recipe>` mirror — are
> **created and verified but NEVER merged by an agent**. Those change what deploys on other people's
> infrastructure, so a human merges them. The split is deliberate: agents own the tooling, the
> operator owns the recipes.
If work has already landed on `main` without a PR, do not rewrite published history to fix it.
Create a branch pinned at the pre-work commit and open the PR against that, so the diff is still
reviewable and merging only advances the pointer (see PRs #2-#5, 2026-08-11).
+76 -3
View File
@@ -39,6 +39,8 @@ keeps landing in pass 2, the fix is a new deterministic method in pass 1. §4c i
```
advisory-scan.py <recipe> [--from <version>] [--to <version>]
[--image <name>=<from>:<to>]... [--adjudicate] [--json] [--registry DIR]
advisory-scan.py <recipe> --compose-to <URL> [--compose-from <URL>] # windows derived, not typed
```
| Input | Meaning |
@@ -46,6 +48,8 @@ advisory-scan.py <recipe> [--from <version>] [--to <version>]
| `<recipe>` | Recipe name; selects `cc-ci-plan/upstream/<recipe>.md` (the per-recipe URL registry) |
| `--from` / `--to` | The **primary app image's** version window being upgraded across |
| `--image NAME=FROM:TO` | A **sidecar image and the versions it moved between** (repeatable, all in ONE call). `NAME` is substring-matched against source repo names. Malformed values warn on stderr and are skipped. Without it that image's advisories stay unclassified. |
| `--compose-to URL` | **Derive every window by diffing this compose against its baseline**, instead of typing `--from/--to/--image`. Point it at a PR's `compose.yml`. |
| `--compose-from URL` | Baseline for the above. Default: the same repo's **default branch, resolved from the API** — never assumed to be `main`. |
| `--adjudicate` | Run pass 2: append the evidence dossier for judgement |
| `--registry` | Registry dir; also `CCCI_UPSTREAM_REGISTRY` |
| `GITHUB_TOKEN` / `GITHUB_TOKEN_FILE` | Read-only token; **rate limit only** (60/hr anonymous → 5000/hr). Default file `/srv/cc-ci/.github-token`, mode 600. Public advisories need **no scopes**. |
@@ -103,10 +107,43 @@ URLs containing `<`, `>`, `{`, `}`, `VERSION`, or `vX.Y.Z` are **skipped as temp
human documentation (`…/changelog/v<VERSION>/`), not fetchable, and counting them as failures is wrong.
This is the source that would have caught gitea: the vendor blog names both CVEs, the GitHub release
page names neither. A CVE found **only** here carries no version data, so pass 1 cannot place it — it
goes to pass 2 (§6).
page names neither.
### 2c. OSV.dev — supplementary
**When the page is a changelog organised by release, each CVE is attributed to the release heading it
appears under** (`Changes with nginx 1.31.3`, `## v1.31.3`, …) and that becomes its fixed-in version.
Without this, a project that publishes no advisory feed can never contribute a CVE:
> **nginx publishes NO GitHub security advisories.** Every nginx CVE we can see comes from
> `nginx.org/en/CHANGES`. Scraping ids out of it without attributing them to a release left them with
> no patched version, so they were never classifiable — and every nginx bump in the fleet reported
> **0** forever. nginx is a sidecar in most recipes. Measured: `1.31.1 → 1.31.3` fixes **six** CVEs
> (three in .2, three in .3); lasuite-docs#7 went 0 → 6 and lasuite-drive#6 went 0 → 3 on this alone.
A changelog CVE is tied to a window by the **image name appearing in the page URL** (window `nginx` ↔
`nginx.org/...`). A CVE found on a vendor page with no attributable release still has no version data,
so pass 1 cannot place it — it goes to pass 2 (§6).
### 2c. NVD by CPE — the fallback for projects that publish nothing
Declared per recipe in the registry as `nvd-cpe: <image-key> = <cpe:2.3:...>`.
> **Why it exists.** Two recipes could not see CVEs *at all*: `mattermost-lts` (empty GitHub advisory
> feed, security bulletins rendered client-side so a text sweep finds nothing) and `mumble` (nothing
> published anywhere the registry points). Their scans returned `?` — nothing measured. NVD is
> CPE-indexed and carries structured ranges, so it answers where the vendor does not: mattermost
> 10.5.0 → 10.12.4 now scores **165**, and mumble finds `CVE-2025-71264` (fixed 1.6.870).
Two range forms, both used:
| NVD field | meaning | how it is judged |
|---|---|---|
| `versionEndExcluding X` | fixed in X exactly | a normal patched version (§4a) |
| `versionEndIncluding X` | affected **up to and including** X; fix version unpublished | fixed when the upgrade crosses X, i.e. `from ≤ X < to` |
**NVD lags the vendor** — it had neither gitea CVSS-9.8 RCE at publication — so this is a fallback,
never a replacement for 2a/2b. Unauthenticated calls are rate-limited (~5/30s), hence the retry.
### 2d. OSV.dev — supplementary
Only when the recipe has an entry in `OSV_PACKAGES` (ecosystem + package) and a version is given.
@@ -140,8 +177,44 @@ Two invariants govern this step, both learned from a wrong answer in production.
> `null` / `UNKNOWN`, never `0`. A `0` in a security column asserts safety. Equally, an advisory that
> cannot be judged is **indeterminate** (§4d) — never silently counted as "not fixed".
### 3b. Deriving the windows from a compose diff (`--compose-to`)
Typing `--from/--to/--image` by hand means someone has to remember that the recipe also bumped its
redis. That is how sidecar CVEs went uncounted for months. This mode reads the windows off the diff:
1. Fetch both compose files (baseline = the repo's **default branch from the API**, since several
recipes keep a stale `main` beside a live `master`).
2. Parse `{service: (image-repo, tag)}` — keyed by **service, not image repo**, because an upgrade
may change the repo itself (plausible moved `plausible/analytics` →
`ghcr.io/plausible/community-edition`; keyed by repo that reads as one image vanishing and an
unrelated one appearing, losing the app window entirely).
3. Every service whose tag or repo changed becomes a window. The `app` service drives `--from/--to`
(coop-cloud convention: it is the recipe's primary image); the rest become `--image` windows.
Unchanged images produce no window — inventing one would be a false count.
4. The derived windows are printed to stderr before the scan, so the inputs are auditable.
Image names are matched against advisory sources **both ways** — an image name is often longer than
its source repo (`clickhouse/clickhouse-server` vs `ClickHouse/ClickHouse`) and sometimes shorter
(`redis` vs `redis/redis`).
Verified on plausible PR #5: from the compose URL alone it derives `v2.0.0 → v3.2.1` plus
`clickhouse-server 23.4.2.11-alpine → 24.12-alpine`, and reports **6** — identical to the
hand-specified args.
`--from/--to/--image` remain available for finer-grained checks (scanning a window that is not a
literal compose diff, e.g. "what would the compatibility-safe target fix?").
### 4a. By patched version (preferred — exact)
**A fix on the line you are upgrading FROM was already yours.** Projects that maintain several lines
patch them all at once: mattermost fixed `CVE-2025-11794` in 10.11.4, 10.12.1 *and* 10.5.12. An
upgrade 10.11.22 → 10.12.4 crosses 10.12.1, so a naive window test counts it — but 10.11.22 is
already past 10.11.4, so the deployment had the fix before the upgrade. Counting it credits the
upgrade with work it did not do. This check is **skipped for placeholder versions** (`7.4.X` parses
to a bare `7.4`, which would read as "already fixed at 7.4" and silently drop a real fix — exactly
how redis `CVE-2024-46981` was lost when the rule was first added).
`patched_versions` is a **range expression** (`">= 2.18.1"`), possibly several joined by `;`. Extract
every version-looking token; the advisory is **fixed-by-this-upgrade** if **any** patched version `p`
satisfies `from < p <= to` — exclusive lower (a fix already in the version you were on is not this
+354 -6
View File
@@ -44,7 +44,9 @@ import json
import os
import re
import sys
import time
import urllib.error
import urllib.parse
import urllib.request
REGISTRY_DIR = os.environ.get("CCCI_UPSTREAM_REGISTRY", "/srv/cc-ci/cc-ci-plan/upstream")
@@ -164,6 +166,27 @@ def _vkey(v: str | None) -> tuple:
return tuple(out)
def _already_fixed_on_from_line(kf: tuple, cands: list[tuple]) -> bool:
"""Was it ALREADY fixed on the line we are upgrading FROM?
The mirror image of _superseded_on_target_line, and just as necessary. mattermost fixes each CVE
across several maintained lines at once — CVE-2025-11794 is patched in 10.11.4, 10.12.1 and
10.5.12. Upgrading 10.11.22 -> 10.12.4 crosses 10.12.1, so a naive window test counts it; but
10.11.22 is already past 10.11.4, so the deployment HAD the fix before the upgrade. Counting it
credits the upgrade with work it did not do."""
if len(kf) < 2:
return False
line = kf[:2]
for c in cands:
if len(c) < 2 or c[:2] != line:
continue
n = max(len(kf), len(c))
pad = lambda z: z + (0,) * (n - len(z))
if pad(c) <= pad(kf):
return True
return False
def _superseded_on_target_line(kt: tuple, cands: list[tuple]) -> bool:
"""Does a patched version on the TARGET's own release line sit ABOVE the target?
@@ -272,6 +295,51 @@ def github_advisories(urls: list[str]) -> list[dict]:
return results
# Release headings in a vendor changelog. nginx's CHANGES uses "Changes with nginx 1.31.3", most
# markdown changelogs use "## 1.31.3" / "## v1.31.3".
_HEADING_RE = re.compile(
r"^\s*(?:#{1,4}\s*)?(?:Changes with\s+\S+\s+|Version\s+|Release\s+)?v?(\d+\.\d+(?:\.\d+)*)\s*$"
r"|^\s*Changes with\s+\S+\s+(\d+\.\d+(?:\.\d+)*)", re.I)
def _changelog_versions(text: str) -> dict:
"""{cve: version} for a changelog that is ORGANISED BY RELEASE.
Why this exists: nginx publishes NO GitHub security advisories. Every nginx CVE we can see comes
from nginx.org/en/CHANGES, and scraping ids out of it without attributing them to a release
leaves them with no patched version — so they can never be classified, and an nginx bump reports
0 CVEs forever. nginx 1.31.1 -> 1.31.3 in fact fixes SIX (three in .2, three in .3), and nginx is
a sidecar in most of the fleet, so that was a fleet-wide blind spot.
Attributes each CVE to the nearest PRECEDING release heading — the release that fixed it.
"""
plain = re.sub(r"<[^>]+>", " ", text)
out, cur = {}, None
for line in plain.splitlines():
m = _HEADING_RE.match(line)
if m:
cur = m.group(1) or m.group(2)
continue
if cur:
for cve in CVE_RE.findall(line):
out.setdefault(cve, cur)
return out
_BLOB_RE = re.compile(r"^https://github\.com/([^/]+)/([^/]+)/blob/(.+)$")
def _raw_if_blob(url: str) -> str:
"""A GitHub *blob* URL is an HTML viewer, not the file.
The registry pointed ONLYOFFICE's CHANGELOG.md at its blob page. Fetching that returns 636KB of
markup in which the release headings do not survive HTML-stripping, so 24 CVEs were visible and
NONE attributable to a release — the same shape of blind spot as nginx. The raw URL attributes
all 24. Normalising here fixes every registry entry at once, present and future."""
m = _BLOB_RE.match(url)
return f"https://raw.githubusercontent.com/{m.group(1)}/{m.group(2)}/{m.group(3)}" if m else url
def vendor_pages(urls: list[str]) -> list[dict]:
"""Fetch each registry URL and regex out CVE ids, with a little surrounding context."""
out = []
@@ -284,20 +352,93 @@ def vendor_pages(urls: list[str]) -> list[dict]:
# correct; counting them as failures would wrongly mark the recipe's count unreliable.
out.append({"source": u, "status": "skipped: template URL (not fetchable)", "cves": [], "context": {}})
continue
entry = {"source": u, "status": "ok", "cves": [], "context": {}}
entry = {"source": u, "status": "ok", "cves": [], "context": {}, "fixed_in": {}}
try:
text = _fetch(u)
text = _fetch(_raw_if_blob(u))
plain = re.sub(r"<[^>]+>", " ", text)
for cve in sorted(set(CVE_RE.findall(plain))):
entry["cves"].append(cve)
i = plain.find(cve)
entry["context"][cve] = re.sub(r"\s+", " ", plain[max(0, i - 160) : i + 200]).strip()
entry["fixed_in"] = _changelog_versions(text)
except Exception as e: # noqa: BLE001
entry["status"] = f"error: {type(e).__name__}: {e}"
out.append(entry)
return out
NVD_API = "https://services.nvd.nist.gov/rest/json/cves/2.0"
NVD_CPE_RE = re.compile(r"^\s*[-*]?\s*nvd-cpe:\s*(\S+)\s*=\s*(cpe:2\.3:[^\s`]+)", re.M | re.I)
def registry_cpes(recipe: str, registry_dir: str) -> list[tuple[str, str]]:
"""[(image-key, cpe)] declared in the recipe's registry as `nvd-cpe: <key> = <cpe>`."""
path = os.path.join(registry_dir, f"{recipe}.md")
try:
return [(m.group(1), m.group(2)) for m in NVD_CPE_RE.finditer(open(path).read())]
except OSError:
return []
def nvd_advisories(cpe: str, key: str) -> dict:
"""CVEs for a CPE from NVD, with the version data the classifier needs.
THE FALLBACK FOR PROJECTS THAT PUBLISH NOTHING MACHINE-READABLE. mattermost's GitHub advisory
feed is empty and its security bulletins are client-side rendered; mumble publishes neither. Both
scanned as `?` — nothing measured — until here. NVD is CPE-indexed and carries structured ranges:
versionEndExcluding X -> fixed in X exactly (a patched version)
versionEndIncluding X -> affected up to and INCLUDING X, fixed in some later release. The
exact fix version is unknown, but the upgrade fixes it whenever it
crosses X — recorded as `affected_max` and judged in the classifier.
NVD LAGS the vendor (it had neither gitea CVSS-9.8 RCE at publication), so this is a fallback,
never a replacement for 2a/2b. Unauthenticated calls are rate-limited to ~5/30s, hence the retry.
"""
entry = {"source": f"nvd:{key}", "status": "ok", "advisories": []}
url = f"{NVD_API}?resultsPerPage=2000&virtualMatchString={urllib.parse.quote(cpe)}"
data = None
for attempt in range(3):
try:
data = json.loads(_fetch(url))
break
except Exception as e: # noqa: BLE001
if attempt == 2:
entry["status"] = f"error: {type(e).__name__}"
return entry
time.sleep(8)
for v in (data or {}).get("vulnerabilities", []):
c = v.get("cve") or {}
cid = c.get("id")
if not cid:
continue
fixed, affected_max = set(), set()
for cfg in c.get("configurations", []):
for node in cfg.get("nodes", []):
for m in node.get("cpeMatch", []):
if m.get("versionEndExcluding"):
fixed.add(m["versionEndExcluding"])
elif m.get("versionEndIncluding"):
affected_max.add(m["versionEndIncluding"])
sev = None
for mk in ("cvssMetricV31", "cvssMetricV30", "cvssMetricV2"):
got = (c.get("metrics") or {}).get(mk) or []
if got:
sev = (got[0].get("cvssData") or {}).get("baseSeverity")
break
entry["advisories"].append({
"cve": cid, "ghsa": None, "severity": (sev or "").lower() or None,
"summary": next((d.get("value") for d in c.get("descriptions", [])
if d.get("lang") == "en"), "")[:200],
"vulnerable_range": None,
"patched": "; ".join(sorted(fixed)) or None,
"affected_max": "; ".join(sorted(affected_max)) or None,
"url": f"https://nvd.nist.gov/vuln/detail/{cid}",
"published_at": c.get("published"), "description": None, "cvss": None,
})
return entry
def osv(recipe: str, version: str | None) -> dict | None:
pkg = OSV_PACKAGES.get(recipe)
if not pkg or not version:
@@ -595,7 +736,8 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
e = report["cves"].setdefault(cve, {"sources": [], "severity": None, "ghsa": None,
"vulnerable_range": None, "patched": None,
"context": None, "published_at": None,
"description": None, "url": None, "cvss": None})
"description": None, "url": None, "cvss": None,
"changelog_fixed_in": None, "affected_max": None})
if src not in e["sources"]:
e["sources"].append(src)
for k, v in extra.items():
@@ -612,11 +754,22 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
context=a.get("summary"), published_at=a.get("published_at"),
description=a.get("description"), url=a.get("url"), cvss=a.get("cvss"))
for key, cpe in registry_cpes(recipe, registry_dir):
entry = nvd_advisories(cpe, key)
report["sources"].append({"source": entry["source"], "status": entry["status"],
"found": len(entry.get("advisories") or [])})
for a in entry.get("advisories", []):
record(a["cve"], entry["source"], severity=a.get("severity"),
patched=a.get("patched"), affected_max=a.get("affected_max"),
context=a.get("summary"), published_at=a.get("published_at"),
url=a.get("url"))
for entry in vendor_pages(urls):
report["sources"].append({"source": entry["source"], "status": entry["status"],
"found": len(entry.get("cves", []))})
for cve in entry.get("cves", []):
record(cve, entry["source"], context=entry["context"].get(cve))
record(cve, entry["source"], context=entry["context"].get(cve),
changelog_fixed_in=(entry.get("fixed_in") or {}).get(cve))
for version in filter(None, (v_from, v_to)):
o = osv(recipe, version)
@@ -651,17 +804,39 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
#
# A source with no window is not classified: its advisories are listed as unclassified so they
# stay visible without inflating the count.
gh_sources = [x["source"] for x in report["sources"] if x["source"].startswith("github-advisories:")]
gh_sources = [x["source"] for x in report["sources"]
if x["source"].startswith(("github-advisories:", "nvd:"))]
primary = gh_sources[0] if (gh_sources and (v_from or v_to)) else None
report["primary_source"] = primary
windows = {} # source name -> (from, to)
window_key = {} # source name -> the image name it covers
if primary:
windows[primary] = (v_from, v_to)
window_key[primary] = primary.split("/")[-1]
# The app's window must also cover its NVD entry. NVD sources are keyed by IMAGE name
# (`mattermost-team-edition`) while the advisory feed is keyed by REPO (`mattermost/
# mattermost`), so without this the fallback source that exists precisely because the feed
# is empty would itself go unwindowed — and mumble/mattermost would still report nothing.
pname = primary.split("/")[-1].lower()
for src in gh_sources:
if src.startswith("nvd:") and src not in windows:
k = src.split(":", 1)[1].lower()
if pname in k or k in pname:
windows[src] = (v_from, v_to)
window_key[src] = k
for key, wf, wt in (images or []):
for src in gh_sources:
if key.lower() in src.lower() and src not in windows:
if src in windows:
continue
# Match BOTH ways: an image name is often longer than its source repo
# (`clickhouse/clickhouse-server` vs source `ClickHouse/ClickHouse`) and sometimes
# shorter (`redis` vs `redis/redis`). One-directional matching silently dropped the
# clickhouse window when the key was derived from a compose file.
k, name = key.lower(), src.split("/")[-1].lower()
if k in src.lower() or name in k:
windows[src] = (wf, wt)
window_key[src] = key
report["windows"] = {k: {"from": f, "to": t} for k, (f, t) in windows.items()}
def _classify_window(src, wf, wt):
@@ -680,12 +855,31 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
continue
patched = e.get("patched") or ""
cands = [_vkey(t) for t in re.findall(r"\d+(?:\.\d+)*", patched)]
# NEVER on a placeholder: "7.4.X" parses to the bare 7.4, which then reads as
# "already fixed at 7.4" and silently drops a real fix (redis CVE-2024-46981).
# A placeholder means the fix version is unknown — that is the indeterminate path.
if (kf and kt and not PLACEHOLDER_RE.search(patched)
and _already_fixed_on_from_line(kf, cands)):
# already had it before the upgrade
e.setdefault("classification", "outside-window")
continue
if kf and kt and _superseded_on_target_line(kt, cands):
# The target's own line got the fix LATER than the target: not fixed here.
e.setdefault("classification", "outside-window")
continue
if kf and kt and any(_within(kf, kt, c) for c in cands):
got.add(cve)
elif kf and kt and e.get("affected_max"):
# NVD's `versionEndIncluding X`: affected up to and INCLUDING X, fixed in some
# later release. The exact fix version is unpublished, but the upgrade delivers
# it whenever it crosses X — i.e. from <= X < to.
for t in re.findall(r"\d+(?:\.\d+)*", e["affected_max"]):
x = _vkey(t)
n = max(len(kf), len(kt), len(x))
pad = lambda z: z + (0,) * (n - len(z))
if x and pad(kf) <= pad(x) < pad(kt):
got.add(cve)
break
elif not patched or PLACEHOLDER_RE.search(patched):
# No fix version published ("TBD") or only a placeholder ("7.4.X" — which could
# be 7.4.1, inside the window). We cannot say either way, so say so.
@@ -719,6 +913,32 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
report["cves"][cve]["classification"] = f"fixed-by-this-upgrade ({method}) via {src}"
fixed_set.add(cve)
# A CVE seen only in a vendor CHANGELOG has no advisory feed behind it, but the changelog says
# which release fixed it (see _changelog_versions). Tie it to a window by the image name
# appearing in the page URL — nginx's window is `nginx`, and its changelog is nginx.org/... .
# Without this, projects that publish no GitHub advisories (nginx being the big one) can never
# contribute a CVE, and every nginx bump in the fleet silently reports 0.
from_changelog = {}
for cve, e in report["cves"].items():
if cve in fixed_set or not e.get("changelog_fixed_in"):
continue
for src, (wf, wt) in windows.items():
key = (window_key.get(src) or "").lower()
if not key:
continue
if not any(key in s_.lower() for s_ in e["sources"] if s_.startswith("http")):
continue
kf, kt = _vkey(wf), _vkey(wt)
cand = _vkey(e["changelog_fixed_in"])
if kf and kt and cand and _within(kf, kt, cand):
e["classification"] = (f"fixed-by-this-upgrade (named under {e['changelog_fixed_in']} "
f"in the vendor changelog) via {src}")
fixed_set.add(cve)
from_changelog[cve] = e["changelog_fixed_in"]
break
if from_changelog:
report["resolved_by_changelog"] = from_changelog
unknown = []
for cve, e in report["cves"].items():
if cve in fixed_set:
@@ -893,6 +1113,113 @@ def markdown(rep: dict) -> str:
return "\n".join(L)
def _gitea_auth(url: str) -> dict:
"""Basic auth for the private mirror, from /srv/cc-ci/.testenv.
Sent as a HEADER, never embedded in the URL: in-URL credentials leak into shell history, process
lists and error messages, and urllib mis-parses a password containing a colon."""
host = re.sub(r"^https?://", "", url).split("/")[0]
env = {}
try:
for ln in open(os.environ.get("CCCI_TESTENV", "/srv/cc-ci/.testenv")):
if "=" in ln and not ln.strip().startswith("#"):
k, v = ln.strip().split("=", 1)
env[k] = v.strip().strip("\"'")
except OSError:
return {}
if host != env.get("GITEA_URL", "git.autonomic.zone"):
return {}
u, pw = env.get("GITEA_USERNAME"), env.get("GITEA_PASSWORD")
if not (u and pw):
return {}
import base64 as _b64
return {"Authorization": "Basic " + _b64.b64encode(f"{u}:{pw}".encode()).decode()}
def _compose_images(url: str) -> dict[str, tuple[str, str]]:
"""{service: (image-repo, tag)} for a compose file.
Keyed by SERVICE, not by image repo, because an upgrade may change the repo itself: plausible
moved `plausible/analytics` -> `ghcr.io/plausible/community-edition`. Keyed by repo that reads
as one image vanishing and an unrelated one appearing, and the app's version window is lost —
which is exactly the upgrade most worth scanning."""
txt = _fetch(url, _gitea_auth(url))
out, svc = {}, None
in_services = False
for line in txt.splitlines():
if re.match(r"^services:\s*$", line):
in_services = True
continue
if in_services and re.match(r"^\S", line):
in_services = False
if not in_services:
continue
m = re.match(r"^ (\S+):\s*$", line)
if m:
svc = m.group(1)
continue
m = re.match(r"^\s+image:\s*[\"']?([^\"'\s]+)", line)
if m and svc:
ref = m.group(1).split("@", 1)[0]
if "${" in ref or "$(" in ref:
continue
repo, _, tag = ref.rpartition(":")
if repo and tag:
out[svc] = (repo, tag)
return out
def _default_branch_compose(url: str) -> str | None:
"""Same repo as `url`, but its DEFAULT branch — resolved from the API, never assumed.
Several coopcloud recipes keep a stale `main` beside the real default `master` (gitea's `main`
is 1.24.2-rootless while `master` has 1.27.1-rootless), so guessing the branch produces a
confidently wrong baseline."""
m = re.match(r"(https?://[^/]+)/([^/]+)/([^/]+)/(?:raw|src)/branch/[^/]+/(.*)$", url)
if not m:
return None
host, owner, repo, path = m.groups()
try:
meta = json.loads(_fetch(f"{host}/api/v1/repos/{owner}/{repo}", _gitea_auth(host)))
br = meta.get("default_branch")
except Exception: # noqa: BLE001
return None
return f"{host}/{owner}/{repo}/raw/branch/{br}/{path}" if br else None
def windows_from_compose(to_url: str, from_url: str | None = None) -> tuple[list, str | None]:
"""Derive the scan's version windows by DIFFING two compose files.
This is the deterministic alternative to a human (or a model) deciding which `--image` args a
given upgrade needs. Point it at a PR's compose and it reads the windows straight off the diff:
every image whose tag changed becomes a window, every image that did not change is correctly
left out, and nothing depends on anyone remembering that the recipe also bumped its redis.
Returns (windows, note) where windows is [(image-name, from, to)].
"""
if from_url is None:
from_url = _default_branch_compose(to_url)
if not from_url:
raise SystemExit("could not resolve a baseline compose; pass --compose-from explicitly")
new, old = _compose_images(to_url), _compose_images(from_url)
app, others = None, []
for svc, (repo, tag) in sorted(new.items()):
if svc not in old:
continue
prev_repo, prev_tag = old[svc]
if prev_tag == tag and prev_repo == repo:
continue
# The `app` service is the recipe's primary image by coop-cloud convention; its window drives
# --from/--to so the scan's primary advisory source is judged against it. Everything else is
# a sidecar window keyed by its image name.
if svc == "app":
app = (repo.split("/")[-1], prev_tag, tag)
else:
others.append((repo.split("/")[-1], prev_tag, tag))
wins = ([app] if app else []) + others
return wins, f"baseline {from_url}"
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("recipe")
@@ -904,6 +1231,13 @@ def main() -> int:
"fix version published), fetch their full text + references and append a "
"block for the agent to judge. Additive: it never changes the count above.")
ap.add_argument("--registry", default=REGISTRY_DIR)
ap.add_argument("--compose-to", default=None, metavar="URL",
help="derive the windows by DIFFING this compose against its baseline, instead "
"of passing --from/--to/--image by hand. Point it at a PR's compose.yml "
"(e.g. .../raw/branch/<pr-branch>/compose.yml).")
ap.add_argument("--compose-from", default=None, metavar="URL",
help="baseline compose for --compose-to. Default: the same repo's DEFAULT "
"branch, resolved from the API (never assumed to be `main`).")
ap.add_argument("--image", action="append", default=[], metavar="NAME=FROM:TO",
help="a sidecar image and the versions it moved between, e.g. "
"--image redis=7.4:8.10 (repeatable). NAME matches a source repo name; "
@@ -911,6 +1245,20 @@ def main() -> int:
"being left unclassified.")
a = ap.parse_args()
images = []
if a.compose_to:
wins, note = windows_from_compose(a.compose_to, a.compose_from)
if not wins:
print(f"### Advisory scan — {a.recipe}\n\n**No image versions changed between the two "
f"compose files, so this upgrade fixes no CVEs by definition.**\n\n_{note}_")
return 0
print(f"_derived from compose diff ({note}):_", file=sys.stderr)
for n_, f_, t_ in wins:
print(f"_ {n_}: {f_}{t_}_", file=sys.stderr)
# The `app` service (first entry when present) drives --from/--to; the rest are --image
# windows. Passing every window as --image too is harmless: each is matched by name against
# the advisory sources, and an unmatched name is simply ignored.
a.v_from, a.v_to = a.v_from or wins[0][1], a.v_to or wins[0][2]
images = list(wins[1:])
for spec in a.image:
name, _, rng = spec.partition('=')
vf, _, vt = rng.partition(':')
+76
View File
@@ -42,6 +42,11 @@ _spec = importlib.util.spec_from_file_location("resolve_images", os.path.join(HE
RI = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(RI)
# advisory-scan supplies the source-fetching + changelog-attribution used by --security-sources
_aspec = importlib.util.spec_from_file_location("advisory_scan", os.path.join(HERE, "advisory-scan.py"))
A = importlib.util.module_from_spec(_aspec)
_aspec.loader.exec_module(A)
REGISTRY_DIR = os.environ.get("CCCI_UPSTREAM_REGISTRY", os.path.join(HERE, "upstream"))
USED_RECIPES = os.path.join(HERE, "used-recipes.md")
DEPRECATION_RE = re.compile(
@@ -108,6 +113,45 @@ def newest_tag_date(registry: str, repo: str, tag: str) -> str | None:
return (d.get("results") or [{}])[0].get("last_updated")
def security_source_audit(recipe: str) -> list[dict]:
"""Per source: are its CVEs USABLE, or merely visible?
The nginx lesson. nginx publishes no GitHub advisories; all its CVEs live in nginx.org/en/CHANGES.
The scan saw them and could do nothing with them, because nothing said which release fixed which
CVE — so every nginx bump in the fleet reported 0. Attribution (advisory-scan §2b) fixed that for
changelogs organised by release, but a page that lists CVEs with NO release structure is still a
blind spot: visible, uncountable. This finds those.
Per source: `advisory-feed` (structured, best), `changelog` (CVEs attributable to a release),
`unattributable` (CVEs present but no release structure — BLIND), or `no-cve-data`.
"""
urls, _ = _registry_urls(recipe)
out = []
# NVD CPE entries are a first-class source: for projects publishing nothing machine-readable
# (mattermost, mumble) they are the ONLY structured source, and omitting them here made two
# recipes look permanently blind after they had been fixed.
for key, cpe in A.registry_cpes(recipe, REGISTRY_DIR):
e = A.nvd_advisories(cpe, key)
n = len(e.get("advisories") or [])
out.append({"source": e["source"] + f" ({cpe.split(':')[4]}/{cpe.split(':')[3]})",
"kind": "advisory-feed" if n else "no-cve-data",
"status": e["status"], "cves": n, "usable": n})
for entry in A.github_advisories(urls):
out.append({"source": entry["source"], "kind": "advisory-feed",
"status": entry["status"], "cves": len(entry.get("advisories") or []),
"usable": len(entry.get("advisories") or [])})
for entry in A.vendor_pages(urls):
if entry["status"].startswith("skipped"):
continue
n = len(entry.get("cves") or [])
attributed = len(entry.get("fixed_in") or {})
kind = ("no-cve-data" if n == 0 else
"changelog" if attributed else "unattributable")
out.append({"source": entry["source"], "kind": kind, "status": entry["status"],
"cves": n, "usable": attributed})
return out
def audit_recipe(recipe: str, ssh: str | None, quiet_days: int) -> dict:
out = {"recipe": recipe, "findings": [], "images": [], "sources": []}
try:
@@ -214,9 +258,41 @@ def main() -> int:
ap.add_argument("--ssh", default=None)
ap.add_argument("--quiet-days", type=int, default=365)
ap.add_argument("--json", action="store_true")
ap.add_argument("--security-sources", action="store_true",
help="audit whether each recipe's CVE sources are USABLE (structured advisory "
"feed / release-attributable changelog) or merely visible")
a = ap.parse_args()
recipes = a.recipes or all_recipes()
if a.security_sources:
# What matters is whether the RECIPE can see CVEs at all — not whether some individual page
# is unparseable. A page with no release structure is harmless when the same project also
# publishes an advisory feed (redis, gitea, minio, clickhouse all do); it is only a blind
# spot when nothing else covers that project.
blind_recipes, noisy = [], 0
for r in recipes:
rows = security_source_audit(r)
feeds = [x for x in rows if x["kind"] == "advisory-feed" and x["cves"] > 0]
logs = [x for x in rows if x["kind"] == "changelog"]
unattr = [x for x in rows if x["kind"] == "unattributable"]
noisy += len(unattr)
usable = len(feeds) + len(logs)
if usable == 0:
blind_recipes.append(r)
print(f"!! {r}: NO USABLE CVE SOURCE — {len(unattr)} unparseable page(s), "
f"0 advisory feeds, 0 attributable changelogs")
for x in rows:
print(f" {x['kind']:15} {x['source'][:64]} ({x['cves']} CVEs)")
else:
print(f"OK {r}: {len(feeds)} advisory-feed(s), {len(logs)} changelog(s)"
+ (f", {len(unattr)} unparseable page(s) (redundant — covered by a feed)"
if unattr else ""))
for x in logs:
print(f" changelog {x['source'][:62]} ({x['usable']}/{x['cves']})")
print(f"\n{len(recipes)} recipes · {len(blind_recipes)} with NO usable CVE source"
+ (f": {', '.join(blind_recipes)}" if blind_recipes else "")
+ f" · {noisy} unparseable page(s) elsewhere (harmless where a feed covers them)")
return 0
reports = [audit_recipe(r, a.ssh, a.quiet_days) for r in recipes]
if a.json:
print(json.dumps(reports, indent=2))
+155
View File
@@ -537,6 +537,161 @@ class TestReleaseLineSemantics(unittest.TestCase):
self.assertEqual(rep["fixed_by_this_upgrade"], ["CVE-2025-49844"])
class TestAlreadyFixedOnFromLine(unittest.TestCase):
"""A fix that landed on the line we upgrade FROM was already ours before the upgrade."""
def test_backport_to_our_own_line_is_not_credited(self):
# mattermost patches every maintained line at once. 10.11.22 -> 10.12.4 crosses 10.12.1, but
# 10.11.22 is already past 10.11.4, so the deployment HAD the fix. Counting it credits the
# upgrade with work it did not do.
rep = run_scan([gh("mattermost/mattermost",
[adv("CVE-1", patched="10.11.4; 10.12.1; 10.5.12")])],
v_from="10.11.22", v_to="10.12.4",
urls=["https://github.com/mattermost/mattermost"])
self.assertEqual(rep["fixed_by_this_upgrade"], [])
def test_a_fix_ABOVE_our_position_on_the_same_line_still_counts(self):
rep = run_scan([gh("mattermost/mattermost", [adv("CVE-2", patched="10.11.30; 10.12.1")])],
v_from="10.11.22", v_to="10.12.4",
urls=["https://github.com/mattermost/mattermost"])
self.assertEqual(rep["fixed_by_this_upgrade"], ["CVE-2"])
def test_placeholders_never_feed_this_rule(self):
# "7.4.X" parses to a bare 7.4, which would read as "already fixed at 7.4" and silently drop
# a real fix — this is exactly how redis CVE-2024-46981 was lost when the rule was added.
rep = run_scan([gh("redis/redis", [adv("CVE-3", patched="6.2.X, 7.2.X, 7.4.X")])],
v_from="7.4", v_to="8.10", urls=["https://github.com/redis/redis"])
self.assertIn("CVE-3", rep["indeterminate"])
self.assertEqual(rep["fixed_by_this_upgrade"], [])
class TestChangelogAttribution(unittest.TestCase):
"""Projects that publish no advisory feed still say which release fixed what — in their changelog."""
CHANGES = """
Changes with nginx 1.31.3 11 Aug 2026
*) Security: a flaw ... (CVE-2026-60005)
*) Security: another ... (CVE-2026-56434)
Changes with nginx 1.31.2 04 Aug 2026
*) Security: something ... (CVE-2026-48142)
Changes with nginx 1.31.1 21 Jul 2026
*) Security: older ... (CVE-2026-9256)
Changes with nginx 1.20.0 01 Jan 2021
*) Security: ancient ... (CVE-2013-2028)
"""
def test_each_cve_is_attributed_to_the_release_that_fixed_it(self):
got = A._changelog_versions(self.CHANGES)
self.assertEqual(got["CVE-2026-60005"], "1.31.3")
self.assertEqual(got["CVE-2026-48142"], "1.31.2")
self.assertEqual(got["CVE-2026-9256"], "1.31.1")
self.assertEqual(got["CVE-2013-2028"], "1.20.0")
def _scan(self, wfrom, wto):
# nginx publishes NO GitHub advisories — the feed is empty and the changelog is everything.
return run_scan(
[gh("nginx/nginx", [])],
[{"source": "https://nginx.org/en/CHANGES", "status": "ok",
"cves": sorted(A._changelog_versions(self.CHANGES)),
"context": {}, "fixed_in": A._changelog_versions(self.CHANGES)}],
images=[("nginx", wfrom, wto)], urls=["https://github.com/nginx/nginx"])
def test_window_counts_only_the_releases_it_crosses(self):
rep = self._scan("1.31.1", "1.31.3") # 1.31.1 is the FROM, so its CVE is already fixed
self.assertEqual(set(rep["fixed_by_this_upgrade"]),
{"CVE-2026-48142", "CVE-2026-56434", "CVE-2026-60005"})
def test_a_narrower_window_counts_fewer(self):
rep = self._scan("1.31.2", "1.31.3")
self.assertEqual(set(rep["fixed_by_this_upgrade"]), {"CVE-2026-56434", "CVE-2026-60005"})
def test_ancient_entries_are_not_swept_in(self):
# The changelog lists the project's whole history; only the crossed releases may count.
rep = self._scan("1.31.1", "1.31.3")
self.assertNotIn("CVE-2013-2028", rep["fixed_by_this_upgrade"])
def test_evidence_is_recorded(self):
rep = self._scan("1.31.1", "1.31.3")
self.assertEqual(rep["resolved_by_changelog"]["CVE-2026-60005"], "1.31.3")
class TestComposeDerivedWindows(unittest.TestCase):
"""Windows read off a compose diff, so nobody has to remember which --image args an upgrade needs."""
OLD = """
services:
app:
image: "plausible/analytics:v2.0.0"
db:
image: pgautoupgrade/pgautoupgrade:18-alpine
plausible_events_db:
image: clickhouse/clickhouse-server:23.4.2.11-alpine
volumes:
data:
"""
NEW = """
services:
app:
image: "ghcr.io/plausible/community-edition:v3.2.1"
db:
image: pgautoupgrade/pgautoupgrade:18-alpine
plausible_events_db:
image: clickhouse/clickhouse-server:24.12-alpine
volumes:
data:
"""
def _windows(self, old=None, new=None):
pages = {"to": new if new is not None else self.NEW,
"from": old if old is not None else self.OLD}
with unittest.mock.patch.object(A, "_fetch", lambda u, h=None: pages["to" if "to" in u else "from"]), \
unittest.mock.patch.object(A, "_gitea_auth", lambda u: {}):
return A.windows_from_compose("http://x/to", "http://x/from")[0]
def test_app_service_leads_and_sidecars_follow(self):
w = self._windows()
self.assertEqual(w[0], ("community-edition", "v2.0.0", "v3.2.1"))
self.assertIn(("clickhouse-server", "23.4.2.11-alpine", "24.12-alpine"), w)
def test_unchanged_images_are_not_windows(self):
# pgautoupgrade is identical in both; inventing a window for it would be a false count.
self.assertNotIn("pgautoupgrade", [n for n, _, _ in self._windows()])
def test_a_changed_image_REPO_is_still_the_same_service(self):
# plausible/analytics -> ghcr.io/plausible/community-edition. Keyed by image repo this reads
# as one image vanishing and another appearing, and the app window is lost entirely.
w = self._windows()
self.assertTrue(any(n == "community-edition" and f == "v2.0.0" for n, f, _ in w))
def test_no_change_yields_no_windows(self):
self.assertEqual(self._windows(old=self.NEW, new=self.NEW), [])
def test_templated_tags_are_skipped(self):
new = self.NEW.replace('ghcr.io/plausible/community-edition:v3.2.1', 'ghost:${IMAGE_VERSION}')
self.assertNotIn("ghost", [n for n, _, _ in self._windows(new=new)])
class TestImageNameMatching(unittest.TestCase):
"""An image name and its advisory source rarely spell each other exactly."""
def test_matches_when_the_image_name_is_LONGER_than_the_source(self):
# clickhouse/clickhouse-server vs source ClickHouse/ClickHouse — one-directional matching
# dropped this window silently when the key came from a compose file.
rep = run_scan([gh("ClickHouse/ClickHouse", [adv("CVE-1", patched="23.10.2.13")])],
images=[("clickhouse-server", "23.4.2.11", "24.12")],
urls=["https://github.com/ClickHouse/ClickHouse"])
self.assertIn("github-advisories:ClickHouse/ClickHouse", rep["windows"])
self.assertEqual(rep["cve_count_fixed"], 1)
def test_matches_when_the_image_name_is_SHORTER_than_the_source(self):
rep = run_scan([gh("redis/redis", [adv("CVE-2", patched="7.4.1")])],
images=[("redis", "7.4", "8.10")], urls=["https://github.com/redis/redis"])
self.assertEqual(rep["cve_count_fixed"], 1)
class TestAdjudicationEvidenceAssembly(unittest.TestCase):
"""Pass 2's JUDGEMENT is a model's and not testable; what IS testable is what it gets shown."""
+5
View File
@@ -18,6 +18,11 @@
- AUTO_MIGRATIONS=true means DB migrations run automatically on backend startup. No manual step needed.
- Minio tag uses a date-based RELEASE.YYYY-MM-DDTHH-MM-SSZ format — abra cannot parse it for upgrades;
check manually on https://github.com/minio/minio/releases.
- **2026-08-14: Minio stopped publishing Docker images after RELEASE.2025-09-07T16-13-09Z.**
GitHub has a newer release (`RELEASE.2025-10-15T17-29-55Z`, published 2025-10-16, with CVE fix
GHSA-jjjj-jwhf-8rgr), but the Docker image was never pushed to Docker Hub (returns 404; release
notes say "clone the source and build the latest container"). quay.io checked — only 2022-era
tags. As of this date, `RELEASE.2025-09-07T16-13-09Z` IS the newest available Docker image.
- v5.2.0 adds two optional new env vars: DOCUMENT_ALL_ENDPOINT_ENABLED and OIDC_OP_USER_ENDPOINT_FORMAT.
Both are backward-compatible (no action required for existing deployments).
- Recipe version label convention: 0.X.Y+vA.B.C where A.B.C is the impress version.
+9
View File
@@ -75,3 +75,12 @@
recreate DB, reimport dump. `DROP DATABASE WITH (FORCE)` requires PostgreSQL 13+ — safe on postgres:15-alpine.
The previous inline-label approach (no restore hook) was a defect: raw PGDATA restore without a reload
was a silent no-op. Fixed in PR #2 (restore fix cherry-picked from PR #1 ci/pg-restore).
## NVD CPE fallback
This project publishes nothing machine-readable we can reach — no GitHub advisory feed,
no release-attributable changelog — so its CVE count was `?` (nothing measured). NVD is
CPE-indexed and carries structured version ranges, so it can answer where the vendor
cannot. It LAGS the vendor, so it is a fallback, never the primary source.
- nvd-cpe: mattermost-team-edition = cpe:2.3:a:mattermost:mattermost_server:*:*:*:*:*:*:*:*
- nvd-cpe: postgres = cpe:2.3:a:postgresql:postgresql:*:*:*:*:*:*:*:*
+8
View File
@@ -19,3 +19,11 @@
- The server image tag is `v<version>-<build>` (e.g. `v1.6.870-4`); the trailing number is the image
build, not an app version, and moves independently of upstream releases — `abra recipe upgrade`
reports "no new versions" for it, so use `resolve-images.py` to see those bumps.
## NVD CPE fallback
This project publishes nothing machine-readable we can reach — no GitHub advisory feed,
no release-attributable changelog — so its CVE count was `?` (nothing measured). NVD is
CPE-indexed and carries structured version ranges, so it can answer where the vendor
cannot. It LAGS the vendor, so it is a fallback, never the primary source.
- nvd-cpe: mumble-server = cpe:2.3:a:mumble:mumble:*:*:*:*:*:*:*:*