Compare commits
30
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
bd7196bf9e | ||
|
|
508d6bb0de | ||
|
|
72b3e1223a | ||
|
|
aa6a3f095c | ||
|
|
9f103c3516 | ||
|
|
371546d1f8 | ||
|
|
b41eeaed75 | ||
|
|
e5cfa67e6e | ||
|
|
a1357e6866 | ||
|
|
f59eb1480f | ||
|
|
babb83debf | ||
|
|
93fe2b6f1a | ||
|
|
4e286ba777 | ||
|
|
cd1ec01b23 | ||
|
|
0fe2827889 | ||
|
|
bb76edde9f | ||
|
|
7f3627e76c | ||
|
|
6234e36261 | ||
|
|
78bbf763ff | ||
|
|
c57ddfd7f1 | ||
|
|
5d422207f0 | ||
|
|
4b8b617d8c | ||
|
|
53b1a76f70 | ||
|
|
f4ed2e3fbe | ||
|
|
7df1792dc9 | ||
|
|
afa6922093 | ||
|
|
50776ecb0c | ||
|
|
1b3ae31091 | ||
|
|
c97798ff8f | ||
|
|
4a043b12ca |
@@ -95,8 +95,9 @@ For each recipe produce **one window per image**: `current pinned tag → newest
|
||||
the sidecars (redis, postgres, nginx …), not just the app — a sidecar bump is where discourse's only
|
||||
CRITICAL came from, and an image with no window is not counted at all.
|
||||
|
||||
- **No upgrade available** → the recipe is `UPTODATE`; its CVE count is **`0`**, not `?`. There is
|
||||
nothing an upgrade could fix. Record it and move on.
|
||||
- **No upgrade available** → the recipe is `UPTODATE`; its **CVEs fixed by an available upgrade**
|
||||
count is `0`. This does not assert that the running image has no known CVEs. If advisory sources
|
||||
failed or are blind, use `?` and say why.
|
||||
- **No output at all is NOT "no upgrade".** An abra call that times out, FATAs, or prints nothing
|
||||
leaves the recipe **unverified** — treat it as a distinct outcome, never fold it into up-to-date.
|
||||
Re-run it, and if it still yields nothing, resolve the versions by direct registry check (box item 4).
|
||||
@@ -111,9 +112,17 @@ python3 cc-ci-plan/audit-sources.py --security-sources
|
||||
A recipe whose sources yield **no CVE data at all** cannot produce a meaningful `0` — nothing was
|
||||
measured, the same way a missing registry file cannot. Render those as **`?`**, not `0`.
|
||||
|
||||
**The fleet is currently at zero such recipes.** The last two — `mattermost-lts` (empty advisory
|
||||
feed, client-side-rendered bulletins) and `mumble` (nothing published anywhere) — were fixed by
|
||||
declaring an NVD CPE in their registry:
|
||||
Do not assume that the fleet has zero blind recipes. Run the audit every sweep and inspect source
|
||||
statuses; an HTTP error or rate limit is not evidence of no CVEs. `rustdesk-server` has an NVD CPE
|
||||
scoped to the OSS server edition; a successful zero-result query is a checked zero for that source,
|
||||
not a reason to count Client or Server Pro vulnerabilities against it.
|
||||
|
||||
Check `test -s /srv/cc-ci/.github-token` before a fleet scan. Without that read-only token,
|
||||
GitHub's anonymous API limit can be exhausted partway through the fleet; HTTP 403 results are
|
||||
failed sources, so the affected CVE counts remain unknown.
|
||||
|
||||
Two older blind spots — `mattermost-lts` and `mumble` — were addressed by declaring NVD CPEs in
|
||||
their registries:
|
||||
```
|
||||
- nvd-cpe: mattermost-team-edition = cpe:2.3:a:mattermost:mattermost_server:*:*:*:*:*:*:*:*
|
||||
```
|
||||
@@ -169,7 +178,8 @@ Severity outranks raw count: 2 CVSS-9.8 RCEs matter more than 120 medium plugin
|
||||
- `table[]` — every recipe swept. `recipe`; `change` = the window you scanned, e.g.
|
||||
`1.27.0 → 1.27.1 · redis 7.4 → 8.10`; `status` = `UPTODATE` when nothing is available, else
|
||||
`PENDING` (an upgrade exists and is not yet taken); **`cve`** = the count (integer, `?` only per the
|
||||
rules below); `notes` = severity mix, whether the number is a floor, and `maintained elsewhere` for
|
||||
rules below, or `"YES"` when a confirmed security advisory has no CVE ID and no numbered CVEs
|
||||
were found); `notes` = the advisory ID and severity, whether the number is a floor, and `maintained elsewhere` for
|
||||
`external` rows. **Leave `ci`/`pr` empty — nothing was built and no PR exists.**
|
||||
- `addendum[]` — real anomalies only: registry URLs that failed, recipes whose window could not be
|
||||
established, a scan whose count is a floor with many undetermined advisories.
|
||||
|
||||
@@ -30,7 +30,9 @@ keeps every weekly edition looking the same regardless of which model writes the
|
||||
2. **Survey.** `python3 /srv/cc-ci/cc-ci-plan/recipe-report.py survey <DATE> > /tmp/survey.json`. This
|
||||
gives you: the raw `/upgrade-all` summary; for every recipe mirror its open PRs + the `cc-ci/testme`
|
||||
verdict; and each recipe's **per-recipe upgrade notes** (`upgrade_notes_md` — the breaking-change /
|
||||
**CVE / security** analysis the upgrade did). Read it all.
|
||||
**CVE / security** analysis the upgrade did). Read it all. Also run
|
||||
`python3 /srv/cc-ci/cc-ci-plan/audit-sources.py --security-sources`; a recipe with no verified
|
||||
security source or a failed source must be called out, not presented as a clean CVE scan.
|
||||
|
||||
3. **Review & classify.** The page order is: **short lead → the full wire table (priority-sorted) →
|
||||
Addendum → Security Bulletin → per-recipe "What changed".** Same newspaper style; this order.
|
||||
@@ -46,9 +48,16 @@ keeps every weekly edition looking the same regardless of which model writes the
|
||||
lists **failed sources**, the count is **not** authoritative: render the cve cell as `?`
|
||||
(unknown), never `none` — a blank that reads as "clean" is exactly how two CVSS-9.8 gitea
|
||||
RCEs were reported as "none" on 2026-08-07.
|
||||
- **`?` must stay RARE — it means "we tried and could not tell", not "we didn't look".** Use it
|
||||
ONLY when a scan ran and reported genuinely failed sources, **or when the scan block says
|
||||
COUNT UNKNOWN**. In that case the scan's `0` means *not determined*: publish `?` and say so in
|
||||
- The table's **CVEs** column counts known CVEs addressed by the available recipe upgrade.
|
||||
A zero for an up-to-date recipe means there is no upgrade window to credit; it does not
|
||||
certify that the running images are free of vulnerabilities. If an upgrade exists but its
|
||||
advisory scan is absent, a source failed, or an image window was unmatched, use `?`.
|
||||
If a confirmed security advisory is fixed but has no CVE ID, put `"YES"` in `cve` rather
|
||||
than `0`; describe its advisory ID and severity in Notes. If CVE IDs are also fixed, show
|
||||
the numeric CVE count and mention the additional non-CVE advisories in Notes.
|
||||
- **`?` means the count is not verified.** Use it when an upgrade has no scan, a source failed,
|
||||
an image window was unmatched, or the scan block says COUNT UNKNOWN. In that case publish
|
||||
`?` and say why in
|
||||
the notes; publishing `0` would assert a clean bill of health nothing supports. Note a
|
||||
**version-scheme change is no longer a reason for `?`** — the scan resolves semver→calver jumps
|
||||
(discourse 3.5.3 → 2026.7.1) by falling back to advisory publish dates and reports a real number.
|
||||
|
||||
@@ -143,12 +143,31 @@ ssh cc-ci 'export PATH=/run/current-system/sw/bin:$PATH; R=<r>; \
|
||||
script -qec "abra recipe upgrade $R -m -n" /dev/null; \
|
||||
git -C ~/.abra/recipes/$R stash pop >/dev/null 2>&1 || true'
|
||||
```
|
||||
Build `RECIPES_TO_UPGRADE` = recipes with **≥1 available upgrade** (after the stash, so the untracked
|
||||
overlay doesn't count as dirty). Others go to `SKIPPED_UPFRONT` with a reason (`dirty-worktree` only for
|
||||
real TRACKED edits, `up-to-date`, `not-fetchable`). **A `tag and digest … not supported` parse FATA is
|
||||
NOT `not-fetchable`** — per box item 4, catch it and resolve availability via the direct upstream-tag
|
||||
check; only `not-fetchable` when the mirror genuinely can't be fetched (network/auth), never for a
|
||||
digest-pinned image.
|
||||
**Mandatory registry cross-check for EVERY recipe and EVERY image, even if abra says current:** after
|
||||
reconciling the mirror and refreshing the recipe checkout, run
|
||||
`python3 /srv/cc-ci/cc-ci-plan/resolve-images.py <recipe> --recipe-dir /root/.abra/recipes`
|
||||
on cc-ci. Read the JSON `images`, `upgrades_available`, `unresolved`, and `all_resolved` fields.
|
||||
The candidate list is the **union** of abra's upgrades and the resolver's upgrades. CryptPad's
|
||||
`version-YYYY.M.D` app tag is a known counterexample: abra only reports a parse warning, while the
|
||||
resolver finds the newer image. A warning is never evidence that an image is current. Inspect both
|
||||
`newest_same_shape` and `newest_within_major`; a newer major is still a candidate for planning,
|
||||
though it must not be applied without a compatibility check.
|
||||
|
||||
Build `RECIPES_TO_UPGRADE` from every recipe with at least one candidate. Only label a recipe
|
||||
`up-to-date` if abra completed successfully **and** the resolver checked every relevant image
|
||||
(`all_resolved: true`) **and** neither found a candidate. If an image is unresolved, templated,
|
||||
floating, or lacks comparable tags, investigate it directly and record the evidence. Otherwise
|
||||
record `unverified` with the image and reason; never fold it into `up-to-date`. `dirty-worktree`
|
||||
applies only to real tracked edits. A tag+digest parse failure is not `not-fetchable`.
|
||||
|
||||
Run `python3 /srv/cc-ci/cc-ci-plan/audit-sources.py --security-sources` once for the full inventory
|
||||
and include any `NO VERIFIED CVE SOURCE` recipes or failed advisory sources in the weekly summary.
|
||||
Do not describe their CVE count as clean. This audit is a coverage check, not a substitute for
|
||||
`advisory-scan.py` on each actual upgrade window.
|
||||
Before the audit, check `test -s /srv/cc-ci/.github-token`. If it is missing, record that GitHub
|
||||
advisory checks are subject to the anonymous 60-request/hour limit; any resulting HTTP 403 is a
|
||||
failed source and must become `?` in the report. Never replace it with `0` or silently retry from
|
||||
model memory. The file is a read-only GitHub token location; never print its contents.
|
||||
|
||||
**Reconcile every candidate's mirror during the survey** (even the up-to-date ones), so merged-upstream
|
||||
PRs are closed and every mirror `main` tracks true upstream — fleet-wide, not just where there's a new
|
||||
|
||||
@@ -1331,3 +1331,82 @@ Weekly run 2026-10-02 complete: 20 considered · 9 upgraded (PRs opened/extended
|
||||
Key highlights: gitea upstream shipped its first 2-digit stable (28.0.0-rootless, 2026-09-29 — BREAKING git-egress/actions-retention notes documented in PR #10); ghost 6.67.0 fixes HIGH CVE-2026-84383; nulls held at mysql 8.4 / mariadb 12.3 / postgres 13/15 lines per precedent; matrix-synapse extended to synapse 1.162.0 + MAS 1.26.0; immich bumped v3.2.4 (rc.3.3 declined). Operational: recipe-maintainers/gitea mirror had LOST its master branch (abra whole-recipe FATA) — repaired both refs to upstream HEAD f167b73; reconcile-upstream.sh should learn to recreate a missing upstream-default branch ref, not only sync main. 2 targets (gitea, n8n 2.42.2) were caught in follow-up passes after the initial worklist mis-derivation — nothing left unprocessed. Sweep start: 4 leaked volumes + 46.3 GB images reclaimed (disk 70%->40%); end-of-run reap 0 leaked dev deploys; disk 65% (51G free).
|
||||
Per-recipe logs: .cc-ci-logs/upgrades/<recipe>-upgrade-2026-10-02.md (10 files incl. survey).
|
||||
NEVER MERGED anything; all PRs await operator review + the operator's `abra recipe release <recipe> -x/y/z` after each merge.
|
||||
|
||||
### Event 2026-10-05 ~17:45 UTC — gitea PR #10 stale test fixed (cc-ci PR #44, verified GREEN, NOT merged)
|
||||
Operator invoked --with-tests for gitea PR #10 (28.0.0-rootless, RED on test_lfs_roundtrip run #31).
|
||||
Diagnosis: stale test — `_api()` caught only HTTPError so one urlopen(timeout=20) socket timeout
|
||||
during the 28.0.0 post-restart migration window escaped the poll loop; poll also hard-coded 120s
|
||||
deadline vs recipe_meta HTTP_TIMEOUT=600 (STYLE.md §5). Fix: never-raising _api (mirrors
|
||||
lifecycle.http_fetch) + deadline from recipe_meta.HTTP_TIMEOUT; assertions untouched.
|
||||
cc-ci PR #44 (branch fix/gitea-lfs-restart-wait) also carries the two PRE-EXISTING main lint
|
||||
failures (warm.py ruff format; acme-dns.nix statix W201, eval-identical restructure) that had made
|
||||
push build #33 RED and would have blocked merge — build #34 now success, lint.sh PASS.
|
||||
Verified: full cold suite on host worktree (e83cd46) vs recipe head 91e42c8 → 5/5 tiers pass,
|
||||
test_lfs_roundtrip PASSED; unit sample 315 pass / 4 fail = identical pre-existing failures on main.
|
||||
Pair note posted on both PRs: merge #44, then re-!testme #10. Nothing merged.
|
||||
|
||||
### Event 2026-10-05 ~18:00 UTC — wrong-recipe card screenshot root-caused; runner fix #45 merged; gitea !testme re-run GREEN
|
||||
Operator's !testme on gitea PR #10 (drone build 35) showed a custom-html-tiny screenshot in the card.
|
||||
Root cause: drone build numbers restarted when its DB was re-created (2026-09-27) while
|
||||
/var/lib/cc-ci-runs still holds old-era dirs (numbered to 1377) — build 35 collided with a May-31
|
||||
run 35 whose fossil screenshot.png (12950 B = exactly custom-html-tiny's real shot) was embedded
|
||||
because screenshot.capture()'s hook branch SKIPPED the snap when out_path existed (no hook saves
|
||||
itself). Run itself was genuinely gitea level 5. Fix (cosmetics-only, R7): results.fresh_artifact_dir()
|
||||
wipes owned artifacts at run start + capture() always snaps; 4 unit tests (non-vacuous: capture test
|
||||
fails on main). Adversary verification PASS (independent fresh clones + own E2E collision test).
|
||||
cc-ci PR #45 merged (squash, 35ba9976); push self-tests 38/39 green; lint gate now green on main
|
||||
(#44 had already fixed the two pre-existing lint failures).
|
||||
Merged #44 first (operator informed; verified earlier via full cold harness run).
|
||||
Re-ran !testme on gitea PR #10 → drone build 40: GREEN, level 5, card links /runs/40/, screenshot
|
||||
is a fresh PNG of the run's own gitea (footer 1.24.2 = the upgrade-tier BASE deploy, the last-green
|
||||
warm canonical — pre-existing capture timing, correct behavior, only freshness was broken before).
|
||||
Gitea PR #10 now fully green incl. the fixed LFS test (uses merged #44 test). NOTHING else merged;
|
||||
recipe PR #10 + unrelated PR #4 still await operator merge + `abra recipe release gitea -x`.
|
||||
Cleanup: host /tmp/v /tmp/r, local worktrees, test-file mis-commit in orch cc-ci clone reverted.
|
||||
### Event 2026-10-23 04:15 UTC — weekly /upgrade-all run complete (DEFAULT mode, nothing merged)
|
||||
Run 2026-10-23: 19 considered · 9 upgraded with !testme GREEN · 2 PRs open pending the warm-canonical
|
||||
minio-prepull fix (lasuite-docs PR #11, lasuite-drive PR #8 — !testme RED at the upgrade BASE prepull:
|
||||
warm canonicals 0.4.0+v5.4.1 / 0.12.0+v0.21.0 pin the REMOVED docker.io/minio/minio; PR comments
|
||||
explain; operator: advance those canonicals then re-post !testme) · 0 failed · 8 skipped up-to-date
|
||||
(cryptpad discourse drone hedgedoc keycloak mailu mumble wordpress — several had stale survey data;
|
||||
each re-verified against true upstream before skipping). GREEN PRs: bluesky-pds #6 · custom-html #10 ·
|
||||
custom-html-tiny #9 · ghost #8 · gitea #11 · immich #6 · lasuite-meet #12 · matrix-synapse #8 · n8n #9.
|
||||
Summary: .cc-ci-logs/upgrades/upgrade-all-2026-10-23.md (per-recipe logs alongside). Step-0 sweep
|
||||
reclaimed 19GB images (disk 61%->49%); end-of-run reap 0 leaked dev deploys.
|
||||
NEVER MERGED anything; PRs await operator review + `abra recipe release` after each merge.
|
||||
### Event 2026-10-30 02:00 UTC — weekly /upgrade-all run complete (DEFAULT mode, nothing merged)
|
||||
Run 2026-10-30: 19 considered · 12 upgraded with !testme GREEN · 1 PR open pending its stale warm-canonical
|
||||
base (lasuite-docs PR #11 — !testme RED at the base INSTALL rung: canonical 0.4.0+v5.4.1 references the
|
||||
removed collab_ss secret; last week's minio blocker is already fixed via #47; head linted + live 12/12;
|
||||
operator: seed a v6-cold canonical, re-post !testme) · 1 failed (mailu PR #9 — restore tier RED 3x on a
|
||||
PRE-EXISTING upstream defect: main's new backup support lacks a restart post-hook so /data/main.db is
|
||||
never restored; PR + operator comment with options) · 5 skipped (gitea + lasuite-meet were already merged
|
||||
upstream by their turn; custom-html-tiny drone mumble up-to-date). GREEN PRs: bluesky-pds #6 · cryptpad #6
|
||||
(2026.5.1→2026.10.0 breaking server rewrite handled) · custom-html #10→11 (stale nginx digest re-pin) ·
|
||||
discourse #12 · ghost #8 · hedgedoc #3 (pgautoupgrade 16→18 in-place) · immich #7 (pgvectors 0.3.0
|
||||
same-scheme re-tag) · keycloak #13 (mariadb 12.3→13.0 + AUTO_UPGRADE) · lasuite-drive #8 (+collabora +
|
||||
onlyoffice direct checks) · matrix-synapse #8 · n8n #10 · wordpress #3.
|
||||
Summary: .cc-ci-logs/upgrades/upgrade-all-2026-10-30.md (per-recipe logs alongside). Step-0 sweep
|
||||
reclaimed 66.76GB images + orphan stacks (disk 92%->49%); end-of-run reap 0 leaked dev deploys.
|
||||
NEVER MERGED anything; PRs await operator review + `abra recipe release` after each merge.
|
||||
### Event 2026-10-30 19:00–21:30 UTC — rustdesk-server one-off upgrade + FULL ENROLLMENT (orchestrator)
|
||||
One-off upgrade requested from scratch: public mirror `recipe-maintainers/rustdesk-server` created +
|
||||
synced; recipe PR https://git.autonomic.zone/recipe-maintainers/rustdesk-server/pulls/1 (1.1.12→1.1.16,
|
||||
branch upgrade-2e65135, unmerged — operator merges). Upstream checked (low-churn, stray 1.4.5 docker
|
||||
tags = aborted push; CVE fix POV: UDP punch-hole amplification GH #670 + mio bump; scan 0 CVE-ids,
|
||||
client-side CVEs fixed in client 1.4.9). Dev-deploy converged clean twice, then !== enrollment: tests
|
||||
authored (status-key health sidecar overlay compose.ccci.yml serving the private-tool-unreachable
|
||||
real hbbs public key; ports 21115-21119 + UDP; ws 101 on 21118/19; identity-key cross-source assert;
|
||||
punch-hole-amplification regression test w/ version-gate), committed d71a53c, cc-ci PR #48.
|
||||
Verify: run 1 RED (custom tier) — hand-rolled version read returned None so the punch-hole gate
|
||||
didn't skip on the 1.1.12 head; fixed via harness lifecycle.deployed_identity + loud fail on
|
||||
unparsable tag (1653582, f5f11cd); runs 2+3 GREEN level 5/5 (recipe main with gate skipping, AND
|
||||
the upgrade head with the punch-hole test passing actively). PR #48 evidence-commented + MERGED
|
||||
(200); --cc-ci-only deploy: flake.lock cc-ci 2f6787f→a8b836e, lock commit 4e286ba pushed,
|
||||
/etc/cc-ci ff to a8b836e, detached nixos-rebuild test → health (0 failed, swarm 1/1) → switch
|
||||
(21:17 UTC). Bridge env confirms POLL_REPOS += recipe-maintainers/rustdesk-server (1/1, no auth
|
||||
errors). !testme posted on recipe PR #1 via testme-on-pr.sh → build 119 VERDICT=GREEN (level 5,
|
||||
lint pass, clean_teardown, no_secret_leak; card embedded in PR by bridge). Card saved:
|
||||
.cc-ci-logs/upgrades/rustdesk-server-testme-20261030.png. Next weekly /upgrade-all picks
|
||||
rustdesk-server up automatically (used-recipes.md row committed cd1ec01). /root/cc-ci-test-verify
|
||||
removed. Operator merges PR #1 then `abra recipe release`.
|
||||
|
||||
@@ -23,3 +23,4 @@ restarts the loops on boot. Count the lines below to see how often it's happenin
|
||||
- 2026-05-30 17:03:05 BST — reboot detected; loops auto-started by systemd (resuming phase index 6). boot_id=f565f752-0463-42db-b787-9e0db35a5e3f
|
||||
- 2026-05-31 03:38:29 UTC — reboot detected; loops auto-started by systemd (resuming phase index 5). boot_id=51c17fc3-8391-4109-bce2-413fbee6f26d
|
||||
- 2026-08-23 03:11:24 UTC — reboot detected; loops auto-started by systemd (resuming phase index 14). boot_id=8215fc5e-8032-4844-b2b8-e0f8c0d9b13e
|
||||
- 2026-09-28 19:54:59 UTC — reboot detected; loops auto-started by systemd (resuming phase index 14). boot_id=66dadc4e-7f44-493a-aee9-fa76952d78c3
|
||||
|
||||
+171
-27
@@ -40,6 +40,7 @@ from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import gzip
|
||||
import ipaddress
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
@@ -166,6 +167,43 @@ def _vkey(v: str | None) -> tuple:
|
||||
return tuple(out)
|
||||
|
||||
|
||||
def _nvd_affects(version: str | None, ranges: list[dict]) -> bool | None:
|
||||
"""Whether NVD marks a concrete image version affected; None means tag lacks precision."""
|
||||
current = _vkey(version)
|
||||
if not current:
|
||||
return None
|
||||
uncertain = False
|
||||
for row in ranges:
|
||||
exact = _vkey(row.get("version")) if row.get("version") not in (None, "*") else ()
|
||||
bounds = [(name, _vkey(row.get(name))) for name in (
|
||||
"versionStartIncluding", "versionStartExcluding",
|
||||
"versionEndIncluding", "versionEndExcluding") if row.get(name)]
|
||||
versions = ([exact] if exact else []) + [value for _, value in bounds]
|
||||
if any(not value for value in versions):
|
||||
uncertain = True
|
||||
continue
|
||||
if any(len(value) > len(current) and value[:len(current)] == current for value in versions):
|
||||
uncertain = True # e.g. postgres:13-alpine does not say which 13.x patch is running
|
||||
continue
|
||||
def compare(other):
|
||||
length = max(len(current), len(other))
|
||||
return (current + (0,) * (length - len(current))) > (other + (0,) * (length - len(other)))
|
||||
if exact and current != exact:
|
||||
continue
|
||||
def outside(name, value):
|
||||
if name == "versionStartExcluding":
|
||||
return not compare(value)
|
||||
if name == "versionStartIncluding":
|
||||
return not compare(value) and current != value
|
||||
if name == "versionEndIncluding":
|
||||
return compare(value)
|
||||
return compare(value) or current == value # versionEndExcluding
|
||||
if any(outside(name, value) for name, value in bounds):
|
||||
continue
|
||||
return True
|
||||
return None if uncertain else False
|
||||
|
||||
|
||||
def _already_fixed_on_from_line(kf: tuple, cands: list[tuple]) -> bool:
|
||||
"""Was it ALREADY fixed on the line we are upgrading FROM?
|
||||
|
||||
@@ -231,6 +269,16 @@ def registry_urls(recipe: str, registry_dir: str) -> tuple[list[str], str | None
|
||||
# recipe render '?' for no real reason — that is what put immich and n8n in the unknown
|
||||
# column on 2026-08-07 (https://docs.n8n.io/release-notes/` ← note the backtick).
|
||||
u = u.rstrip("`'\"*.,;:>)")
|
||||
if "…" in u or "..." in u:
|
||||
continue # prose placeholder, not a fetchable security source
|
||||
host = urllib.parse.urlsplit(u).hostname
|
||||
if not host or host == "localhost" or "." not in host:
|
||||
continue # a container/loopback URL in operational notes, not a public advisory source
|
||||
try:
|
||||
if not ipaddress.ip_address(host).is_global:
|
||||
continue
|
||||
except ValueError:
|
||||
pass
|
||||
if u and u not in urls:
|
||||
urls.append(u)
|
||||
return urls, path
|
||||
@@ -369,15 +417,57 @@ def vendor_pages(urls: list[str]) -> list[dict]:
|
||||
|
||||
NVD_API = "https://services.nvd.nist.gov/rest/json/cves/2.0"
|
||||
NVD_CPE_RE = re.compile(r"^\s*[-*]?\s*nvd-cpe:\s*(\S+)\s*=\s*(cpe:2\.3:[^\s`]+)", re.M | re.I)
|
||||
_NVD_CACHE: dict[str, dict] = {}
|
||||
COMMON_IMAGE_CPES = {
|
||||
"postgres": "cpe:2.3:a:postgresql:postgresql:*:*:*:*:*:*:*:*",
|
||||
"mariadb": "cpe:2.3:a:mariadb:mariadb:*:*:*:*:*:*:*:*",
|
||||
"mysql": "cpe:2.3:a:oracle:mysql:*:*:*:*:*:*:*:*",
|
||||
}
|
||||
|
||||
|
||||
def registry_cpes(recipe: str, registry_dir: str) -> list[tuple[str, str]]:
|
||||
"""[(image-key, cpe)] declared in the recipe's registry as `nvd-cpe: <key> = <cpe>`."""
|
||||
"""Explicit CPEs plus common database CPEs for images listed in the registry table."""
|
||||
path = os.path.join(registry_dir, f"{recipe}.md")
|
||||
try:
|
||||
return [(m.group(1), m.group(2)) for m in NVD_CPE_RE.finditer(open(path).read())]
|
||||
text = open(path).read()
|
||||
except OSError:
|
||||
return []
|
||||
out = [(m.group(1), m.group(2)) for m in NVD_CPE_RE.finditer(text)]
|
||||
explicit = {key for key, _ in out}
|
||||
for line in text.splitlines():
|
||||
if not line.startswith("|"):
|
||||
continue
|
||||
cells = [cell.strip().strip("`") for cell in line.strip().strip("|").split("|")]
|
||||
if len(cells) < 2:
|
||||
continue
|
||||
image = cells[1].split("/")[-1].lower()
|
||||
if image in COMMON_IMAGE_CPES and image not in explicit:
|
||||
out.append((image, COMMON_IMAGE_CPES[image]))
|
||||
explicit.add(image)
|
||||
return out
|
||||
|
||||
|
||||
def registry_image_sources(recipe: str, registry_dir: str) -> dict[str, set[str]]:
|
||||
"""Map image repo basenames to GitHub advisory sources from the registry table."""
|
||||
path = os.path.join(registry_dir, f"{recipe}.md")
|
||||
try:
|
||||
lines = open(path).read().splitlines()
|
||||
except OSError:
|
||||
return {}
|
||||
out: dict[str, set[str]] = {}
|
||||
for line in lines:
|
||||
if not line.startswith("|"):
|
||||
continue
|
||||
cells = [cell.strip().strip("`") for cell in line.strip().strip("|").split("|")]
|
||||
if len(cells) < 3:
|
||||
continue
|
||||
image = cells[1].split("/")[-1].lower()
|
||||
match = re.search(r"https?://github\.com/([^/\s|]+)/([^/\s|)]+)", cells[2])
|
||||
if not match or image in ("image", "---", ""):
|
||||
continue
|
||||
source = f"github-advisories:{match.group(1)}/{match.group(2).removesuffix('.git')}"
|
||||
out.setdefault(image, set()).add(source)
|
||||
return out
|
||||
|
||||
|
||||
def nvd_advisories(cpe: str, key: str) -> dict:
|
||||
@@ -395,6 +485,8 @@ def nvd_advisories(cpe: str, key: str) -> dict:
|
||||
NVD LAGS the vendor (it had neither gitea CVSS-9.8 RCE at publication), so this is a fallback,
|
||||
never a replacement for 2a/2b. Unauthenticated calls are rate-limited to ~5/30s, hence the retry.
|
||||
"""
|
||||
if cpe in _NVD_CACHE:
|
||||
return {**_NVD_CACHE[cpe], "source": f"nvd:{key}"}
|
||||
entry = {"source": f"nvd:{key}", "status": "ok", "advisories": []}
|
||||
url = f"{NVD_API}?resultsPerPage=2000&virtualMatchString={urllib.parse.quote(cpe)}"
|
||||
data = None
|
||||
@@ -407,15 +499,31 @@ def nvd_advisories(cpe: str, key: str) -> dict:
|
||||
entry["status"] = f"error: {type(e).__name__}"
|
||||
return entry
|
||||
time.sleep(8)
|
||||
for v in (data or {}).get("vulnerabilities", []):
|
||||
vulnerabilities = (data or {}).get("vulnerabilities", [])
|
||||
if (data or {}).get("totalResults", len(vulnerabilities)) > len(vulnerabilities):
|
||||
entry["status"] = "error: NVD result set truncated"
|
||||
return entry
|
||||
for v in vulnerabilities:
|
||||
c = v.get("cve") or {}
|
||||
cid = c.get("id")
|
||||
if not cid:
|
||||
if not cid or c.get("vulnStatus") == "Rejected":
|
||||
continue
|
||||
fixed, affected_max = set(), set()
|
||||
fixed, affected_max, affected_ranges = set(), set(), []
|
||||
cpe_product = ":".join(cpe.split(":")[:5]) + ":"
|
||||
cpe_edition = cpe.split(":")[9] if len(cpe.split(":")) > 9 else "*"
|
||||
for cfg in c.get("configurations", []):
|
||||
for node in cfg.get("nodes", []):
|
||||
for m in node.get("cpeMatch", []):
|
||||
criteria = m.get("criteria") or ""
|
||||
if not m.get("vulnerable") or not criteria.startswith(cpe_product):
|
||||
continue
|
||||
edition = criteria.split(":")[9] if len(criteria.split(":")) > 9 else "*"
|
||||
if cpe_edition != "*" and edition not in (cpe_edition, "*"):
|
||||
continue
|
||||
affected_ranges.append({k: m.get(k) for k in (
|
||||
"versionStartIncluding", "versionStartExcluding",
|
||||
"versionEndIncluding", "versionEndExcluding") if m.get(k)})
|
||||
affected_ranges[-1]["version"] = criteria.split(":")[5]
|
||||
if m.get("versionEndExcluding"):
|
||||
fixed.add(m["versionEndExcluding"])
|
||||
elif m.get("versionEndIncluding"):
|
||||
@@ -433,9 +541,11 @@ def nvd_advisories(cpe: str, key: str) -> dict:
|
||||
"vulnerable_range": None,
|
||||
"patched": "; ".join(sorted(fixed)) or None,
|
||||
"affected_max": "; ".join(sorted(affected_max)) or None,
|
||||
"affected_ranges": affected_ranges or None,
|
||||
"url": f"https://nvd.nist.gov/vuln/detail/{cid}",
|
||||
"published_at": c.get("published"), "description": None, "cvss": None,
|
||||
})
|
||||
_NVD_CACHE[cpe] = entry
|
||||
return entry
|
||||
|
||||
|
||||
@@ -737,7 +847,8 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
|
||||
"vulnerable_range": None, "patched": None,
|
||||
"context": None, "published_at": None,
|
||||
"description": None, "url": None, "cvss": None,
|
||||
"changelog_fixed_in": None, "affected_max": None})
|
||||
"changelog_fixed_in": None, "affected_max": None,
|
||||
"affected_ranges": None})
|
||||
if src not in e["sources"]:
|
||||
e["sources"].append(src)
|
||||
for k, v in extra.items():
|
||||
@@ -748,8 +859,9 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
|
||||
report["sources"].append({"source": entry["source"], "status": entry["status"],
|
||||
"found": len(entry.get("advisories", []))})
|
||||
for a in entry.get("advisories", []):
|
||||
if a.get("cve"):
|
||||
record(a["cve"], entry["source"], severity=a.get("severity"), ghsa=a.get("ghsa"),
|
||||
advisory_id = a.get("cve") or a.get("ghsa")
|
||||
if advisory_id:
|
||||
record(advisory_id, entry["source"], severity=a.get("severity"), ghsa=a.get("ghsa"),
|
||||
vulnerable_range=a.get("vulnerable_range"), patched=a.get("patched"),
|
||||
context=a.get("summary"), published_at=a.get("published_at"),
|
||||
description=a.get("description"), url=a.get("url"), cvss=a.get("cvss"))
|
||||
@@ -761,6 +873,7 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
|
||||
for a in entry.get("advisories", []):
|
||||
record(a["cve"], entry["source"], severity=a.get("severity"),
|
||||
patched=a.get("patched"), affected_max=a.get("affected_max"),
|
||||
affected_ranges=a.get("affected_ranges"),
|
||||
context=a.get("summary"), published_at=a.get("published_at"),
|
||||
url=a.get("url"))
|
||||
|
||||
@@ -825,18 +938,25 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
|
||||
if pname in k or k in pname:
|
||||
windows[src] = (v_from, v_to)
|
||||
window_key[src] = k
|
||||
image_sources = registry_image_sources(recipe, registry_dir)
|
||||
for key, wf, wt in (images or []):
|
||||
matched = False
|
||||
explicit = image_sources.get(key.lower(), set())
|
||||
for src in gh_sources:
|
||||
if src in windows:
|
||||
continue
|
||||
# Match BOTH ways: an image name is often longer than its source repo
|
||||
# (`clickhouse/clickhouse-server` vs source `ClickHouse/ClickHouse`) and sometimes
|
||||
# shorter (`redis` vs `redis/redis`). One-directional matching silently dropped the
|
||||
# clickhouse window when the key was derived from a compose file.
|
||||
k, name = key.lower(), src.split("/")[-1].lower()
|
||||
if k in src.lower() or name in k:
|
||||
if src in explicit or k in src.lower() or name in k:
|
||||
if src in windows:
|
||||
matched |= windows[src] == (wf, wt)
|
||||
continue # two images in one monorepo share a source and version window
|
||||
windows[src] = (wf, wt)
|
||||
window_key[src] = key
|
||||
matched = True
|
||||
if not matched:
|
||||
report.setdefault("unmatched_image_windows", []).append(key)
|
||||
report["windows"] = {k: {"from": f, "to": t} for k, (f, t) in windows.items()}
|
||||
|
||||
def _classify_window(src, wf, wt):
|
||||
@@ -844,6 +964,21 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
|
||||
|
||||
`indeterminate` = advisories from this source that the method COULD NOT JUDGE (no usable
|
||||
patched version, or no publish date). They are neither counted nor dismissed."""
|
||||
if src.startswith("nvd:"):
|
||||
got, undecidable = set(), set()
|
||||
for advisory, evidence in report["cves"].items():
|
||||
if src not in evidence["sources"]:
|
||||
continue
|
||||
if not evidence.get("affected_ranges"):
|
||||
undecidable.add(advisory)
|
||||
continue
|
||||
old = _nvd_affects(wf, evidence["affected_ranges"])
|
||||
new = _nvd_affects(wt, evidence["affected_ranges"])
|
||||
if old is True and new is False:
|
||||
got.add(advisory)
|
||||
elif old is None or (old is True and new is None):
|
||||
undecidable.add(advisory)
|
||||
return got, "NVD affected-version ranges", None, False, undecidable
|
||||
kf, kt = _vkey(wf), _vkey(wt)
|
||||
# A version-SCHEME change (semver 3.5.3 → calver 2026.7.1) makes numeric ordering
|
||||
# meaningless: 2025.12.2 compares "newer" than 3.5.3 while shipping earlier.
|
||||
@@ -1012,6 +1147,7 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
|
||||
report["cves"][cve]["classification"] = "indeterminate: no fix version published"
|
||||
unknown = [c for c in unknown if c not in fixed_set]
|
||||
report["fixed_by_this_upgrade"] = sorted(fixed_set)
|
||||
report["fixed_non_cve_advisories"] = sorted(x for x in fixed_set if not x.startswith("CVE-"))
|
||||
report["indeterminate"] = sorted(indeterminate)
|
||||
report["cve_count_indeterminate"] = len(indeterminate)
|
||||
report["unclassified"] = sorted(unknown)
|
||||
@@ -1020,24 +1156,22 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
|
||||
# A scan with NO usable source has not measured anything, so it must not report a number —
|
||||
# least of all 0, which asserts safety. mumble had no cc-ci-plan/upstream/mumble.md at all and
|
||||
# still produced "0 identified", which was then published as a clean 0 in a CVE report.
|
||||
usable_sources = [
|
||||
s for s in report["sources"]
|
||||
if s["status"] == "ok" or s["status"].startswith("no-advisories-published")
|
||||
]
|
||||
usable_sources = [s for s in report["sources"] if s["status"] == "ok"]
|
||||
no_sources = not usable_sources
|
||||
if no_sources:
|
||||
report["no_usable_sources"] = True
|
||||
report["count_known"] = not unresolved_any and not no_sources
|
||||
report["cve_count_fixed"] = (len(fixed_set)
|
||||
if (not unresolved_any and not no_sources) else None)
|
||||
report["cve_count_total_seen"] = len(report["cves"])
|
||||
# Only GENUINE failures make a count unreliable. "no-advisories-published" (404: the repo has
|
||||
# no advisory feed) and "skipped: template URL" are benign and must not degrade the verdict.
|
||||
report["sources_failed"] = [
|
||||
s["source"]
|
||||
for s in report["sources"]
|
||||
if not (s["status"] == "ok" or s["status"].startswith(("no-advisories-published", "skipped:")))
|
||||
]
|
||||
coverage_failed = bool(report.get("unmatched_image_windows")) or bool(v_from and not primary)
|
||||
report["count_known"] = not unresolved_any and not no_sources and not coverage_failed and not report["sources_failed"]
|
||||
report["cve_count_fixed"] = (sum(x.startswith("CVE-") for x in fixed_set)
|
||||
if report["count_known"] else None)
|
||||
report["cve_count_total_seen"] = sum(x.startswith("CVE-") for x in report["cves"])
|
||||
# Only GENUINE failures make a count unreliable. "no-advisories-published" (404: the repo has
|
||||
# no advisory feed) and "skipped: template URL" are benign and must not degrade the verdict.
|
||||
report["sources_benign"] = [
|
||||
s["source"]
|
||||
for s in report["sources"]
|
||||
@@ -1052,16 +1186,20 @@ def markdown(rep: dict) -> str:
|
||||
f"{rep.get('from') or '?'} → {rep.get('to') or '?'}"]
|
||||
if not rep.get("count_known", True):
|
||||
L.append("\n**CVEs fixed by this upgrade: UNKNOWN — the scan could NOT determine a count.**")
|
||||
if rep.get("fixed_by_this_upgrade"):
|
||||
L.append(f"\nAt least {len(rep['fixed_by_this_upgrade'])} fixed advisory/advisories were identified: "
|
||||
+ ", ".join(rep["fixed_by_this_upgrade"][:15]))
|
||||
if rep.get("no_usable_sources"):
|
||||
L.append("\n⚠ This is NOT zero. **No usable source was checked at all** — the registry "
|
||||
"file `cc-ci-plan/upstream/<recipe>.md` is missing or every source failed, so "
|
||||
"nothing was measured. Render this recipe's cve cell as `?`, never `0`, and add "
|
||||
"the registry file.")
|
||||
else:
|
||||
L.append("\n⚠ This is NOT zero. A version-scheme change (e.g. semver → calver) makes "
|
||||
"numeric ordering meaningless across this jump, so no advisory could be "
|
||||
"classified. Render this recipe's cve cell as `?`, never `0`. Read the vendor's "
|
||||
"release notes for the jump and count by hand.")
|
||||
L.append("\n⚠ This is NOT zero. At least one version window or advisory source could "
|
||||
"not be checked completely. Render this recipe's cve cell as `?`, never `0`.")
|
||||
if rep.get("unmatched_image_windows"):
|
||||
L.append("\n⚠ image windows with no matching advisory source: "
|
||||
+ ", ".join(rep["unmatched_image_windows"]))
|
||||
if rep["unclassified"]:
|
||||
L.append(f"\nAdvisories seen but unclassifiable ({len(rep['unclassified'])}) — includes "
|
||||
f"other images in this recipe: " + ", ".join(rep["unclassified"][:12]))
|
||||
@@ -1072,7 +1210,8 @@ def markdown(rep: dict) -> str:
|
||||
f"reading in the upgrade step._")
|
||||
return "\n".join(L)
|
||||
ind = rep.get("cve_count_indeterminate") or 0
|
||||
if rep["fixed_by_this_upgrade"]:
|
||||
fixed_cves = [x for x in rep["fixed_by_this_upgrade"] if x.startswith("CVE-")]
|
||||
if fixed_cves:
|
||||
floor = " (at least — see undetermined below)" if ind else ""
|
||||
L.append(f"\n**CVEs fixed by this upgrade: {rep['cve_count_fixed']}**{floor}\n")
|
||||
cb = rep.get("classified_by") or {}
|
||||
@@ -1086,12 +1225,17 @@ def markdown(rep: dict) -> str:
|
||||
L.append("")
|
||||
L.append("| CVE | severity | fixed in | advisory | source |")
|
||||
L.append("|---|---|---|---|---|")
|
||||
for cve in rep["fixed_by_this_upgrade"]:
|
||||
for cve in fixed_cves:
|
||||
e = rep["cves"][cve]
|
||||
L.append(f"| {cve} | {e.get('severity') or '?'} | {e.get('patched') or '?'} | "
|
||||
f"{e.get('ghsa') or '-'} | {e['sources'][0]} |")
|
||||
else:
|
||||
L.append("\n**CVEs fixed by this upgrade: 0 identified by the deterministic scan.**")
|
||||
if rep.get("fixed_non_cve_advisories"):
|
||||
L.append("\n**Security advisories fixed without CVE IDs: "
|
||||
+ str(len(rep["fixed_non_cve_advisories"])) + "** — "
|
||||
+ ", ".join(rep["fixed_non_cve_advisories"][:15])
|
||||
+ ". In the report CVEs column, use YES when the CVE count is zero.")
|
||||
if ind:
|
||||
L.append(f"\n⚠ **{ind} advisory/advisories could NOT be judged** — the vendor published no fix "
|
||||
f"version (GitHub carries `TBD` or a placeholder like `7.4.X`) and the vulnerable "
|
||||
|
||||
+15
-18
@@ -134,7 +134,7 @@ def security_source_audit(recipe: str) -> list[dict]:
|
||||
e = A.nvd_advisories(cpe, key)
|
||||
n = len(e.get("advisories") or [])
|
||||
out.append({"source": e["source"] + f" ({cpe.split(':')[4]}/{cpe.split(':')[3]})",
|
||||
"kind": "advisory-feed" if n else "no-cve-data",
|
||||
"kind": "advisory-feed", "verified_empty": n == 0,
|
||||
"status": e["status"], "cves": n, "usable": n})
|
||||
for entry in A.github_advisories(urls):
|
||||
out.append({"source": entry["source"], "kind": "advisory-feed",
|
||||
@@ -227,16 +227,7 @@ def audit_recipe(recipe: str, ssh: str | None, quiet_days: int) -> dict:
|
||||
|
||||
|
||||
def _registry_urls(recipe: str):
|
||||
path = os.path.join(REGISTRY_DIR, f"{recipe}.md")
|
||||
if not os.path.exists(path):
|
||||
return [], None
|
||||
text = open(path).read()
|
||||
urls = []
|
||||
for u in re.findall(r"https?://[^\s)|\]]+", text):
|
||||
u = u.rstrip("`'\"*.,;:>)")
|
||||
if u and u not in urls:
|
||||
urls.append(u)
|
||||
return urls, path
|
||||
return A.registry_urls(recipe, REGISTRY_DIR)
|
||||
|
||||
|
||||
def all_recipes() -> list[str]:
|
||||
@@ -272,23 +263,29 @@ def main() -> int:
|
||||
blind_recipes, noisy = [], 0
|
||||
for r in recipes:
|
||||
rows = security_source_audit(r)
|
||||
feeds = [x for x in rows if x["kind"] == "advisory-feed" and x["cves"] > 0]
|
||||
logs = [x for x in rows if x["kind"] == "changelog"]
|
||||
feeds = [x for x in rows if x["kind"] == "advisory-feed"
|
||||
and (x["cves"] > 0 or x.get("verified_empty"))
|
||||
and x["status"] == "ok"]
|
||||
logs = [x for x in rows if x["kind"] == "changelog" and x["status"] == "ok"]
|
||||
unattr = [x for x in rows if x["kind"] == "unattributable"]
|
||||
failed = [x for x in rows if x["status"].startswith("error:")]
|
||||
noisy += len(unattr)
|
||||
usable = len(feeds) + len(logs)
|
||||
if usable == 0:
|
||||
blind_recipes.append(r)
|
||||
print(f"!! {r}: NO USABLE CVE SOURCE — {len(unattr)} unparseable page(s), "
|
||||
f"0 advisory feeds, 0 attributable changelogs")
|
||||
print(f"!! {r}: NO VERIFIED CVE SOURCE — {len(unattr)} unparseable page(s), "
|
||||
f"0 populated advisory feeds, 0 attributable changelogs, {len(failed)} failed source(s)")
|
||||
for x in rows:
|
||||
print(f" {x['kind']:15} {x['source'][:64]} ({x['cves']} CVEs)")
|
||||
unit = "advisories" if x["kind"] == "advisory-feed" else "CVE IDs"
|
||||
print(f" {x['kind']:15} {x['source'][:64]} ({x['cves']} {unit}; {x['status']})")
|
||||
else:
|
||||
print(f"OK {r}: {len(feeds)} advisory-feed(s), {len(logs)} changelog(s)"
|
||||
+ (f", {len(unattr)} unparseable page(s) (redundant — covered by a feed)"
|
||||
if unattr else ""))
|
||||
+ (f", {len(unattr)} unparseable page(s)" if unattr else "")
|
||||
+ (f", {len(failed)} failed source(s) — counts incomplete" if failed else ""))
|
||||
for x in logs:
|
||||
print(f" changelog {x['source'][:62]} ({x['usable']}/{x['cves']})")
|
||||
for x in failed:
|
||||
print(f" FAILED {x['source'][:90]} ({x['status']})")
|
||||
print(f"\n{len(recipes)} recipes · {len(blind_recipes)} with NO usable CVE source"
|
||||
+ (f": {', '.join(blind_recipes)}" if blind_recipes else "")
|
||||
+ f" · {noisy} unparseable page(s) elsewhere (harmless where a feed covers them)")
|
||||
|
||||
@@ -30,7 +30,7 @@ SPEC SHAPE (the agent writes this JSON):
|
||||
{"date":"YYYY-MM-DD","subtitle":"Week of <human date>",
|
||||
"lead":"<ONE short paragraph>",
|
||||
"table":[{"recipe":"x","change":"a → b","status":"GREEN|FAILED|STALE|SKIPPED|UPTODATE", # -> TESTS col
|
||||
"cve":16, # number of CVEs this PR fixes; 0/omit for none
|
||||
"cve":16, # CVE count; "YES" for advisory without CVE ID; "?" unknown
|
||||
"ci":"build 154 ✓","ci_url":"…","pr":"#4","pr_url":"…","notes":"…"}],
|
||||
# ROWS SORTED by recommended priority to address — recipes WITH CVEs first, then failures,
|
||||
# then stale-tests, then routine green, then up-to-date/skipped.
|
||||
@@ -232,6 +232,8 @@ def _table(rows, repo_url=None):
|
||||
# int is the confirmed CVE count; 0/omit is a confirmed-clean scan.
|
||||
if isinstance(cve, str) and cve.strip() == "?":
|
||||
cve_cell = '<span class="muted" title="advisory scan incomplete or absent — CVE count unknown">?</span>'
|
||||
elif isinstance(cve, str) and cve.strip().upper() == "YES":
|
||||
cve_cell = '<span class="cve">YES</span>'
|
||||
elif isinstance(cve, (int, float)) and cve:
|
||||
cve_cell = f'<span class="cve">{int(cve)}</span>'
|
||||
else:
|
||||
|
||||
@@ -163,7 +163,8 @@ def parse_ref(ref: str) -> dict:
|
||||
return {"registry": host, "repo": repo, "tag": tag, "digest": digest}
|
||||
|
||||
|
||||
HUB_RECENT_PAGES = int(os.environ.get("RESOLVE_IMAGES_HUB_PAGES", "10"))
|
||||
HUB_RECENT_PAGES = int(os.environ.get("RESOLVE_IMAGES_HUB_PAGES", "40"))
|
||||
_HUB_TRUNCATED: set[str] = set()
|
||||
|
||||
|
||||
def _hub_tag_exists(repo: str, tag: str) -> bool:
|
||||
@@ -177,10 +178,10 @@ def _hub_tag_exists(repo: str, tag: str) -> bool:
|
||||
def _hub_tags(repo: str) -> list[str]:
|
||||
"""Recently-pushed tags, newest first.
|
||||
|
||||
Popular Docker Hub repos carry many thousands of tags, so a full enumeration is impractical —
|
||||
but it is also unnecessary: a tag NEWER than the one we run must have been pushed AFTER it, so
|
||||
ordering by last_updated and reading a bounded recent window is sufficient to find any upgrade.
|
||||
(ghcr offers no ordering, which is why that path needs a different strategy.)
|
||||
Popular Docker Hub repos carry many thousands of tags, so enumerate a bounded recent window.
|
||||
A rebuilt old tag can displace a newer version from that window, so callers MUST treat a
|
||||
truncated listing with no detected upgrade as unverified, not current. (ghcr offers no recency
|
||||
ordering, which is why that path needs a different strategy.)
|
||||
"""
|
||||
tags, url = [], (f"https://hub.docker.com/v2/repositories/{repo}/tags"
|
||||
f"?page_size=100&ordering=last_updated")
|
||||
@@ -191,6 +192,8 @@ def _hub_tags(repo: str) -> list[str]:
|
||||
url = d.get("next")
|
||||
if not url:
|
||||
break
|
||||
if url:
|
||||
_HUB_TRUNCATED.add(repo)
|
||||
return tags
|
||||
|
||||
|
||||
@@ -236,15 +239,26 @@ def _oci_tags(host: str, repo: str) -> list[str]:
|
||||
hdrs = _oci_bearer(host, repo)
|
||||
tags, url = [], f"https://{host}/v2/{repo}/tags/list?n=1000"
|
||||
for _ in range(MAX_TAG_PAGES):
|
||||
for attempt in range(RETRIES):
|
||||
req = urllib.request.Request(url, headers={"User-Agent": UA, **hdrs})
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=TIMEOUT) as r:
|
||||
tags += (json.load(r) or {}).get("tags") or []
|
||||
link = r.headers.get("Link", "") or ""
|
||||
break
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code not in (429, 503) or attempt == RETRIES - 1:
|
||||
raise
|
||||
retry_after = e.headers.get("Retry-After") or ""
|
||||
delay = min(int(retry_after) if retry_after.isdigit() else 2 ** (attempt + 1), 60)
|
||||
time.sleep(delay)
|
||||
m = re.search(r'<([^>]+)>;\s*rel="next"', link)
|
||||
if not m:
|
||||
break
|
||||
nxt = m.group(1)
|
||||
url = f"https://{host}{nxt}" if nxt.startswith("/") else nxt
|
||||
else:
|
||||
raise RuntimeError(f"tag listing truncated after {MAX_TAG_PAGES} pages")
|
||||
return tags
|
||||
|
||||
|
||||
@@ -299,6 +313,39 @@ def _release_fallback_repos(registry: str, repo: str) -> list[tuple[str, str]]:
|
||||
return cands
|
||||
|
||||
|
||||
def _oci_tag_exists(host: str, repo: str, tag: str) -> bool:
|
||||
"""Verify a release candidate is actually published as an image manifest."""
|
||||
hdrs = {"User-Agent": UA,
|
||||
"Accept": "application/vnd.oci.image.index.v1+json, "
|
||||
"application/vnd.docker.distribution.manifest.list.v2+json, "
|
||||
"application/vnd.oci.image.manifest.v1+json",
|
||||
**_oci_bearer(host, repo)}
|
||||
url = f"https://{host}/v2/{repo}/manifests/{urllib.parse.quote(tag, safe='')}"
|
||||
try:
|
||||
with urllib.request.urlopen(urllib.request.Request(url, headers=hdrs, method="HEAD"),
|
||||
timeout=TIMEOUT) as response:
|
||||
return response.status == 200
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code == 404:
|
||||
return False
|
||||
raise
|
||||
|
||||
|
||||
def _release_manifest_fallback(registry: str, repo: str, current: str) -> tuple[list[str], str] | None:
|
||||
"""For huge GHCR registries, use release tags only after checking actual manifests."""
|
||||
for owner, name in _release_fallback_repos(registry, repo):
|
||||
releases = github_release_tags(owner, name)
|
||||
if current not in releases:
|
||||
continue # incomplete release window; it cannot establish a current baseline
|
||||
candidates = sorted((tag for tag in set(releases)
|
||||
if shape(tag) == shape(current) and vkey(tag) >= vkey(current)),
|
||||
key=vkey, reverse=True)
|
||||
for tag in candidates:
|
||||
if _oci_tag_exists(registry, repo, tag):
|
||||
return [current, tag], f"github-releases+registry-manifest:{owner}/{name}"
|
||||
return None
|
||||
|
||||
|
||||
_TAG_CACHE: dict[tuple[str, str], tuple[list[str], str | None]] = {}
|
||||
|
||||
|
||||
@@ -335,8 +382,16 @@ def resolve(ref: str) -> dict:
|
||||
"upgrade_available": False}
|
||||
tags, err = list_tags(info["registry"], info["repo"])
|
||||
if err:
|
||||
try:
|
||||
fallback = (_release_manifest_fallback(info["registry"], info["repo"], info["tag"])
|
||||
if "tag listing truncated" in err else None)
|
||||
except Exception as e: # noqa: BLE001 — a failed fallback must stay unresolved
|
||||
out["status"] = f"error: registry {err}; release fallback {type(e).__name__}: {e}"
|
||||
return out
|
||||
if not fallback:
|
||||
out["status"] = f"error: {err}"
|
||||
return out
|
||||
tags, out["source"] = fallback
|
||||
out["tags_seen"] = len(set(tags))
|
||||
# INTEGRITY CHECK: the tag we are currently running MUST appear in the listing. If it does not,
|
||||
# the listing is incomplete and any "newest" derived from it is a guess — ghcr paginates to tens
|
||||
@@ -367,8 +422,9 @@ def resolve(ref: str) -> dict:
|
||||
same = [t for t in set(tags) if shape(t) == want and vkey(t)]
|
||||
out["candidates"] = len(same)
|
||||
if not same:
|
||||
# Not a failure: digest-only pins and `latest`/`stable` have no comparable siblings.
|
||||
out["status"] = "no comparable tags (shape has no numeric siblings)"
|
||||
# A floating or unusual tag cannot prove that an image is current. Keep it visible to
|
||||
# callers rather than letting an empty candidate set become a false UPTODATE verdict.
|
||||
out["status"] = "unverifiable: no comparable numeric tags"
|
||||
return out
|
||||
newest = max(same, key=vkey)
|
||||
out["newest_same_shape"] = newest
|
||||
@@ -377,6 +433,10 @@ def resolve(ref: str) -> dict:
|
||||
if within:
|
||||
out["newest_within_major"] = max(within, key=vkey)
|
||||
out["upgrade_available"] = bool(cur and vkey(newest) > cur)
|
||||
if (info["registry"] in ("docker.io", "registry-1.docker.io")
|
||||
and info["repo"] in _HUB_TRUNCATED and not out["upgrade_available"]):
|
||||
out["status"] = (f"unverifiable: Docker Hub listing truncated after "
|
||||
f"{HUB_RECENT_PAGES * 100} recent tags")
|
||||
return out
|
||||
|
||||
|
||||
@@ -436,16 +496,16 @@ def main() -> int:
|
||||
"recipe": a.recipe,
|
||||
"images": results,
|
||||
"upgrades_available": [r["ref"] for r in results if r["upgrade_available"]],
|
||||
"unresolved": [r["ref"] for r in results if r["status"].startswith("error")],
|
||||
"unresolved": [r["ref"] for r in results if r["status"].startswith(("error", "skipped", "unverifiable"))],
|
||||
# The whole point: distinguish "checked, current" from "could not check".
|
||||
"all_resolved": not any(r["status"].startswith("error") for r in results),
|
||||
"all_resolved": bool(results) and all(r["status"] == "ok" for r in results),
|
||||
}
|
||||
if not a.table:
|
||||
print(json.dumps(report, indent=2))
|
||||
return 0
|
||||
print(f"{a.recipe} — {len(results)} images")
|
||||
for r in results:
|
||||
flag = "UPGRADE" if r["upgrade_available"] else ("ERROR" if r["status"].startswith("error") else "current")
|
||||
flag = "UPGRADE" if r["upgrade_available"] else ("UNKNOWN" if r["status"] != "ok" else "current")
|
||||
print(f" [{flag:7}] {r['repo']}:{r['tag']}" + (" (digest-pinned)" if r["digest"] else ""))
|
||||
print(f" shape={r['shape']} candidates={r['candidates']}"
|
||||
f" newest_same_shape={r['newest_same_shape']}"
|
||||
|
||||
@@ -22,6 +22,7 @@ import io
|
||||
import json
|
||||
import os
|
||||
import pathlib
|
||||
import tempfile
|
||||
import sys
|
||||
import unittest
|
||||
import unittest.mock
|
||||
@@ -51,7 +52,7 @@ def vendor(url, cves=(), status="ok"):
|
||||
|
||||
|
||||
def run_scan(gh_entries=(), vendor_entries=(), tag_dates=None, *, v_from=None, v_to=None,
|
||||
images=None, recipe="fixture", urls=None, releases=None):
|
||||
images=None, recipe="fixture", urls=None, releases=None, image_sources=None):
|
||||
"""scan() with every network call replaced by fixtures.
|
||||
|
||||
`releases` maps CVE id -> tags whose release notes name it (the third, release-note method)."""
|
||||
@@ -61,6 +62,7 @@ def run_scan(gh_entries=(), vendor_entries=(), tag_dates=None, *, v_from=None, v
|
||||
with unittest.mock.patch.object(A, "registry_urls", lambda r, d: (list(urls), "/fake/reg.md")), \
|
||||
unittest.mock.patch.object(A, "github_advisories", lambda u: list(gh_entries)), \
|
||||
unittest.mock.patch.object(A, "vendor_pages", lambda u: list(vendor_entries)), \
|
||||
unittest.mock.patch.object(A, "registry_image_sources", lambda r, d: image_sources or {}), \
|
||||
unittest.mock.patch.object(A, "osv", lambda r, v: None), \
|
||||
unittest.mock.patch.object(A, "_tag_date", lambda o, r, v: tag_dates.get(v)), \
|
||||
unittest.mock.patch.object(A, "release_fix_versions", lambda src, cve: list(releases.get(cve, []))):
|
||||
@@ -153,6 +155,26 @@ class TestRegistryUrls(unittest.TestCase):
|
||||
p.write_text(text)
|
||||
return A.registry_urls("r", tmp)
|
||||
|
||||
def test_prose_ellipsis_is_not_a_security_source(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
urls, _ = self._write(directory,
|
||||
"https://github.com/cryptpad/cryptpad and example `https://…` URLs "
|
||||
"http://docspec:3000/conversion http://127.0.0.1:3000/health")
|
||||
self.assertEqual(urls, ["https://github.com/cryptpad/cryptpad"])
|
||||
|
||||
def test_image_source_table_maps_monorepo_images(self):
|
||||
mapping = A.registry_image_sources("lasuite-drive", str(HERE / "upstream"))
|
||||
self.assertIn("github-advisories:suitenumerique/drive", mapping["drive-backend"])
|
||||
self.assertIn("github-advisories:CollaboraOnline/online", mapping["code"])
|
||||
|
||||
def test_common_database_cpes_follow_registry_images(self):
|
||||
self.assertIn(("postgres", A.COMMON_IMAGE_CPES["postgres"]),
|
||||
A.registry_cpes("immich", str(HERE / "upstream")))
|
||||
self.assertIn(("mariadb", A.COMMON_IMAGE_CPES["mariadb"]),
|
||||
A.registry_cpes("gitea", str(HERE / "upstream")))
|
||||
self.assertIn(("rustdesk-server", "cpe:2.3:a:rustdesk:rustdesk_server:*:*:*:*:oss:*:*:*"),
|
||||
A.registry_cpes("rustdesk-server", str(HERE / "upstream")))
|
||||
|
||||
def test_strips_trailing_markdown_punctuation(self):
|
||||
# Production defect: a captured backtick 404'd the fetch and rendered n8n/immich as '?'.
|
||||
import tempfile
|
||||
@@ -174,6 +196,25 @@ class TestRegistryUrls(unittest.TestCase):
|
||||
self.assertEqual((urls, path), ([], None))
|
||||
|
||||
|
||||
class TestNvdCompleteness(unittest.TestCase):
|
||||
def test_truncated_results_are_not_a_clean_zero(self):
|
||||
payload = json.dumps({"totalResults": 2, "vulnerabilities": [{"cve": {"id": "CVE-2026-0001"}}]})
|
||||
with unittest.mock.patch.object(A, "_fetch", return_value=payload):
|
||||
row = A.nvd_advisories("cpe:2.3:a:example:server:*:*:*:*:*:*:*:*", "server")
|
||||
self.assertEqual(row["status"], "error: NVD result set truncated")
|
||||
self.assertEqual(row["advisories"], [])
|
||||
|
||||
def test_database_backport_is_not_credited_again_on_major_upgrade(self):
|
||||
ranges = [
|
||||
{"version": "*", "versionStartIncluding": "15.0", "versionEndExcluding": "15.14"},
|
||||
{"version": "*", "versionStartIncluding": "16.0", "versionEndExcluding": "16.5"},
|
||||
]
|
||||
self.assertIs(A._nvd_affects("15.13", ranges), True)
|
||||
self.assertIs(A._nvd_affects("15.19", ranges), False)
|
||||
self.assertIs(A._nvd_affects("18.6", ranges), False)
|
||||
self.assertIsNone(A._nvd_affects("15-alpine", ranges))
|
||||
|
||||
|
||||
# ── C. --image argument parsing ───────────────────────────────────────────────────────────────────
|
||||
|
||||
class TestImageArgParsing(unittest.TestCase):
|
||||
@@ -290,12 +331,32 @@ class TestPerImageWindows(unittest.TestCase):
|
||||
self.assertEqual(rep["windows"]["github-advisories:discourse/discourse"],
|
||||
{"from": "3.5.3", "to": "3.5.4"})
|
||||
|
||||
def test_unmatched_image_name_is_silently_ignored(self):
|
||||
# Documents CURRENT behaviour: a typo'd name costs coverage without warning.
|
||||
def test_unmatched_image_name_makes_count_unknown(self):
|
||||
# A typo'd sidecar name must not silently turn its CVEs into a clean zero.
|
||||
rep = run_scan([self.APP], v_from="3.5.3", v_to="3.5.4",
|
||||
images=[("nosuchimage", "1", "2")], urls=self.URLS[:1])
|
||||
self.assertEqual(list(rep["windows"]), ["github-advisories:discourse/discourse"])
|
||||
self.assertFalse(rep["count_known"])
|
||||
self.assertEqual(rep["unmatched_image_windows"], ["nosuchimage"])
|
||||
self.assertIsNone(rep["cve_count_fixed"])
|
||||
|
||||
def test_registry_maps_image_to_differently_named_source(self):
|
||||
rep = run_scan([self.APP, gh("suitenumerique/drive", [adv("CVE-DRIVE-1", patched="0.25.3")])],
|
||||
v_from="3.5.3", v_to="3.5.4", images=[("drive-backend", "0.22.0", "0.25.3")],
|
||||
urls=self.URLS,
|
||||
image_sources={"drive-backend": {"github-advisories:suitenumerique/drive"}})
|
||||
self.assertEqual(rep.get("unmatched_image_windows"), None)
|
||||
self.assertIn("CVE-DRIVE-1", rep["fixed_by_this_upgrade"])
|
||||
|
||||
def test_monorepo_images_share_identical_window(self):
|
||||
source = "github-advisories:suitenumerique/drive"
|
||||
rep = run_scan([gh("suitenumerique/drive", [adv("CVE-DRIVE-1", patched="0.25.3")])],
|
||||
v_from="0.22.0", v_to="0.25.3",
|
||||
images=[("drive-backend", "0.22.0", "0.25.3")],
|
||||
image_sources={"drive-backend": {source}})
|
||||
self.assertTrue(rep["count_known"])
|
||||
self.assertEqual(rep.get("unmatched_image_windows"), None)
|
||||
self.assertEqual(rep["cve_count_fixed"], 1)
|
||||
|
||||
|
||||
class TestSchemeChangeDateFallback(unittest.TestCase):
|
||||
@@ -358,6 +419,15 @@ class TestCountGuarantees(unittest.TestCase):
|
||||
self.assertTrue(rep["count_known"])
|
||||
self.assertIn("0 identified", A.markdown(rep))
|
||||
|
||||
def test_ghsa_without_cve_is_preserved_as_security_advisory(self):
|
||||
entry = {"cve": None, "ghsa": "GHSA-aaaa-bbbb-cccc", "severity": "low",
|
||||
"patched": "1.1", "summary": "security fix", "published_at": None}
|
||||
rep = run_scan([gh("app/app", [entry])], v_from="1.0", v_to="1.1")
|
||||
self.assertTrue(rep["count_known"])
|
||||
self.assertEqual(rep["cve_count_fixed"], 0)
|
||||
self.assertEqual(rep["fixed_non_cve_advisories"], ["GHSA-aaaa-bbbb-cccc"])
|
||||
self.assertIn("use YES", A.markdown(rep))
|
||||
|
||||
def test_404_advisory_feed_is_benign_not_a_failure(self):
|
||||
rep = run_scan([gh("app/app", [adv("CVE-1", patched="1.1")]),
|
||||
gh("side/car", [], status="no-advisories-published")],
|
||||
@@ -377,6 +447,13 @@ class TestCountGuarantees(unittest.TestCase):
|
||||
v_from="1.0", v_to="1.1")
|
||||
self.assertIn("github-advisories:app/app", rep["sources_failed"])
|
||||
self.assertIn("FAILED", A.markdown(rep))
|
||||
self.assertFalse(rep["count_known"])
|
||||
|
||||
def test_only_404_feed_is_not_evidence_of_clean_scan(self):
|
||||
rep = run_scan([gh("app/app", [], status="no-advisories-published")],
|
||||
v_from="1.0", v_to="1.1")
|
||||
self.assertTrue(rep["no_usable_sources"])
|
||||
self.assertFalse(rep["count_known"])
|
||||
|
||||
def test_no_window_given_classifies_nothing(self):
|
||||
rep = run_scan([gh("app/app", [adv("CVE-1", patched="1.1")])])
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Offline regressions for the weekly image survey's false-current cases."""
|
||||
|
||||
import importlib.util
|
||||
import pathlib
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
HERE = pathlib.Path(__file__).resolve().parent
|
||||
spec = importlib.util.spec_from_file_location("resolve_images", HERE / "resolve-images.py")
|
||||
resolver = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(resolver)
|
||||
|
||||
|
||||
class TestResolverCoverage(unittest.TestCase):
|
||||
def test_cryptpad_calver_tag_is_found_independently_of_abra(self):
|
||||
with mock.patch.object(resolver, "list_tags", return_value=(
|
||||
["version-2026.5.1", "version-2026.10.0", "opendesk-2026.08.04"], None)):
|
||||
row = resolver.resolve("cryptpad/cryptpad:version-2026.5.1")
|
||||
self.assertTrue(row["upgrade_available"])
|
||||
self.assertEqual(row["newest_same_shape"], "version-2026.10.0")
|
||||
|
||||
def test_floating_and_templated_tags_cannot_prove_current(self):
|
||||
with mock.patch.object(resolver, "list_tags", return_value=(["alpine", "1.2.3"], None)):
|
||||
floating = resolver.resolve("atmoz/sftp:alpine")
|
||||
templated = resolver.resolve("ghost:${IMAGE_VERSION}-alpine")
|
||||
self.assertTrue(floating["status"].startswith("unverifiable:"))
|
||||
self.assertTrue(templated["status"].startswith("skipped:"))
|
||||
|
||||
def test_truncated_registry_listing_cannot_prove_current(self):
|
||||
with mock.patch.object(resolver, "list_tags", return_value=(["1.2.3"], None)):
|
||||
with mock.patch.object(resolver, "_HUB_TRUNCATED", {"library/example"}):
|
||||
row = resolver.resolve("example:1.2.3")
|
||||
self.assertTrue(row["status"].startswith("unverifiable:"))
|
||||
|
||||
def test_huge_ghcr_listing_uses_only_published_release_manifests(self):
|
||||
ref = "ghcr.io/immich-app/immich-machine-learning:v3.3.5"
|
||||
with mock.patch.object(resolver, "list_tags", return_value=([], "RuntimeError: tag listing truncated")), \
|
||||
mock.patch.object(resolver, "github_release_tags",
|
||||
return_value=["v3.3.6", "v3.3.5"]), \
|
||||
mock.patch.object(resolver, "_oci_tag_exists", side_effect=lambda h, r, t: t == "v3.3.5"):
|
||||
row = resolver.resolve(ref)
|
||||
self.assertEqual(row["status"], "ok")
|
||||
self.assertFalse(row["upgrade_available"]) # release exists; image does not yet
|
||||
self.assertIn("registry-manifest", row["source"])
|
||||
with mock.patch.object(resolver, "list_tags", return_value=([], "RuntimeError: tag listing truncated")), \
|
||||
mock.patch.object(resolver, "github_release_tags",
|
||||
return_value=["v3.3.6", "v3.3.5"]), \
|
||||
mock.patch.object(resolver, "_oci_tag_exists", return_value=True):
|
||||
upgraded = resolver.resolve(ref)
|
||||
self.assertTrue(upgraded["upgrade_available"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -2,11 +2,26 @@
|
||||
|
||||
| service | image | source repo | releases / changelog |
|
||||
|---------|-------|-------------|----------------------|
|
||||
| app | cryptpad/cryptpad | https://github.com/cryptpad/cryptpad | https://github.com/cryptpad/cryptpad/releases |
|
||||
| app | cryptpad/cryptpad | https://github.com/cryptpad/cryptpad | https://github.com/cryptpad/cryptpad/releases · https://hub.docker.com/r/cryptpad/cryptpad/tags |
|
||||
| web | nginx | https://github.com/nginx/nginx | https://nginx.org/en/CHANGES |
|
||||
|
||||
## Standing notes
|
||||
- `abra recipe upgrade` cannot parse the `version-YYYY.M.D` tag format of the cryptpad/cryptpad image,
|
||||
so the app image is bumped by hand; the nginx sidecar is what `abra recipe upgrade` actually moves.
|
||||
- Only use tags in the canonical `version-YYYY.M.x` scheme. The `opendesk-YYYY.MM.DD` tags on
|
||||
Docker Hub are openDesk-specific builds, not newer CryptPad releases — e.g. `opendesk-2026.08.04`
|
||||
(built 2026-08-04) still contains CryptPad `2026.5.1` in `/cryptpad/package.json`.
|
||||
- CryptPad's GitHub releases and Docker Hub tags are not simultaneous: `2026.10.0` was released
|
||||
2026-10-21; `version-2026.10.0` appeared on Docker Hub on 2026-10-23 (first seen by the weekly
|
||||
upgrade run on 2026-10-30). Re-check the Docker Hub tags page before treating a GitHub release as
|
||||
available for the recipe.
|
||||
- The 2026.10.0 release is a breaking one for the recipe: full server rewrite (separate
|
||||
`cryptpad-server` package), split `config.js` → `config/infra.js` (public origin/ports move),
|
||||
one proxy port for HTTP + websocket (no separate `3003` websocket port), SSO plugin v1.0.0
|
||||
required, and Euro-Office 9.3 replacing the OnlyOffice editors. Recipe handling in the
|
||||
`upgrade-2026.10.0` PR: `CPAD_MAIN_DOMAIN`/`CPAD_SANDBOX_DOMAIN` must be full `https://…` URLs
|
||||
(the new server assigns them verbatim to `infra.public.origin`/`sandboxOrigin`; the image's
|
||||
entrypoint auto-creates `config/infra.js`), nginx `/cryptpad_websocket` proxies to `:3000`,
|
||||
`abra.sh` config/nginx/SSO-entrypoint version labels bumped, SSO default 1.0.0.
|
||||
- nginx is HTTP/1.1-only here (sidecar on :8083, `proxy_http_version 1.1`), so HTTP/2/3 CVE changes in
|
||||
nginx releases generally don't affect this recipe — but still note them.
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
|
||||
## Standing notes
|
||||
- nginx even-numbered minor versions (1.28.x, 1.30.x) are mainline; odd (1.27.x, 1.29.x) are stable. The recipe tracks mainline.
|
||||
- nginx 1.31.7 (14 Oct 2026) is a security patch release with FIVE security fixes: CVE-2026-15176 (buffer overflow in a worker using `alias` when the URI was changed by `rewrite`), CVE-2026-93479 (buffer overflow evaluating string expressions with variables — affects document-root calculation and request logging in http/stream), CVE-2026-95101 (HTTP/3: part of an unknown frame treated as request body — bypasses `client_max_body_size` / injects into proxied request), CVE-2026-97629 (TLS session-ticket key rotation left one key uninitialized — session forgery), CVE-2026-97205 (use-after-free with async file I/O when an SSI page has `include virtual`). Features: geoip MMDB support, backend SSL cert verification by IP, QUIC reconfiguration + binary upgrade with `quic_bpf`. Change: `ngx_http_v2_module` now uses RFC 9218 prioritisation (RFC 7540 priority signals ignored). Bugfixes (X-Accel-Redirect cached response segfault, dav COPY/MOVE with alias segfault, QUIC flood false positives, SSL close_notify handling). NONE affect the custom-html static-file-serving use case (no HTTP/3/QUIC, no AIO/SSI, no `rewrite`+`alias` path mapping, no session-ticket rotation config) — but the 5 CVEs are why this bump is warranted. No breaking config changes; no operator action beyond the image pull.
|
||||
- compose.git-pull.yml is an optional overlay for git-pull functionality; alpine/git version lives there.
|
||||
- compose.sftp.yml and compose.sso.yml are other optional overlays; linuxserver/openssh-server uses `latest` tag and is not version-pinned.
|
||||
- nginx 1.31.6 (15 Sep 2026) is a security patch release: CVE-2026-90439 (heap buffer overflow in a worker under certain configurations when using HTTP/3 with OpenSSL 3.5.0 and earlier). Change: QUIC transport parameters extension received in an SSL connection is now always ignored. Bugfixes: binary upgrade with control API socket + perl module; predicate-location evaluation error ignored; nested location lookup with regex/predicate locations; segfault reading config with `geo` … `ranges` + corrupted binary base file. None affect the custom-html static-file-serving use case (no HTTP/3/QUIC, no predicate locations, no perl, no `geo`).
|
||||
|
||||
@@ -41,3 +41,7 @@ in this file, so keep vendor security pages listed here.
|
||||
- **1.26.3**: carries a regression (#38177 "context deadline exceeded" opening repo code pages) — upstream says upgrade straight to 1.26.4, skip 1.26.3.
|
||||
- **1.27.0** (released 2026-07-13, MINOR with BREAKING changes): `Feat(actions)!: improve support for reusable workflows` (#37478) and `Use Content-Security-Policy: script nonce` (#37232, may break custom inline JS / reverse-proxy CSP). Many security + feature additions. A dedicated minor-bump run should evaluate CSP / reusable-workflow impact before adopting.
|
||||
- **1.27.1** (released 2026-07-27, PATCH): security fix (oauth2 mandatory 2FA enforcement on authorize/grant endpoints, #38606), API swagger alignment, diff-contrast enhancement, and many bugfixes (actions reusable-workflow/job-stranding, OIDC end-session, repo-deletion cleanup, webhook/mail). No breaking changes; safe patch bump from 1.27.0. Release notes: https://github.com/go-gitea/gitea/releases/tag/v1.27.1
|
||||
- **28.0.0** (released 2026-09-29, NEW MAJOR, BREAKING): git network operations now route through an internal proxy and egress settings changed (`Fix(git)!`, #39426) — outbound Git may need `[egress]` allow-listing; Actions adds `RUN_RETENTION_DAYS` (#38855). Also a large SECURITY batch (SSH key fingerprint identification, rejected invalid/duplicate Git objects, repo-scoped team authz, x/crypto DoS). GitHub notes: https://github.com/go-gitea/gitea/releases/tag/v28.0.0 — vendor post: https://blog.gitea.com/release-of-28.0.0/
|
||||
- **28.1.0** (released 2026-10-06, MINOR): 9 security fixes (push-mirror permission/`DISABLE_NEW_PUSH`, local-path import authz, repository media stored XSS, OAuth2 refresh-token kind, issue-attachment authz, `FORCE_PRIVATE` push-to-create, private RSS/Atom feed leak, tag-delete deleting releases, migration page-size DoS), plus MySQL/MariaDB `READ COMMITTED` and go-git repack fixes. No breaking changes. GitHub notes: https://github.com/go-gitea/gitea/releases/tag/v28.1.0 — vendor post (CVE list): https://blog.gitea.com/release-of-28.1.0/
|
||||
- **28.2.0** (released 2026-10-18, MINOR): 9 newly-fixed Gitea CVEs (SSH cert principal confusion, OpenID linking `state`, locked-issue reactions authz, release-attachment metadata leak, `/{username}/-/repositories` panic, draft-release attachment disclosure, Actions artifact v4 body limit, CSRF on repo topics, cross-org label deletion) plus dependency CVE-2026-78659 (`golang.org/x/net` v0.60.0). Three more CVEs are disclosed here but were already fixed in ≤28.1 (CVE-2026-110324, CVE-2026-109844, CVE-2026-97859). No breaking changes and no Postgres requirement. GitHub notes: https://github.com/go-gitea/gitea/releases/tag/v28.2.0 — vendor post (CVE list): https://blog.gitea.com/release-of-28.2.0/
|
||||
- **app.ini writability**: Gitea 1.24+ writes runtime config (e.g. `WORK_PATH`) back to `app.ini`; mounting it directly from a read-only swarm config makes that write fail (non-fatal but warned). The unmerged mirror PR #4 (`fix-app-ini-writable`) seeds a writable copy in the `config` volume. Image bumps must not change the config mount / `docker-setup.sh` handling on their own.
|
||||
|
||||
@@ -95,8 +95,45 @@
|
||||
(v3.2.2→v3.2.4, app+ML only, fresh PR). Direct registry check confirmed
|
||||
`immich-server/ml:v3.2.4` exist; `abra recipe upgrade` still FATAs on the tag+digest DB pin.
|
||||
Valkey CVE note: CVE-2026-56684 / CVE-2026-63639 (UAF→RCE) are fixed in 9.1.1 and are therefore
|
||||
**already closed** by the recipe's pinned 9.1.1 — not by this app bump; ignore the live-`9`-tag
|
||||
** already closed** by the recipe's pinned 9.1.1 — not by this app bump; ignore the live-`9`-tag
|
||||
drift. CVE-2026-25243 (valkey RESTORE zipmap) is vendor-page-only and undecided.
|
||||
- **immich-server v3.3.0 (2026-10-07) + v3.3.1 (2026-10-08, latest stable) pins** (from
|
||||
`docker/docker-compose.yml` @ the v3.3.1 tag): `postgres:14-vectorchord0.4.3-pgvectors0.2.0@sha256:bcf63357…`
|
||||
(UNCHANGED vs v3.2.4) and `valkey:9@sha256:c123e3715db63d06d4ad6964884037aa0d5d4d703939b9929954112889708e1d`
|
||||
— the **valkey pin moved** 9.1.1 (`70739f85…`) → **9.1.2 (`c123e371…`)**, which is now immich's own
|
||||
official tested pin (the live `9` tag has since drifted to `418652cf…`; keep immich's `c123e371…`
|
||||
per the 2026-07-17 precedent). `example.env` @ v3.3.1 is **byte-identical** to v3.2.4. v3.3.0 is a
|
||||
feature release (people sharing / cluster groups, birthday memories, auto-stacked mobile edits,
|
||||
higher-quality thumbnails via linear-light resampling, OAuth claims synced on every login, PP-OCRv6
|
||||
OCR models); v3.3.1 is a patch (people-sharing + mobile scroll fixes). **No breaking server-side
|
||||
changes, no required config, no operator action.** New TypeORM migrations auto-run
|
||||
(`ConvertUserOAuthIdEmptyStringToNull`, `RenameGeoNamesCountries`, `PersonSharing`,
|
||||
`AddPersonUserTableSharedBySharedWithConstraint`) — all succeeded in a live v3.2.4→v3.3.1 deploy.
|
||||
Opt-in only: `MACHINE_LEARNING_MODEL_REVISION=v2` (optimized models, numerically equivalent, no job
|
||||
re-run) and the people-sharing feature (user-triggered button, not a deploy migration). Done in the
|
||||
2026-10-09 upgrade (v3.2.4→v3.3.1, app+ML+valkey, extending PR #6; fresh `!testme` GREEN — build #45).
|
||||
- **immich-server v3.3.3 (2026-10-14) + v3.3.4 (2026-10-20) + v3.3.5 (2026-10-21, latest stable) pins** (from
|
||||
`docker/docker-compose.yml` @ the v3.3.5 tag): `postgres:14-vectorchord0.4.3-pgvectors0.2.0@sha256:bcf63357…`
|
||||
(UNCHANGED since v3.3.0) and `valkey:9@sha256:c123e371…` (immich's tested 9.1.2, also unchanged). Upstream
|
||||
coopcloud published v3.3.4 as `1.12.0+v3.3.4` (`6b294fce`) and moved the recipe's `redis` pin to the then-live
|
||||
`418652cf…`; that divergence is left as upstream has it (no reason to churn a sidecar in an app bump).
|
||||
`docker/docker-compose.yml` and `docker/example.env` are byte-identical between v3.3.4 and v3.3.5.
|
||||
v3.3.3 = bug fixes (mobile memories date, people-query perf); v3.3.4 = further people-query perf + translation;
|
||||
**v3.3.5 = security patch** (certifi CA-trust fix + October 2026 Node.js security patches). No schema
|
||||
migration, no config/env change, no breaking server-side change. Done in the 2026-10-23 upgrade
|
||||
(v3.3.4→v3.3.5, app+ML only, extending PR #6; live v3.3.4→v3.3.5 dev deploy healthy — migrations finished,
|
||||
`vector`/`vchord` present, no pgvectors drop; `!testme` GREEN first run — build #78).
|
||||
- **2026-10-30: postgres sidecar pgvectors 0.2.0 → 0.3.0 (same PG14 + VectorChord 0.4.3 scheme).** With
|
||||
app/ML already at v3.3.5, the only newer same-scheme tag is
|
||||
`ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.3.0` (multi-arch OCI index digest
|
||||
`sha256:87c050465fb969a68c7ac23e375e21f4c95cfacd0edce5fa1bc31e63b7891891`; re-verified via ghcr +
|
||||
`docker buildx imagetools inspect`). immich v3.3.5's own compose still names 0.2.0, but immich only
|
||||
supports `vchord`/`vector` now — `vectors` (pgvecto.rs) is not in its `VECTOR_EXTENSIONS`, so the
|
||||
sidecar library is unused; 0.3.0 is backward-compatible (`ALTER EXTENSION vectors UPDATE`), immich
|
||||
added 0.3.0 support in immich-app/immich#11624, and this recipe previously shipped 0.3.0 with
|
||||
v3.0.1/v2.7.5. Direct dev deploy: 0.2.0-created volume reused by the 0.3.0 task, PG14 clean start,
|
||||
`vchord 0.4.3`/`vector 0.8.1` installed, `/api/server/version` = 3.3.5, `!testme` GREEN (build #105).
|
||||
Recipe PR extends upstream main `ac03d23`; publish with `abra recipe release immich -z`.
|
||||
- **2026-08-07 INFRA note: cc-ci runner's gitea clone-token is STALE (HTTP 401).** `!testme` build
|
||||
#1210 died at the recipe `git clone` step (~5s, before any deploy) with `could not read Username for
|
||||
'https://git.autonomic.zone'` — the runner's mounted gitea token `13e299f2…` is rejected (verified
|
||||
|
||||
@@ -6,6 +6,9 @@
|
||||
| backend | lasuite/impress-backend | https://github.com/suitenumerique/docs | https://github.com/suitenumerique/docs/releases |
|
||||
| celery | lasuite/impress-backend | https://github.com/suitenumerique/docs | https://github.com/suitenumerique/docs/releases |
|
||||
| y-provider | lasuite/impress-y-provider | https://github.com/suitenumerique/docs | https://github.com/suitenumerique/docs/releases |
|
||||
| yhub | lasuite/impress-yhub | https://github.com/suitenumerique/docs | https://github.com/suitenumerique/docs/releases |
|
||||
| yhub-valkey | valkey/valkey | https://github.com/valkey-io/valkey | https://github.com/valkey-io/valkey/releases |
|
||||
| jwt-keys | lasuite/impress-yhub (Node crypto sidecar) | https://github.com/suitenumerique/docs | https://github.com/suitenumerique/docs/blob/main/UPGRADE.md |
|
||||
| docspec | ghcr.io/docspec/api | https://github.com/docspec/docspec | https://github.com/docspec/docspec/releases |
|
||||
| db | pgautoupgrade/pgautoupgrade | https://github.com/pgautoupgrade/docker-pgautoupgrade | https://github.com/pgautoupgrade/docker-pgautoupgrade/releases |
|
||||
| redis | redis | https://github.com/redis/redis | https://hub.docker.com/_/redis/tags |
|
||||
@@ -13,8 +16,20 @@
|
||||
| web | nginx | https://github.com/nginx/nginx | https://nginx.org/en/CHANGES |
|
||||
|
||||
## Standing notes
|
||||
- impress-frontend, impress-backend (used for both backend + celery services), impress-y-provider all
|
||||
share the same version tag from the suitenumerique/docs monorepo. Upgrade all four in lockstep.
|
||||
- **v6.0.x replaces the collaboration server**: hocuspocus/y-provider collaboration is gone; a new
|
||||
`yhub` service (port 3002) serves `/collaboration/*`, with its own `yhub` PostgreSQL database and a
|
||||
Valkey store. `y-provider` is conversion-only. New RSA keys (`JWT_PRIVATE_KEY_FILE` backend,
|
||||
`YHUB_JWT_PRIVATE_KEY_FILE` yhub), `COLLABORATION_SERVER_SECRET` removed. Docs must be migrated into
|
||||
yhub (`SOFT_MIGRATION=true` + `backend migrate_documents`). See UPGRADE.md `[6.0.0]`.
|
||||
- **v6.0.2** moves yhub to `@y/hub 0.10.1`, adding the `yhub_ydoc_versions_v1` table/indexes: re-run the
|
||||
yhub schema script (`init-db`) — the recipe does it on every yhub start.
|
||||
- **The recipe generates the two RSA keys at runtime** into a `jwt_keys` volume (`jwt-keys` sidecar)
|
||||
because abra cannot generate PEM secrets (`generate=false`) and Swarm secrets must pre-exist.
|
||||
- **docspec**: since Docs v5.7.0 the Rust `ghcr.io/docspec/api` (port 3000) replaces the archived Elixir
|
||||
`ghcr.io/docspecio/api` (port 4000); Docs v6.0.2 expects `DOCSPEC_API_URL=http://docspec:3000/conversion`.
|
||||
- impress-frontend, impress-backend (used for both backend + celery services), impress-y-provider,
|
||||
impress-yhub (and the jwt-keys sidecar image) all share the same version tag from the
|
||||
suitenumerique/docs monorepo. Upgrade all of them in lockstep.
|
||||
- AUTO_MIGRATIONS=true means DB migrations run automatically on backend startup. No manual step needed.
|
||||
- Minio tag uses a date-based RELEASE.YYYY-MM-DDTHH-MM-SSZ format — abra cannot parse it for upgrades;
|
||||
check manually on https://github.com/minio/minio/releases.
|
||||
|
||||
@@ -302,3 +302,87 @@
|
||||
`Array.merge`→`mergeIntoObject`; 2.37.0 JSON content-type on decorator body routes + binary-data
|
||||
endpoint adapt; 2.39.0 workflow-version endpoint deprecation; 2.40.x `POST /rest/login`
|
||||
`emailOrLdapLoginId` rename). Recommended release: `-y`.
|
||||
- 2.42.3 (2026-10-05, patch): API fix (let global members grant variable scopes to API keys) + core
|
||||
fix (keep the task runner running on an unhandled promise rejection) + editor feature (Gateway
|
||||
credits promotions on community nodes).
|
||||
- 2.42.4 (2026-10-07, patch): release body empty (republish / no listed changes).
|
||||
- 2.42.5 (2026-10-08, patch): 1 editor fix (show the Agent artifact while the builder creates it).
|
||||
- 2.42.6 (2026-10-09, patch): core feature (block create/update of workflows with deprecated nodes) +
|
||||
Gemini node feature (Nano Banana 2.1 + 2 Lite in the image-model list). Backport of the 2.43.3 pair
|
||||
onto the stable 2.42 line.
|
||||
- 2.42.7 (2026-10-12, patch): 2 core fixes (concurrent credential tests no longer share one node-type
|
||||
registry; resume tool calls for deferred tools a skill loaded).
|
||||
- 2.43.0 (2026-10-06, Pre-release; the 2.43 feature minor): a large feature/bugfix release. Features:
|
||||
Agent Builder / Instance AI / editor improvements + ~many core/API/editor fixes (valid schemas for
|
||||
untyped values in the Public API spec; global members can grant variable scopes to API keys; return
|
||||
an execution when its stored trace context is incomplete; Azure OpenAI Chat Model; etc.). **No
|
||||
breaking compose/env/migration changes; no `N8N_*` env renames.** (2.43.1/2.43.2 = patch bugfixes:
|
||||
editor Assistant/Execute UI; Kimi K3 + OpenAI-compatible Assistant providers self-hosted; Agent
|
||||
artifact preview; agent-builder prompt-caching/background-sandbox perf.)
|
||||
- 2.43.3 (2026-10-09, Pre-release): core feature (block create/update of workflows with deprecated
|
||||
nodes) + Gemini node feature (Nano Banana 2.1 + 2 Lite in the image-model list).
|
||||
- 2.43.4 (2026-10-12, patch, **stable line**): 2 core fixes (handle each Chat Hub segment once in
|
||||
queue mode; resume tool calls for deferred tools a skill loaded).
|
||||
- 2.43.5 (2026-10-14, patch, stable line): core fix (validate the `size` arg of the `chunk()`
|
||||
expression extension) + editor feature (show Gemini models by name / prioritised first).
|
||||
- 2.43.6 (2026-10-15, patch, **stable line — `stable` floating Docker tag points here**, digest
|
||||
`sha256:dd580fe1…`): 1 core fix (create an XML parser per webhook request body). Backport of the
|
||||
2.44.2 fix onto the stable 2.43 line.
|
||||
- 2.44.0 (2026-10-13, **Pre-release / beta line — `beta` floating Docker tag points to 2.44.2**): the
|
||||
2.44 feature minor. Large core/editor change set (credential/node-type policy enforcement across
|
||||
agent tools, credential tests/probes, package import, folder/credential/delete-transfer moves;
|
||||
block the assistant when AI credits hit zero; per-process crash-journal paths; configurable
|
||||
healthcheck timeout; Microsoft SharePoint Trigger node; many bugfixes). **No breaking
|
||||
compose/env/volume/label changes; no `N8N_*` renames.**
|
||||
- 2.44.1 (2026-10-14, Pre-release/beta): 1 editor feature (Gemini models by name / prioritised first).
|
||||
- 2.44.2 (2026-10-15, Pre-release/beta; **newest numeric tag abra lists**; Docker Hub `beta` floating
|
||||
tag = `sha256:b57e7a9c…`): 1 core fix (create an XML parser per webhook request body).
|
||||
- 2026-10-09 run (not separately logged here): PR #9 (branch `upgrade-a144baf`) extended from the
|
||||
2.42.2 of the 2026-10-02 run to **2.43.2** (newest numeric tag abra listed that day). 2.43.2 was
|
||||
still Pre-release on GitHub at the time (tracking-the-newest-tag precedent).
|
||||
- 2026-10-16 run: reconcile confirmed coopcloud upstream main still `9141e23` (3.5.1+2.40.5) and
|
||||
PR #9 (branch `upgrade-a144baf`, at 2.43.2) still open/unmerged, so per the one-evolving-PR
|
||||
contract the run extends PR #9 **2.43.2 → 2.43.6**. Operator guidance this run: verify whether
|
||||
2.44.x is the newest STABLE line and take the newest SAFE patch. **2.44.x is the beta/pre-release
|
||||
line** — GitHub marks 2.44.0/2.44.1/2.44.2 `prerelease=true`, and Docker Hub's `beta` floating tag
|
||||
points at 2.44.2 while `stable` points at 2.43.6 (verified live). So the safe target is **2.43.6**
|
||||
(newest patch on the now-stable 2.43 line), NOT the beta 2.44.2. 2.43.2→2.43.6 is patch-only
|
||||
(2.43.3 deprecated-node workflow guard + Gemini image list; 2.43.4 Chat-Hub queue-mode + deferred
|
||||
tools; 2.43.5 `chunk()` size-arg validation + Gemini picker; 2.43.6 per-request XML parser). No
|
||||
breaking compose/env/migration changes. Live-deployed on cc-ci `dev-n8n` (2.40.5 → 2.43.6): TypeORM
|
||||
migrations completed cleanly, editor `/rest/login` 401 (auth-gated, as expected), `/healthz` 200 —
|
||||
converged. Advisory scan 2.43.2→2.43.6: 0 CVEs fixed (9 sources, none failed). Recommended
|
||||
release: `-y` (relative to the published label app tag 2.40.5 the delta spans feature minors
|
||||
2.41/2.42/2.43).
|
||||
- 2.43.7 (2026-10-16, patch): core suspend in-flight production executions at worker shutdown.
|
||||
- 2.43.8 (2026-10-19, patch): core turn off expression spans by default (backport of 2.44.4).
|
||||
- 2.44.3 (2026-10-16, patch): core keep Observer detail lines instead of discarding the response.
|
||||
- 2.44.4 (2026-10-19, patch): core reuse connections in the log-streaming webhook destination; turn
|
||||
off expression spans by default.
|
||||
- 2.44.5 (2026-10-21, patch): core key queue job outcomes by job on shutdown resume; editor load
|
||||
credential types on startup.
|
||||
- 2.44.6 (2026-10-22, patch; **`stable` floating tag points here**): Postgres Node prevent global
|
||||
pgtypes overwrite.
|
||||
- 2.45.0 (2026-10-20, Pre-release; the 2.45 feature minor): automated migrations for removed nodes;
|
||||
node and credential type policies enabled by default; durable scheduler enabled by default;
|
||||
Idempotency-Key on Public API POSTs; workflow migration-findings endpoints/notifications; Kafka v2
|
||||
default for new nodes; Monday.com V2 default; AI Agent/Assistant, editor and node fixes; UI rebrand.
|
||||
**No breaking compose/env/volume/label changes; no `N8N_*` renames.**
|
||||
- 2.45.1 (2026-10-21, Pre-release): core claim an idle worker's next queue job and its lock atomically;
|
||||
Microsoft Teams Trigger Graph subscription renewal via lifecycle notifications; editors can triage
|
||||
migration findings.
|
||||
- 2.45.2 (2026-10-22, Pre-release; **`beta` floating tag points here — newest numeric tag**): core read
|
||||
type-availability policies from memory at enforcement points; Postgres Node prevent global pgtypes
|
||||
overwrite; migration report CSV export; OAuth-confirmation/email logo update.
|
||||
- 2026-10-23 run: reconcile confirmed coopcloud upstream main moved to `c834df2` ("NODE_FUNCTION_ALLOW_EXTERNAL
|
||||
in env file", on top of 3.5.1+2.40.5) and PR #9 (branch `upgrade-a144baf`, at 2.43.6) still
|
||||
open/unmerged. Per the one-evolving-PR contract, PR #9 extended **2.40.5 → 2.45.2** (explicit target
|
||||
= highest tag abra lists). 2.45.x is the current Pre-release/beta line; **stable = 2.44.6** (verified
|
||||
live via Docker Hub floating tags), flagged to the operator. Window spans feature minors 2.41–2.45,
|
||||
but release notes show **no breaking compose/env/volume/label changes and no `N8N_*` renames**.
|
||||
Live-deployed on cc-ci `dev-n8n`: TypeORM migrations completed cleanly through
|
||||
`CreatePromotionRunAndReviewTables1792139680417`, editor accessible, `/healthz` 200, `/rest/login`
|
||||
401; runtime deprecation warnings only (`WEBHOOK_URL`→`N8N_WEBHOOK_URL`, internal runner mode,
|
||||
unverified-packages default, runner-timeout/compression-limit default reductions). Advisory scan
|
||||
2.40.5→2.45.2: 0 CVEs identified, BUT the GitHub advisory sources failed (unauthenticated rate
|
||||
limit) — scan incomplete, not to be read as clean. Recommended release: `-y`.
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
# Upstream sources — rustdesk-server
|
||||
|
||||
| service | image | source repo | releases / changelog |
|
||||
|---------|-------|-------------|----------------------|
|
||||
| app | rustdesk/rustdesk-server | https://github.com/rustdesk/rustdesk-server | https://github.com/rustdesk/rustdesk-server/releases |
|
||||
| signal | rustdesk/rustdesk-server | https://github.com/rustdesk/rustdesk-server | https://github.com/rustdesk/rustdesk-server/releases |
|
||||
|
||||
- nvd-cpe: rustdesk-server = cpe:2.3:a:rustdesk:rustdesk_server:*:*:*:*:oss:*:*:*
|
||||
|
||||
## Standing notes
|
||||
- Enrolled in the cc-ci weekly set. The NVD CPE above is scoped to the open source server edition;
|
||||
do not count RustDesk Client or Server Pro CVEs for this recipe.
|
||||
- Both services (`app` = hbbs, `signal` = hbbr) use the SAME image — always bump them together.
|
||||
- Docker Hub has stray `1.4.5-*` arch tags (pushed 2026-01-12, top-level `1.4.5` tag deleted, no
|
||||
GitHub release) — an aborted/alias push, NOT a real version line. The real line is `1.1.x`;
|
||||
GitHub "Latest" + docker `latest` agree (1.1.16 as of 2026-07-20). Ignore 1.4.5.
|
||||
- Recipe uses `-k _` on both hbbs and hbbr (enforces key verification, enforced-key mode since
|
||||
1.1.11) and publishes host-mode ports 21115-21119 (hbbs) + 21117/21119 (hbbr) — direct
|
||||
client connections, only 21118/21119 are the web-socket/http endpoints behind nothing; the
|
||||
traefik labels on `app` are vestigial (hbbs has no HTTP :80 service; healthcheck commented out).
|
||||
- 1.1.15 raised **default bandwidth limits** and added auth-failure logging (fail2ban-friendly);
|
||||
1.1.16 fixed **unauthenticated UDP punch-hole reflection/amplification abuse** + critical `mio`
|
||||
0.8.5→0.8.11 issue + i32 overflow (peers offline 24.9–49.7d shown online).
|
||||
@@ -28,10 +28,11 @@ lasuite-drive weekly
|
||||
lasuite-meet weekly
|
||||
mailu weekly
|
||||
matrix-synapse weekly
|
||||
mattermost-lts weekly
|
||||
mattermost-lts external # maintained elsewhere — used/tested by cc-ci but NOT weekly-upgraded
|
||||
mumble weekly
|
||||
n8n weekly
|
||||
plausible external # maintained elsewhere — used/tested by cc-ci but NOT weekly-upgraded
|
||||
rustdesk-server weekly
|
||||
uptime-kuma external # maintained elsewhere — used/tested by cc-ci but NOT weekly-upgraded
|
||||
wordpress weekly
|
||||
```
|
||||
|
||||
Generated
+4
-4
@@ -10,11 +10,11 @@
|
||||
]
|
||||
},
|
||||
"locked": {
|
||||
"lastModified": 1790010051,
|
||||
"narHash": "sha256-0dz8fg4zqg54tk+1A7CHZAw+lK9vWxre6Tioi5B0ocU=",
|
||||
"lastModified": 1793644559,
|
||||
"narHash": "sha256-u6xPHneKwIrd1taLVs1X12dUs46RzTzkZ9fJ8d/ZvYI=",
|
||||
"ref": "refs/heads/main",
|
||||
"rev": "2f6787f0433e7ff3053116039f10c00cde34128f",
|
||||
"revCount": 1558,
|
||||
"rev": "d569e6528f2eba4309db4794b091d6a1ecc029bf",
|
||||
"revCount": 1589,
|
||||
"type": "git",
|
||||
"url": "https://git.autonomic.zone/recipe-maintainers/cc-ci.git"
|
||||
},
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Memory index
|
||||
|
||||
- [Enroll gate fixtures must fail loudly](enroll-gate-fixtures-fail-loudly.md) — version-gate reads via lifecycle.deployed_identity, never None-into-if; post-host-merge deploy = --cc-ci-only after merge (staging obsolete)
|
||||
|
||||
- [Orchestrator host: Hetzner](orchestrator-host-hetzner.md) — runs on Hetzner cpx22; rebuild cmd, loops-service bounce, git-identity gotcha
|
||||
- [Push commits to remote](push-commits-to-remote.md) — push to git.autonomic.zone right after every commit in this repo
|
||||
- [Regression canary cadence](regression-canary-cadence.md) — server E2E canaries run on polish/review/release, not every commit
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
---
|
||||
tags: [cc-ci, enrollment, testing, deploy]
|
||||
---
|
||||
|
||||
# Enroll-verify gate fixtures must fail loudly
|
||||
|
||||
## Context
|
||||
|
||||
During rustdesk-server enrollment (2026-10-30, cc-ci PR #48) the first full-suite verification
|
||||
run came back RED only because of the *fixture*, not the app: the punch-hole regression test's
|
||||
version gate read the deployed image with a hand-rolled `docker service inspect`, got None, and
|
||||
the `if version is not None and version < (1,1,16)` gate silently decided to RUN on a 1.1.12 head
|
||||
where pre-fix answering is the expected behavior. An expected-RED looked like a real bug.
|
||||
|
||||
## Lesson
|
||||
|
||||
- A test gate that cannot decide its condition must `pytest.fail` LOUDLY with the raw value —
|
||||
never default to None-into-an-if (silently runs → wrong RED) or silent skip (vacuous green).
|
||||
- cc-ci already has a first-class image reader: `runner/harness/lifecycle.py::deployed_identity(domain)`
|
||||
→ `{"version","image","chaos"}`; parse the tag from `.image` and strip any `@sha256:` digest.
|
||||
Hand-rolled service inspects in recipe_meta/custom tests are a duplication hazard.
|
||||
- Version-gated custom tests skip on pre-fix heads by design (STYLE §2) — that is fine as long
|
||||
as the version read itself cannot fail silently.
|
||||
|
||||
# cc-ci host deploy mechanism (post host-merge, 2026-09+)
|
||||
|
||||
To run any merged cc-ci main change on the live host now, use the `--cc-ci-only` path from
|
||||
/cc-ci-orchestrator-update — merge the PR first (enrollment PRs: comment verification evidence,
|
||||
then merge — invocation = authorization), then:
|
||||
|
||||
```
|
||||
ssh cc-ci 'sudo -H -u loops bash -lc "cd /srv/cc-ci-orch && nix flake update cc-ci &&
|
||||
git -c user.name=auto-update -c user.email=auto-update@cc-ci commit -qm \"flake.lock: cc-ci -> main\" -- flake.lock &&
|
||||
git push -q origin HEAD:main"'
|
||||
ssh cc-ci 'git -C /etc/cc-ci pull --ff-only --recurse-submodules'
|
||||
systemd-run --unit=ccci-rebuild-test --collect -E HOME=/root -p WorkingDirectory=/srv/cc-ci-orch \
|
||||
bash -lc "nixos-rebuild test --flake /srv/cc-ci-orch#cc-ci" # := health checks := then switch
|
||||
```
|
||||
|
||||
The old staging procedure (branch clone at /root/cc-ci-deploy, referenced by the retired
|
||||
/cc-ci-server-update skill and old enroll SKILL.md step 7 lines) is obsolete — the combined-host
|
||||
flake takes the cc-ci module from the lockfile, so a BRANCH cannot be staged that way; deploy
|
||||
after merge instead.
|
||||
Reference in New Issue
Block a user