Compare commits

...
Author SHA1 Message Date
autonomic-bot a29b993160 Merge pull request 'config: switch upgrader + report to deepseek, keep supervisor on glm' (#12) from config/deepseek-upgrader-model into main 2026-08-16 02:29:20 +00:00
autonomic-bot a0d6fc9417 config: switch upgrader + report to deepseek, keep supervisor on glm
The weekly /upgrade-all parent session and the /recipe-report session now
run on tinfoil/deepseek-v4-pro (LOOP_MODEL + REPORT_MODEL in upgrader.env).
The hourly supervisor stays on opencode-go/glm-5.2 (SUPERVISOR_MODEL default
in launch-supervisor.py, not overridden). Subagents already bind deepseek via
the cc-ci repo's opencode config (fix from 2026-08-10, verified this week:
all 16 subagents across both waves ran deepseek-v4-pro).

LOOP_TIER=zen is kept so the tier check passes; the watchdog's usage-limit
probe sends the deepseek model name to the zen endpoint, which returns 200
(not 429) → resume immediately — correct, since tinfoil has no rolling usage
limit to wait out. Verified the probe behaviour with a direct curl.

Root cause: the 2026-08-14 run stalled mid-recipe on 'Insufficient balance'
(opencode zen workspace balance exhausted), then sat unfinished for 40h while
the supervisor cron spun hourly unable to recover it. Deepseek (pay-per-use
API key) has no rolling balance limit, so this can't recur.

Also documents the session recovery in JOURNAL.md (the stalled run was
completed via a fresh scoped upgrader — the original 2.58M-token session was
unresumable: the inference endpoint silently drops the oversized request).
2026-08-16 02:28:38 +00:00
autonomic-bot 9409adffb8 upstream(n8n): 2.35.x release notes 2026-08-15 21:04:58 +00:00
autonomic-bot 69d1840ea5 upstream(lasuite-docs): note minio Docker images stopped at 2025-09-07 2026-08-14 03:06:49 +00:00
autonomic-bot ab5d11ea2a Merge pull request 'advisory-scan: NVD by CPE, so mattermost and mumble stop scanning as '?'' (#11) from feat/nvd-cpe-source into main 2026-08-11 22:17:13 +00:00
autonomic-bot 74117c2260 cve-check: record the remedy for a blind recipe, not just the symptom
The skill said to render a sourceless recipe as '?'. It now says how to stop it
being sourceless: declare an NVD CPE in the registry. That is what took the fleet
from two blind recipes to zero, and it is the first thing to try when the sweep
flags another.
2026-08-11 22:16:52 +00:00
autonomic-bot 985dc06e47 advisory-scan: NVD by CPE, so mattermost and mumble stop scanning as '?'
Two recipes could not see CVEs at all. mattermost-lts has an empty GitHub advisory
feed and renders its security bulletins client-side, so a text sweep finds nothing;
mumble publishes nothing anywhere the registry points. Both returned '?' - nothing
measured - which is honest but useless.

NVD is CPE-indexed and carries structured version ranges, so it answers where the
vendor does not. Declared per recipe as 'nvd-cpe: <image> = <cpe:2.3:...>'.

  mattermost-lts 10.5.0  -> 10.12.4   165 CVEs
  mattermost-lts 10.11.22 -> 10.12.4    0 CVEs  (measured, not unknown)
  mumble         1.3.0   -> 1.6.870      2 CVEs

Both NVD range forms are used: versionEndExcluding is a patched version;
versionEndIncluding means the fix version is unpublished but the upgrade delivers
it whenever it crosses X.

That 0 for the actual mattermost upgrade is the interesting one, and it needed a
new rule to be correct: a fix on the line you upgrade FROM was already yours.
mattermost patches every maintained line at once, so 10.11.22 -> 10.12.4 crosses
10.12.1 while 10.11.22 already had the 10.11.4 backport. Without the rule the scan
claimed 12 CVEs the upgrade did not deliver.

The rule is skipped for placeholders: '7.4.X' parses to a bare 7.4 and would read
as 'already fixed at 7.4', which silently dropped redis CVE-2024-46981 and took
discourse 140 -> 139 before I caught it.

79 tests. discourse 140 / gitea 2 / mailu 2 / keycloak 12 / plausible 6 unchanged.
Fleet sweep: 0 recipes with no usable CVE source, down from 2.
2026-08-11 22:16:37 +00:00
autonomic-bot 4b9978ac02 Merge pull request 'audit-sources --security-sources: find the recipes that cannot see CVEs at all' (#10) from feat/audit-security-sources into main 2026-08-11 20:02:24 +00:00
autonomic-bot 46ace30b4d audit-sources --security-sources: find the recipes that cannot see CVEs at all
Follow-up to the nginx blind spot. Sweeping all 22 recipes for sources whose CVEs
are USABLE (structured advisory feed, or a changelog attributable to releases)
rather than merely visible.

Before the changelog-attribution fix: 20 unusable sources. After: 5, and all five
are redundant - the same project also publishes an advisory feed (redis, gitea,
minio, clickhouse), so nothing is actually lost.

One real find, same shape as nginx: ONLYOFFICE/DocumentServer publishes NO GitHub
advisories, and the registry pointed its CHANGELOG.md at the GitHub *blob* page -
636KB of markup in which the release headings do not survive HTML-stripping, so 24
CVEs were visible and NONE attributable. The raw URL attributes all 24. Rather than
fix one registry line, advisory-scan now normalises github.com/../blob/.. to
raw.githubusercontent.com, which fixes every entry present and future.
lasuite-drive bumps documentserver, so this was live.

Genuinely blind after all that: mattermost-lts and mumble - no advisory feed, no
attributable changelog, no CVE data anywhere the registry points. mattermost is the
notable one: its bulletins are client-side rendered, so a regex sweep sees nothing.
Their scans can report 0 while nothing was measured, so /cve-check now renders those
recipes as ? and says why.

The audit output distinguishes a blind RECIPE from an unparseable PAGE, because
conflating them made 5 harmless redundancies look like 5 gaps.
2026-08-11 20:02:06 +00:00
autonomic-bot dab3edf3c2 Merge pull request 'advisory-scan: attribute vendor-changelog CVEs to the release that fixed them' (#9) from feat/changelog-version-attribution into main 2026-08-11 19:50:00 +00:00
11 changed files with 416 additions and 13 deletions
+21
View File
@@ -104,6 +104,27 @@ CRITICAL came from, and an image with no window is not counted at all.
distinction was the difference between two false zeros and the truth (both recipes turned out fine, distinction was the difference between two false zeros and the truth (both recipes turned out fine,
but nothing in the survey said so). but nothing in the survey said so).
### 2c. Know which recipes CANNOT see CVEs at all
```
python3 cc-ci-plan/audit-sources.py --security-sources
```
A recipe whose sources yield **no CVE data at all** cannot produce a meaningful `0` — nothing was
measured, the same way a missing registry file cannot. Render those as **`?`**, not `0`.
**The fleet is currently at zero such recipes.** The last two — `mattermost-lts` (empty advisory
feed, client-side-rendered bulletins) and `mumble` (nothing published anywhere) — were fixed by
declaring an NVD CPE in their registry:
```
- nvd-cpe: mattermost-team-edition = cpe:2.3:a:mattermost:mattermost_server:*:*:*:*:*:*:*:*
```
**If this sweep ever reports a blind recipe again, that is the fix**: find the product's CPE at
nvd.nist.gov and add the line. Prefer a real advisory feed or an attributable changelog when one
exists — NVD lags the vendor — but a lagging source beats no source, and it turns a `?` into a
number.
An *unparseable page* is NOT the same thing: it is harmless when the same project also publishes an
advisory feed (redis, gitea, minio, clickhouse all do). Only "no usable source for this image" counts.
### 3. Run the advisory scan over that window ### 3. Run the advisory scan over that window
``` ```
python3 /srv/cc-ci/cc-ci-plan/advisory-scan.py <recipe> --from <old-app> --to <new-app> \ python3 /srv/cc-ci/cc-ci-plan/advisory-scan.py <recipe> --from <old-app> --to <new-app> \
+62 -3
View File
@@ -867,6 +867,65 @@ session cc-ci-orchestrator-stale can be killed; recipe-mirrors org still private
(/srv/cc-ci-orch/cc-ci), and task-tool subagents inherit their parent session's directory. The (/srv/cc-ci-orch/cc-ci), and task-tool subagents inherit their parent session's directory. The
config now lives in the cc-ci repo at that path. VERIFIED end-to-end with the launcher's exact config now lives in the cc-ci repo at that path. VERIFIED end-to-end with the launcher's exact
invocation: parent=glm-5.2, subagent=deepseek-v4-pro read back from the session DB. invocation: parent=glm-5.2, subagent=deepseek-v4-pro read back from the session DB.
LESSON: `opencode debug config` proves resolution, NOT binding — only a live subagent's recorded LESSON: `opencode debug config` proves resolution, NOT binding — only a live subagent's recorded
modelID proves binding. First attempt was a false pass because the probe passed --dir (unlike the modelID proves binding. First attempt was a false pass because the probe passed --dir (unlike the
real launcher) and landed in a different project. real launcher) and landed in a different project.
## Session 2026-08-15 19:25 UTC — opencode glm-5.2
**Left off:** Recovered the stalled 2026-08-14 weekly /upgrade-all run. Killed a supervisor that had
been relaunching hourly for ~40h (balance exhausted), then started a FRESH scoped upgrader. Run is now
progressing (surveying the 9 remaining recipes). Watching it through to completion.
**What happened (the stall):**
- The 2026-08-14 /upgrade-all run (session ses_00200382fffeYIGl2sc3mO9JId) stalled at 03:18 Aug 14
mid-`lasuite-drive` with `Error: Insufficient balance` (opencode zen workspace balance ran out). It
had already done bluesky-pds, ghost, gitea, hedgedoc (PRs) + immich, lasuite-docs (SKIPPED up-to-date)
alphabetically; lasuite-drive had a plan + partial PR #6 but no RESULT/verify.
- The supervisor cron (glm-5.2, opencode-go tier) relaunched an hourly one-shot supervisor ~40 times
to "drive it to completion", but each was also balance-walled (and later, just spinning). The run sat
INCOMPLETE + not progressing for 40h. No weekly summary, no report published for week of Aug 14.
**What I did this session:**
- Diagnosed: the opencode zen endpoint is NOW healthy (direct probe `say OK` → HTTP 200 in 1.35s —
balance is restored). But resuming the ORIGINAL giant session is impossible: it's 2.58M tokens
(267K input + 2.3M cache) and `opencode run -s … --continue` sits idle on `do_epoll_wait` with zero
I/O — the inference endpoint silently drops the oversized request (matches the supervisor's
`socket connection was closed unexpectedly` errors). A fresh small `opencode run` works fine. So the
giant session is unresumable; a fresh start is the only path.
- Killed the stuck supervisor (tmux `cc-ci-supervisor`, proc 377329).
- `UPGRADER_ARGS="lasuite-drive lasuite-meet mailu matrix-synapse mattermost-lts mumble n8n plausible
wordpress --sequential" python3 /srv/cc-ci/cc-ci-plan/launch-upgrader.py fresh` — this killed the
stuck resume, archived the old giant session (`archive-cc-ci-upgrader — 2026-08-14`), reclaimed 10GB
stale images on cc-ci (disk 29%), and started a FRESH small session
`ses_ff920cf39ffeoogwXHTajp94cr` (zen/glm-5.2) scoped to the 9 recipes not yet done this week
(positions 13-21 alphabetically; positions 1-12 were already surveyed — 6 PRs + 6 up-to-date). A
fresh watchdog is watching the new session. The skill is idempotent (reuses existing PRs incl.
lasuite-drive #6, never duplicates), so scoping is safe.
- Confirmed the fresh run is progressing: pane shows it surveying the 9 recipes (verified all present
in abra + all `weekly` tier; currently probing plausible/wordpress tags). Proc alive, log growing.
**Phase / loop state:**
- Build/adversary loops: STOPPED (whole sequence completed 2026-08-01; phase ghost DONE).
- Weekly upgrader: RUNNING (fresh session ses_ff920cf39, scoped 9 recipes, --sequential, watchdog up).
- cc-ci server: healthy (disk 29%, runner active).
**Open items for next session:**
- **Monitor the fresh upgrader to completion.** It will survey the 9 recipes, /recipe-upgrade the
upgradeable ones (subagents, !testme verify, open/extend PRs — NEVER merge), write the weekly summary
to `/srv/cc-ci/.cc-ci-logs/upgrades/`, then `launch-report.py fresh` (the upgrade-all skill does this
itself per SKILL.md §5), print `UPGRADE RUN COMPLETE`, and go idle. If it stalls on a usage limit,
the watchdog auto-resumes the SAME (small) session — that works now.
- **Do NOT try to resume the archived giant session ses_00200382** — it's unresumable (endpoint drops
the 2.58M-token request). It's archived; leave it.
- After the run completes + report publishes, operator review queue = this week's recipe PRs.
- The supervisor cron (hourly at XX:07) should now leave the run alone once it's progressing; if a
supervisor fires while the run is mid-flight, its guardrails say to hand back to the resumed run, not
double-write. No action needed unless it interferes.
**Notes:**
- Root cause of the 40h silence was the same BUG 1 from 2026-08-10 (supervisor progress gate) partly:
the supervisor kept firing because the run never reached "progressing". Now that balance is restored
and a fresh small session is running, the gate should see progress and stand down.
- Lesson: when a weekly run dies mid-flight on a giant context, do NOT resume the original session —
start fresh and scope to the remaining recipes. The /upgrade-all skill is idempotent so this is safe.
+30 -1
View File
@@ -123,7 +123,27 @@ A changelog CVE is tied to a window by the **image name appearing in the page UR
`nginx.org/...`). A CVE found on a vendor page with no attributable release still has no version data, `nginx.org/...`). A CVE found on a vendor page with no attributable release still has no version data,
so pass 1 cannot place it — it goes to pass 2 (§6). so pass 1 cannot place it — it goes to pass 2 (§6).
### 2c. OSV.dev — supplementary ### 2c. NVD by CPE — the fallback for projects that publish nothing
Declared per recipe in the registry as `nvd-cpe: <image-key> = <cpe:2.3:...>`.
> **Why it exists.** Two recipes could not see CVEs *at all*: `mattermost-lts` (empty GitHub advisory
> feed, security bulletins rendered client-side so a text sweep finds nothing) and `mumble` (nothing
> published anywhere the registry points). Their scans returned `?` — nothing measured. NVD is
> CPE-indexed and carries structured ranges, so it answers where the vendor does not: mattermost
> 10.5.0 → 10.12.4 now scores **165**, and mumble finds `CVE-2025-71264` (fixed 1.6.870).
Two range forms, both used:
| NVD field | meaning | how it is judged |
|---|---|---|
| `versionEndExcluding X` | fixed in X exactly | a normal patched version (§4a) |
| `versionEndIncluding X` | affected **up to and including** X; fix version unpublished | fixed when the upgrade crosses X, i.e. `from ≤ X < to` |
**NVD lags the vendor** — it had neither gitea CVSS-9.8 RCE at publication — so this is a fallback,
never a replacement for 2a/2b. Unauthenticated calls are rate-limited (~5/30s), hence the retry.
### 2d. OSV.dev — supplementary
Only when the recipe has an entry in `OSV_PACKAGES` (ecosystem + package) and a version is given. Only when the recipe has an entry in `OSV_PACKAGES` (ecosystem + package) and a version is given.
@@ -186,6 +206,15 @@ literal compose diff, e.g. "what would the compatibility-safe target fix?").
### 4a. By patched version (preferred — exact) ### 4a. By patched version (preferred — exact)
**A fix on the line you are upgrading FROM was already yours.** Projects that maintain several lines
patch them all at once: mattermost fixed `CVE-2025-11794` in 10.11.4, 10.12.1 *and* 10.5.12. An
upgrade 10.11.22 → 10.12.4 crosses 10.12.1, so a naive window test counts it — but 10.11.22 is
already past 10.11.4, so the deployment had the fix before the upgrade. Counting it credits the
upgrade with work it did not do. This check is **skipped for placeholder versions** (`7.4.X` parses
to a bare `7.4`, which would read as "already fixed at 7.4" and silently drop a real fix — exactly
how redis `CVE-2024-46981` was lost when the rule was first added).
`patched_versions` is a **range expression** (`">= 2.18.1"`), possibly several joined by `;`. Extract `patched_versions` is a **range expression** (`">= 2.18.1"`), possibly several joined by `;`. Extract
every version-looking token; the advisory is **fixed-by-this-upgrade** if **any** patched version `p` every version-looking token; the advisory is **fixed-by-this-upgrade** if **any** patched version `p`
satisfies `from < p <= to` — exclusive lower (a fix already in the version you were on is not this satisfies `from < p <= to` — exclusive lower (a fix already in the version you were on is not this
+153 -3
View File
@@ -44,7 +44,9 @@ import json
import os import os
import re import re
import sys import sys
import time
import urllib.error import urllib.error
import urllib.parse
import urllib.request import urllib.request
REGISTRY_DIR = os.environ.get("CCCI_UPSTREAM_REGISTRY", "/srv/cc-ci/cc-ci-plan/upstream") REGISTRY_DIR = os.environ.get("CCCI_UPSTREAM_REGISTRY", "/srv/cc-ci/cc-ci-plan/upstream")
@@ -164,6 +166,27 @@ def _vkey(v: str | None) -> tuple:
return tuple(out) return tuple(out)
def _already_fixed_on_from_line(kf: tuple, cands: list[tuple]) -> bool:
"""Was it ALREADY fixed on the line we are upgrading FROM?
The mirror image of _superseded_on_target_line, and just as necessary. mattermost fixes each CVE
across several maintained lines at once — CVE-2025-11794 is patched in 10.11.4, 10.12.1 and
10.5.12. Upgrading 10.11.22 -> 10.12.4 crosses 10.12.1, so a naive window test counts it; but
10.11.22 is already past 10.11.4, so the deployment HAD the fix before the upgrade. Counting it
credits the upgrade with work it did not do."""
if len(kf) < 2:
return False
line = kf[:2]
for c in cands:
if len(c) < 2 or c[:2] != line:
continue
n = max(len(kf), len(c))
pad = lambda z: z + (0,) * (n - len(z))
if pad(c) <= pad(kf):
return True
return False
def _superseded_on_target_line(kt: tuple, cands: list[tuple]) -> bool: def _superseded_on_target_line(kt: tuple, cands: list[tuple]) -> bool:
"""Does a patched version on the TARGET's own release line sit ABOVE the target? """Does a patched version on the TARGET's own release line sit ABOVE the target?
@@ -303,6 +326,20 @@ def _changelog_versions(text: str) -> dict:
return out return out
_BLOB_RE = re.compile(r"^https://github\.com/([^/]+)/([^/]+)/blob/(.+)$")
def _raw_if_blob(url: str) -> str:
"""A GitHub *blob* URL is an HTML viewer, not the file.
The registry pointed ONLYOFFICE's CHANGELOG.md at its blob page. Fetching that returns 636KB of
markup in which the release headings do not survive HTML-stripping, so 24 CVEs were visible and
NONE attributable to a release — the same shape of blind spot as nginx. The raw URL attributes
all 24. Normalising here fixes every registry entry at once, present and future."""
m = _BLOB_RE.match(url)
return f"https://raw.githubusercontent.com/{m.group(1)}/{m.group(2)}/{m.group(3)}" if m else url
def vendor_pages(urls: list[str]) -> list[dict]: def vendor_pages(urls: list[str]) -> list[dict]:
"""Fetch each registry URL and regex out CVE ids, with a little surrounding context.""" """Fetch each registry URL and regex out CVE ids, with a little surrounding context."""
out = [] out = []
@@ -317,7 +354,7 @@ def vendor_pages(urls: list[str]) -> list[dict]:
continue continue
entry = {"source": u, "status": "ok", "cves": [], "context": {}, "fixed_in": {}} entry = {"source": u, "status": "ok", "cves": [], "context": {}, "fixed_in": {}}
try: try:
text = _fetch(u) text = _fetch(_raw_if_blob(u))
plain = re.sub(r"<[^>]+>", " ", text) plain = re.sub(r"<[^>]+>", " ", text)
for cve in sorted(set(CVE_RE.findall(plain))): for cve in sorted(set(CVE_RE.findall(plain))):
entry["cves"].append(cve) entry["cves"].append(cve)
@@ -330,6 +367,78 @@ def vendor_pages(urls: list[str]) -> list[dict]:
return out return out
NVD_API = "https://services.nvd.nist.gov/rest/json/cves/2.0"
NVD_CPE_RE = re.compile(r"^\s*[-*]?\s*nvd-cpe:\s*(\S+)\s*=\s*(cpe:2\.3:[^\s`]+)", re.M | re.I)
def registry_cpes(recipe: str, registry_dir: str) -> list[tuple[str, str]]:
"""[(image-key, cpe)] declared in the recipe's registry as `nvd-cpe: <key> = <cpe>`."""
path = os.path.join(registry_dir, f"{recipe}.md")
try:
return [(m.group(1), m.group(2)) for m in NVD_CPE_RE.finditer(open(path).read())]
except OSError:
return []
def nvd_advisories(cpe: str, key: str) -> dict:
"""CVEs for a CPE from NVD, with the version data the classifier needs.
THE FALLBACK FOR PROJECTS THAT PUBLISH NOTHING MACHINE-READABLE. mattermost's GitHub advisory
feed is empty and its security bulletins are client-side rendered; mumble publishes neither. Both
scanned as `?` — nothing measured — until here. NVD is CPE-indexed and carries structured ranges:
versionEndExcluding X -> fixed in X exactly (a patched version)
versionEndIncluding X -> affected up to and INCLUDING X, fixed in some later release. The
exact fix version is unknown, but the upgrade fixes it whenever it
crosses X — recorded as `affected_max` and judged in the classifier.
NVD LAGS the vendor (it had neither gitea CVSS-9.8 RCE at publication), so this is a fallback,
never a replacement for 2a/2b. Unauthenticated calls are rate-limited to ~5/30s, hence the retry.
"""
entry = {"source": f"nvd:{key}", "status": "ok", "advisories": []}
url = f"{NVD_API}?resultsPerPage=2000&virtualMatchString={urllib.parse.quote(cpe)}"
data = None
for attempt in range(3):
try:
data = json.loads(_fetch(url))
break
except Exception as e: # noqa: BLE001
if attempt == 2:
entry["status"] = f"error: {type(e).__name__}"
return entry
time.sleep(8)
for v in (data or {}).get("vulnerabilities", []):
c = v.get("cve") or {}
cid = c.get("id")
if not cid:
continue
fixed, affected_max = set(), set()
for cfg in c.get("configurations", []):
for node in cfg.get("nodes", []):
for m in node.get("cpeMatch", []):
if m.get("versionEndExcluding"):
fixed.add(m["versionEndExcluding"])
elif m.get("versionEndIncluding"):
affected_max.add(m["versionEndIncluding"])
sev = None
for mk in ("cvssMetricV31", "cvssMetricV30", "cvssMetricV2"):
got = (c.get("metrics") or {}).get(mk) or []
if got:
sev = (got[0].get("cvssData") or {}).get("baseSeverity")
break
entry["advisories"].append({
"cve": cid, "ghsa": None, "severity": (sev or "").lower() or None,
"summary": next((d.get("value") for d in c.get("descriptions", [])
if d.get("lang") == "en"), "")[:200],
"vulnerable_range": None,
"patched": "; ".join(sorted(fixed)) or None,
"affected_max": "; ".join(sorted(affected_max)) or None,
"url": f"https://nvd.nist.gov/vuln/detail/{cid}",
"published_at": c.get("published"), "description": None, "cvss": None,
})
return entry
def osv(recipe: str, version: str | None) -> dict | None: def osv(recipe: str, version: str | None) -> dict | None:
pkg = OSV_PACKAGES.get(recipe) pkg = OSV_PACKAGES.get(recipe)
if not pkg or not version: if not pkg or not version:
@@ -628,7 +737,7 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
"vulnerable_range": None, "patched": None, "vulnerable_range": None, "patched": None,
"context": None, "published_at": None, "context": None, "published_at": None,
"description": None, "url": None, "cvss": None, "description": None, "url": None, "cvss": None,
"changelog_fixed_in": None}) "changelog_fixed_in": None, "affected_max": None})
if src not in e["sources"]: if src not in e["sources"]:
e["sources"].append(src) e["sources"].append(src)
for k, v in extra.items(): for k, v in extra.items():
@@ -645,6 +754,16 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
context=a.get("summary"), published_at=a.get("published_at"), context=a.get("summary"), published_at=a.get("published_at"),
description=a.get("description"), url=a.get("url"), cvss=a.get("cvss")) description=a.get("description"), url=a.get("url"), cvss=a.get("cvss"))
for key, cpe in registry_cpes(recipe, registry_dir):
entry = nvd_advisories(cpe, key)
report["sources"].append({"source": entry["source"], "status": entry["status"],
"found": len(entry.get("advisories") or [])})
for a in entry.get("advisories", []):
record(a["cve"], entry["source"], severity=a.get("severity"),
patched=a.get("patched"), affected_max=a.get("affected_max"),
context=a.get("summary"), published_at=a.get("published_at"),
url=a.get("url"))
for entry in vendor_pages(urls): for entry in vendor_pages(urls):
report["sources"].append({"source": entry["source"], "status": entry["status"], report["sources"].append({"source": entry["source"], "status": entry["status"],
"found": len(entry.get("cves", []))}) "found": len(entry.get("cves", []))})
@@ -685,7 +804,8 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
# #
# A source with no window is not classified: its advisories are listed as unclassified so they # A source with no window is not classified: its advisories are listed as unclassified so they
# stay visible without inflating the count. # stay visible without inflating the count.
gh_sources = [x["source"] for x in report["sources"] if x["source"].startswith("github-advisories:")] gh_sources = [x["source"] for x in report["sources"]
if x["source"].startswith(("github-advisories:", "nvd:"))]
primary = gh_sources[0] if (gh_sources and (v_from or v_to)) else None primary = gh_sources[0] if (gh_sources and (v_from or v_to)) else None
report["primary_source"] = primary report["primary_source"] = primary
@@ -694,6 +814,17 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
if primary: if primary:
windows[primary] = (v_from, v_to) windows[primary] = (v_from, v_to)
window_key[primary] = primary.split("/")[-1] window_key[primary] = primary.split("/")[-1]
# The app's window must also cover its NVD entry. NVD sources are keyed by IMAGE name
# (`mattermost-team-edition`) while the advisory feed is keyed by REPO (`mattermost/
# mattermost`), so without this the fallback source that exists precisely because the feed
# is empty would itself go unwindowed — and mumble/mattermost would still report nothing.
pname = primary.split("/")[-1].lower()
for src in gh_sources:
if src.startswith("nvd:") and src not in windows:
k = src.split(":", 1)[1].lower()
if pname in k or k in pname:
windows[src] = (v_from, v_to)
window_key[src] = k
for key, wf, wt in (images or []): for key, wf, wt in (images or []):
for src in gh_sources: for src in gh_sources:
if src in windows: if src in windows:
@@ -724,12 +855,31 @@ def scan(recipe: str, v_from: str | None, v_to: str | None, registry_dir: str,
continue continue
patched = e.get("patched") or "" patched = e.get("patched") or ""
cands = [_vkey(t) for t in re.findall(r"\d+(?:\.\d+)*", patched)] cands = [_vkey(t) for t in re.findall(r"\d+(?:\.\d+)*", patched)]
# NEVER on a placeholder: "7.4.X" parses to the bare 7.4, which then reads as
# "already fixed at 7.4" and silently drops a real fix (redis CVE-2024-46981).
# A placeholder means the fix version is unknown — that is the indeterminate path.
if (kf and kt and not PLACEHOLDER_RE.search(patched)
and _already_fixed_on_from_line(kf, cands)):
# already had it before the upgrade
e.setdefault("classification", "outside-window")
continue
if kf and kt and _superseded_on_target_line(kt, cands): if kf and kt and _superseded_on_target_line(kt, cands):
# The target's own line got the fix LATER than the target: not fixed here. # The target's own line got the fix LATER than the target: not fixed here.
e.setdefault("classification", "outside-window") e.setdefault("classification", "outside-window")
continue continue
if kf and kt and any(_within(kf, kt, c) for c in cands): if kf and kt and any(_within(kf, kt, c) for c in cands):
got.add(cve) got.add(cve)
elif kf and kt and e.get("affected_max"):
# NVD's `versionEndIncluding X`: affected up to and INCLUDING X, fixed in some
# later release. The exact fix version is unpublished, but the upgrade delivers
# it whenever it crosses X — i.e. from <= X < to.
for t in re.findall(r"\d+(?:\.\d+)*", e["affected_max"]):
x = _vkey(t)
n = max(len(kf), len(kt), len(x))
pad = lambda z: z + (0,) * (n - len(z))
if x and pad(kf) <= pad(x) < pad(kt):
got.add(cve)
break
elif not patched or PLACEHOLDER_RE.search(patched): elif not patched or PLACEHOLDER_RE.search(patched):
# No fix version published ("TBD") or only a placeholder ("7.4.X" — which could # No fix version published ("TBD") or only a placeholder ("7.4.X" — which could
# be 7.4.1, inside the window). We cannot say either way, so say so. # be 7.4.1, inside the window). We cannot say either way, so say so.
+76
View File
@@ -42,6 +42,11 @@ _spec = importlib.util.spec_from_file_location("resolve_images", os.path.join(HE
RI = importlib.util.module_from_spec(_spec) RI = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(RI) _spec.loader.exec_module(RI)
# advisory-scan supplies the source-fetching + changelog-attribution used by --security-sources
_aspec = importlib.util.spec_from_file_location("advisory_scan", os.path.join(HERE, "advisory-scan.py"))
A = importlib.util.module_from_spec(_aspec)
_aspec.loader.exec_module(A)
REGISTRY_DIR = os.environ.get("CCCI_UPSTREAM_REGISTRY", os.path.join(HERE, "upstream")) REGISTRY_DIR = os.environ.get("CCCI_UPSTREAM_REGISTRY", os.path.join(HERE, "upstream"))
USED_RECIPES = os.path.join(HERE, "used-recipes.md") USED_RECIPES = os.path.join(HERE, "used-recipes.md")
DEPRECATION_RE = re.compile( DEPRECATION_RE = re.compile(
@@ -108,6 +113,45 @@ def newest_tag_date(registry: str, repo: str, tag: str) -> str | None:
return (d.get("results") or [{}])[0].get("last_updated") return (d.get("results") or [{}])[0].get("last_updated")
def security_source_audit(recipe: str) -> list[dict]:
"""Per source: are its CVEs USABLE, or merely visible?
The nginx lesson. nginx publishes no GitHub advisories; all its CVEs live in nginx.org/en/CHANGES.
The scan saw them and could do nothing with them, because nothing said which release fixed which
CVE so every nginx bump in the fleet reported 0. Attribution (advisory-scan §2b) fixed that for
changelogs organised by release, but a page that lists CVEs with NO release structure is still a
blind spot: visible, uncountable. This finds those.
Per source: `advisory-feed` (structured, best), `changelog` (CVEs attributable to a release),
`unattributable` (CVEs present but no release structure BLIND), or `no-cve-data`.
"""
urls, _ = _registry_urls(recipe)
out = []
# NVD CPE entries are a first-class source: for projects publishing nothing machine-readable
# (mattermost, mumble) they are the ONLY structured source, and omitting them here made two
# recipes look permanently blind after they had been fixed.
for key, cpe in A.registry_cpes(recipe, REGISTRY_DIR):
e = A.nvd_advisories(cpe, key)
n = len(e.get("advisories") or [])
out.append({"source": e["source"] + f" ({cpe.split(':')[4]}/{cpe.split(':')[3]})",
"kind": "advisory-feed" if n else "no-cve-data",
"status": e["status"], "cves": n, "usable": n})
for entry in A.github_advisories(urls):
out.append({"source": entry["source"], "kind": "advisory-feed",
"status": entry["status"], "cves": len(entry.get("advisories") or []),
"usable": len(entry.get("advisories") or [])})
for entry in A.vendor_pages(urls):
if entry["status"].startswith("skipped"):
continue
n = len(entry.get("cves") or [])
attributed = len(entry.get("fixed_in") or {})
kind = ("no-cve-data" if n == 0 else
"changelog" if attributed else "unattributable")
out.append({"source": entry["source"], "kind": kind, "status": entry["status"],
"cves": n, "usable": attributed})
return out
def audit_recipe(recipe: str, ssh: str | None, quiet_days: int) -> dict: def audit_recipe(recipe: str, ssh: str | None, quiet_days: int) -> dict:
out = {"recipe": recipe, "findings": [], "images": [], "sources": []} out = {"recipe": recipe, "findings": [], "images": [], "sources": []}
try: try:
@@ -214,9 +258,41 @@ def main() -> int:
ap.add_argument("--ssh", default=None) ap.add_argument("--ssh", default=None)
ap.add_argument("--quiet-days", type=int, default=365) ap.add_argument("--quiet-days", type=int, default=365)
ap.add_argument("--json", action="store_true") ap.add_argument("--json", action="store_true")
ap.add_argument("--security-sources", action="store_true",
help="audit whether each recipe's CVE sources are USABLE (structured advisory "
"feed / release-attributable changelog) or merely visible")
a = ap.parse_args() a = ap.parse_args()
recipes = a.recipes or all_recipes() recipes = a.recipes or all_recipes()
if a.security_sources:
# What matters is whether the RECIPE can see CVEs at all — not whether some individual page
# is unparseable. A page with no release structure is harmless when the same project also
# publishes an advisory feed (redis, gitea, minio, clickhouse all do); it is only a blind
# spot when nothing else covers that project.
blind_recipes, noisy = [], 0
for r in recipes:
rows = security_source_audit(r)
feeds = [x for x in rows if x["kind"] == "advisory-feed" and x["cves"] > 0]
logs = [x for x in rows if x["kind"] == "changelog"]
unattr = [x for x in rows if x["kind"] == "unattributable"]
noisy += len(unattr)
usable = len(feeds) + len(logs)
if usable == 0:
blind_recipes.append(r)
print(f"!! {r}: NO USABLE CVE SOURCE — {len(unattr)} unparseable page(s), "
f"0 advisory feeds, 0 attributable changelogs")
for x in rows:
print(f" {x['kind']:15} {x['source'][:64]} ({x['cves']} CVEs)")
else:
print(f"OK {r}: {len(feeds)} advisory-feed(s), {len(logs)} changelog(s)"
+ (f", {len(unattr)} unparseable page(s) (redundant — covered by a feed)"
if unattr else ""))
for x in logs:
print(f" changelog {x['source'][:62]} ({x['usable']}/{x['cves']})")
print(f"\n{len(recipes)} recipes · {len(blind_recipes)} with NO usable CVE source"
+ (f": {', '.join(blind_recipes)}" if blind_recipes else "")
+ f" · {noisy} unparseable page(s) elsewhere (harmless where a feed covers them)")
return 0
reports = [audit_recipe(r, a.ssh, a.quiet_days) for r in recipes] reports = [audit_recipe(r, a.ssh, a.quiet_days) for r in recipes]
if a.json: if a.json:
print(json.dumps(reports, indent=2)) print(json.dumps(reports, indent=2))
+28
View File
@@ -537,6 +537,34 @@ class TestReleaseLineSemantics(unittest.TestCase):
self.assertEqual(rep["fixed_by_this_upgrade"], ["CVE-2025-49844"]) self.assertEqual(rep["fixed_by_this_upgrade"], ["CVE-2025-49844"])
class TestAlreadyFixedOnFromLine(unittest.TestCase):
"""A fix that landed on the line we upgrade FROM was already ours before the upgrade."""
def test_backport_to_our_own_line_is_not_credited(self):
# mattermost patches every maintained line at once. 10.11.22 -> 10.12.4 crosses 10.12.1, but
# 10.11.22 is already past 10.11.4, so the deployment HAD the fix. Counting it credits the
# upgrade with work it did not do.
rep = run_scan([gh("mattermost/mattermost",
[adv("CVE-1", patched="10.11.4; 10.12.1; 10.5.12")])],
v_from="10.11.22", v_to="10.12.4",
urls=["https://github.com/mattermost/mattermost"])
self.assertEqual(rep["fixed_by_this_upgrade"], [])
def test_a_fix_ABOVE_our_position_on_the_same_line_still_counts(self):
rep = run_scan([gh("mattermost/mattermost", [adv("CVE-2", patched="10.11.30; 10.12.1")])],
v_from="10.11.22", v_to="10.12.4",
urls=["https://github.com/mattermost/mattermost"])
self.assertEqual(rep["fixed_by_this_upgrade"], ["CVE-2"])
def test_placeholders_never_feed_this_rule(self):
# "7.4.X" parses to a bare 7.4, which would read as "already fixed at 7.4" and silently drop
# a real fix — this is exactly how redis CVE-2024-46981 was lost when the rule was added.
rep = run_scan([gh("redis/redis", [adv("CVE-3", patched="6.2.X, 7.2.X, 7.4.X")])],
v_from="7.4", v_to="8.10", urls=["https://github.com/redis/redis"])
self.assertIn("CVE-3", rep["indeterminate"])
self.assertEqual(rep["fixed_by_this_upgrade"], [])
class TestChangelogAttribution(unittest.TestCase): class TestChangelogAttribution(unittest.TestCase):
"""Projects that publish no advisory feed still say which release fixed what — in their changelog.""" """Projects that publish no advisory feed still say which release fixed what — in their changelog."""
+5
View File
@@ -18,6 +18,11 @@
- AUTO_MIGRATIONS=true means DB migrations run automatically on backend startup. No manual step needed. - AUTO_MIGRATIONS=true means DB migrations run automatically on backend startup. No manual step needed.
- Minio tag uses a date-based RELEASE.YYYY-MM-DDTHH-MM-SSZ format — abra cannot parse it for upgrades; - Minio tag uses a date-based RELEASE.YYYY-MM-DDTHH-MM-SSZ format — abra cannot parse it for upgrades;
check manually on https://github.com/minio/minio/releases. check manually on https://github.com/minio/minio/releases.
- **2026-08-14: Minio stopped publishing Docker images after RELEASE.2025-09-07T16-13-09Z.**
GitHub has a newer release (`RELEASE.2025-10-15T17-29-55Z`, published 2025-10-16, with CVE fix
GHSA-jjjj-jwhf-8rgr), but the Docker image was never pushed to Docker Hub (returns 404; release
notes say "clone the source and build the latest container"). quay.io checked — only 2022-era
tags. As of this date, `RELEASE.2025-09-07T16-13-09Z` IS the newest available Docker image.
- v5.2.0 adds two optional new env vars: DOCUMENT_ALL_ENDPOINT_ENABLED and OIDC_OP_USER_ENDPOINT_FORMAT. - v5.2.0 adds two optional new env vars: DOCUMENT_ALL_ENDPOINT_ENABLED and OIDC_OP_USER_ENDPOINT_FORMAT.
Both are backward-compatible (no action required for existing deployments). Both are backward-compatible (no action required for existing deployments).
- Recipe version label convention: 0.X.Y+vA.B.C where A.B.C is the impress version. - Recipe version label convention: 0.X.Y+vA.B.C where A.B.C is the impress version.
+9
View File
@@ -75,3 +75,12 @@
recreate DB, reimport dump. `DROP DATABASE WITH (FORCE)` requires PostgreSQL 13+ — safe on postgres:15-alpine. recreate DB, reimport dump. `DROP DATABASE WITH (FORCE)` requires PostgreSQL 13+ — safe on postgres:15-alpine.
The previous inline-label approach (no restore hook) was a defect: raw PGDATA restore without a reload The previous inline-label approach (no restore hook) was a defect: raw PGDATA restore without a reload
was a silent no-op. Fixed in PR #2 (restore fix cherry-picked from PR #1 ci/pg-restore). was a silent no-op. Fixed in PR #2 (restore fix cherry-picked from PR #1 ci/pg-restore).
## NVD CPE fallback
This project publishes nothing machine-readable we can reach — no GitHub advisory feed,
no release-attributable changelog — so its CVE count was `?` (nothing measured). NVD is
CPE-indexed and carries structured version ranges, so it can answer where the vendor
cannot. It LAGS the vendor, so it is a fallback, never the primary source.
- nvd-cpe: mattermost-team-edition = cpe:2.3:a:mattermost:mattermost_server:*:*:*:*:*:*:*:*
- nvd-cpe: postgres = cpe:2.3:a:postgresql:postgresql:*:*:*:*:*:*:*:*
+8
View File
@@ -19,3 +19,11 @@
- The server image tag is `v<version>-<build>` (e.g. `v1.6.870-4`); the trailing number is the image - The server image tag is `v<version>-<build>` (e.g. `v1.6.870-4`); the trailing number is the image
build, not an app version, and moves independently of upstream releases — `abra recipe upgrade` build, not an app version, and moves independently of upstream releases — `abra recipe upgrade`
reports "no new versions" for it, so use `resolve-images.py` to see those bumps. reports "no new versions" for it, so use `resolve-images.py` to see those bumps.
## NVD CPE fallback
This project publishes nothing machine-readable we can reach — no GitHub advisory feed,
no release-attributable changelog — so its CVE count was `?` (nothing measured). NVD is
CPE-indexed and carries structured version ranges, so it can answer where the vendor
cannot. It LAGS the vendor, so it is a fallback, never the primary source.
- nvd-cpe: mumble-server = cpe:2.3:a:mumble:mumble:*:*:*:*:*:*:*:*
+15
View File
@@ -50,3 +50,18 @@
- 2026-08-07 run: operator directed 2.33.3 -> 2.34.2 (the newest). The whole 2.34.x line is still - 2026-08-07 run: operator directed 2.33.3 -> 2.34.2 (the newest). The whole 2.34.x line is still
marked Pre-release on GitHub (2.33.5 holds the Latest badge); flagged in the PR body. No breaking marked Pre-release on GitHub (2.33.5 holds the Latest badge); flagged in the PR body. No breaking
changes across 2.33.3 -> 2.34.2; rolling upgrade safe (TypeORM migrations auto-run on boot). changes across 2.33.3 -> 2.34.2; rolling upgrade safe (TypeORM migrations auto-run on boot).
- 2.35.0 (2026-08-11, Pre-release): major feature release — self-hosted AI Assistant onboarding,
Simplified Custom Auth credentials, Agent Builder test runs + HITL, Discord agent chat channel,
local agent token counting, **VM expression engine now the default** (was opt-in), MCP SDK v2
migration + MCP 2026-07-28 discovery handshake, Kafka Node v2, Salesforce OAuth2 JWT, GitHub
dispatch timeout, X/Twitter Node OAuth2/API migrated to x.com, Azure Key Vault configurable
endpoints, Postgres-version startup warning, workflow review improvements (diffs, metadata, version
descriptions), and numerous core/editor bugfixes. No breaking compose/config/migration changes.
- 2.35.1 (2026-08-12, Pre-release): 2 core bugfixes — data-tables resume scope, TLS options per hop
through a proxy.
- 2.35.2 (2026-08-13, Pre-release): 1 core bugfix — report real activation mode for triggers via
publication outbox. **Deployed on cc-ci 2026-08-15**: 2.34.4→2.35.2, TypeORM migrations clean,
editor served HTTP 200. No breaking changes, no N8N_* env renames, no required operator action.
- 2.35.3 (2026-08-14, Pre-release): bugfixes (Google Ads v21→v25 API migration, MS Teams OAuth scope
restore, workflow publication outbox abort deadline) + feature (skip update approval for workflows
from same Instance AI session). Not deployed (2.35.2 was the survey target).
@@ -369,12 +369,15 @@ SSHCFG
User = "loops"; Group = "users"; User = "loops"; Group = "users";
WorkingDirectory = "/srv/cc-ci"; WorkingDirectory = "/srv/cc-ci";
# Optional per-run overrides for backend/model (LOOP_BACKEND, LOOP_MODEL, OPENCODE_SHARE, # Optional per-run overrides for backend/model (LOOP_BACKEND, LOOP_MODEL, OPENCODE_SHARE,
# UPGRADER_ARGS, …). The leading "-" makes it optional: absent file → claude/sonnet defaults # UPGRADER_ARGS, …). The leading "-" makes it optional: absent file → claude/sonnet defaults.
# (current behavior). To run the weekly job on e.g. opencode-go/glm-5.2, drop a file with # Current config (as of 2026-08-16): the upgrader + report run on tinfoil/deepseek-v4-pro
# LOOP_BACKEND=opencode # (LOOP_MODEL + REPORT_MODEL in the env file); the hourly SUPERVISOR stays on glm-5.2
# LOOP_MODEL=opencode-go/glm-5.2 # (SUPERVISOR_MODEL defaults to opencode-go/glm-5.2 in launch-supervisor.py, NOT overridden
# No rebuild needed to switch — the env file is read at each timer fire. Holds no secrets # here). Subagents bind deepseek via the cc-ci repo's opencode config. LOOP_TIER=zen is kept
# (the opencode-go API key lives in ~/.local/share/opencode/auth.json, mode 600). # so the tier check passes; the watchdog's usage-limit probe sends the deepseek model name to
# the zen endpoint, which returns 200 (not 429) → resume immediately (correct: tinfoil has no
# rolling usage limit to wait out). No rebuild needed to switch — the env file is read at each
# timer fire. Holds no secrets (the tinfoil API key lives in the opencode config / auth.json).
EnvironmentFile = "-/srv/cc-ci/upgrader.env"; EnvironmentFile = "-/srv/cc-ci/upgrader.env";
}; };
environment = { HOME = "/home/loops"; CLAUDE_BIN = "/home/loops/.local/bin/claude"; }; environment = { HOME = "/home/loops"; CLAUDE_BIN = "/home/loops/.local/bin/claude"; };