merge origin/main

# Conflicts: # cc-ci-plan/JOURNAL.md
2026-09-28 20:03:33 +00:00
parent aaaec02f93 71e3d0e852
commit bf84f7f300
26 changed files with 792 additions and 71 deletions
@@ -31,8 +31,44 @@ handoff).

 ---

-## Session 2026-05-31 ~18:30 UTC — Claude Sonnet 4.6
+## Session 2026-09-14 ~16:45 UTC — opencode glm-5.3-flash (orchestrator) — round 2: Anubis UA

+**Left off:** The real root cause turned out to be TWO independent layers; the mirror-privacy
+fix (earlier session entry today) was necessary but not sufficient. Operator's browser console
+showed CORS failures redirecting to `anubis.swarm.autonomic.zone/.within.website/?redir=…`.
+Reproduced exactly: the `/pr/` proxy forwards the END browser's User-Agent to Gitea; Gitea sits
+behind **Anubis**, which 307-challenges browser-like UAs to `anubis.swarm.autonomic.zone`
+(no CORS headers) → every fetch throws in the browser → all cells "?" (curl passed clean, which
+is why server-side checks and my earlier headless test never saw it — intermittent/rate-dependent
+for my playwright run). Fix: cc-ci **PR #38** adds
+`proxy_set_header User-Agent "ccci-reports-proxy/1.0";` to the reports.nix `/pr/` location.
+Hot-verified on the host by mount-swapping a fixed conf into the running task (one mis-step:
+`--mount-rm`+`--mount-add` same-target order wiped the mount; re-added), scoped live, all 16
+cells rendering with a real Chromium. Merged PR #38, `nix flake update cc-ci`,
+`nixos-rebuild test` → healthy (reports 200, no failed units) → `switch` (flake.lock commit
+9e7770f). Final verify: browser-UA curl 200 both gitea/9 + full headless-Chromium sweep 16/16
+OPEN, zero non-200 /pr fetches. Also flipped memory: `memory/gitea-anubis-ua-challenge.md` +
+MEMORY.md index.
+**Open:** nothing blocking; next weekly /recipe-report and STATUS live-checks carry the fix.
+
+## Session 2026-09-14 ~15:00 UTC — opencode glm-5.3-flash (orchestrator)
+
+**Left off:** Report STATUS column fix. Operator reported the week-2026-09-11 report's live
+PR-STATUS column all "?" — root cause: the tokenless same-origin proxy
+`report./pr/<recipe>/<n>` (cc-ci `nix/modules/reports.nix`) 404s on **private** mirrors; two
+late-enrolled mirrors, `recipe-maintainers/gitea` (2026-06-11) and `wordpress` (2026-08-03),
+had been created `"private":true` from birth — by the stale instruction in
+`/recipe-enroll`'s mirror step (the other 21 mirrors were flipped public on 2026-06-09, and
+the old 'org is private' blocker is long resolved). Fixed: secret-scanned both repos, flipped
+`private=false` (PATCH with bot creds), patched `.opencode/skills/recipe-enroll/SKILL.md` to
+create mirrors `private:false`, updated memory/recipe-mirrors-public-org-blocker.md +
+MEMORY.md index. **Verified in a real headless Chromium (nixpkgs chromium + playwright)**:
+all 16 STATUS rows render `open`, every `/pr/` fetch 200 JSON. Commit 6e93922 pushed. The
+STATUS column refreshes live every 30s; cells go ✓ when a PR merges. No reports.nix change
+was needed (proxy itself was healthy).
+**Open:** nothing on this; the report index regenerates next weekly run.
+
+## Session 2026-05-31 ~18:30 UTC — Claude Sonnet 4.6
 **Left off:** Got opencode/deepseek-v4-pro working as the loop backend. Both builder and
 adversary are actively running on `tinfoil/deepseek-v4-pro` (via `inference.tinfoil.sh`).
 Phase 5 [11/11] in progress. The operator is debugging the opencode web UI visibility and
@@ -1202,6 +1238,56 @@ the host: `opencode-go/deepseek-v4-flash` and `opencode-go/glm-5.3-flash` answer
  `upgrader.env` (`LOOP_TIER=go` maps to the `opencode-go` auth entry; `LOOP_MODEL` overrides the
  tier default). Next fire Fri 2026-09-11 02:00 UTC.
 - The steering orchestrator agent stays on `opencode-go/glm-5.2` (not asked to change).
+
+## Session 2026-09-21 — domain cutover to ci.autonomic.zone (orchestrator)
+
+**What happened.** Morning: hourly supervisor resumed the stalled 09-18 weekly run on the GO tier
+(ZEN endpoint dead server-side — `UnknownError`; run completed 13 green PRs, 0 failed). Published
+the missing week-2026-09-18 report (PR-finding #4; launcher defaults flipped to `go` in
+cc-ci-orchestrator PR #23). Then executed the full domain cutover per
+`cc-ci-plan/plan-domain-migration-ci-autonomic-zone.md` (PR #25).
+
+**Plan deviation (simplification).** No dual-cert SNI: ONE Let's Encrypt cert carries SANs for
+BOTH zones (`ci` + `*.ci` of autonomic.zone AND commoninternet.net) — the unchanged single-pair
+`ssl_cert/ssl_key` traefik reconciler keeps working; Phase 4 reissues without the legacy SANs.
+The new zone's DNS-01 challenge reuses the SAME acme-dns account: storage re-keyed by
+`cc-ci-acme-storage-seed.service` (jq clone of the legacy entry under `ci.autonomic.zone`),
+CNAME already delegated. Proven by a hand lego **staging** run before any production change.
+
+**Merged:** cc-ci #39 (front doors dual Host rules, bridge/dashboard env URLs, drone abra rename,
+runner RPC, naming.py → `*.ci.autonomic.zone`, dual-zone name regexes in lifecycle/warm/prune,
+recipe-report URLs) · cc-ci #40 (seed-unit nesting fix) · cc-ci #41 (have_secret stack-scope —
+caught live: the old stack's `*_rpc_secret_v1` satisfied the check post-rename) · cc-ci #42 (nix
+interpolation escape) · orchestrator #27 (oc.ci host + `opencodeUiExtraHosts`, host self-pins,
+flake bump).
+
+**Deployed** via `nixos-rebuild test` → verify → `switch` (generation `nn1vwiv7v1k…`, running ==
+boot). Cert SANs confirmed 4-name; all 5 front doors answer on BOTH zones
+(200/200/303/401 + traefik 200), TLS verify=0 from outside; zero failed units; disk dropped
+88%→45% after prune.
+
+**Drone migration.** New abra app `drone.ci.autonomic.zone`, FRESH DB (module's
+`DRONE_USER_CREATE` re-injected the sops bridge token). The Gitea OAuth app redirect now has both
+URIs; the client secret was rotated (each Gitea PATCH regenerates it) and synced through
+sops → `sops-install-secrets` → swarm secret v1 → drone. Bootstrapped OAuth
+(`drone login ok (admin=true)`), re-enabled cc-ci + discourse repos, build timeout 60m.
+Webhooks: ghost + discourse repointed (secrets preserved); cc-ci repo's bridge webhook →
+`ci.autonomic.zone/hook`, stale drone hook deleted, Drone's auto-created new-zone hook active.
+Old stack removed + orphaned secrets reaped.
+
+**E2E proof.** `!testme` on keycloak PR #9 → bridge → drone build #1 (new DB numbering) → runner →
+harness → `results.json` + PR card `✅ passed` linking `ci.autonomic.zone/runs/1/summary.png`.
+
+**Deferred / open.**
+- Warm stacks + backupbot stay on the legacy zone (data-warm volumes / restic password tied to
+  abra app names) — post-bake migration; harness regexes accept both zones meanwhile.
+- Docs sweep (~60 references: AGENTS.md, README, skills incl. cc-ci-status front-door list,
+  launcher printed URLs).
+- Phase 4 (after ≥7 clean days): drop legacy SANs (reissue), remove legacy Host arms + host
+  self-pins + Gandi records (`ci`, `*.ci`, `_acme-challenge.ci`), TTLs back to 3600.
+- Host auto-update was `failed` 2026-09-15 (health check) — `/cc-ci-orchestrator-update` still
+  pending; next auto-attempt Tue 09-22.
+
 ## Session 2026-09-28 20:00 UTC — operator-broken cc-ci recovered by plain hard reset

 - Operator reported ci.autonomic.zone down after their own change, supplied a Hetzner API token
@@ -2,8 +2,9 @@
 """cc-ci recipe-report launcher — one-shot agent that runs /recipe-report after the weekly upgrade.

 The report agent's model is configured SEPARATELY from the upgrader (REPORT_BACKEND/REPORT_MODEL),
-but defaults to the same OpenCode subscription on glm-5.2 as the upgrader. The model default tracks
-the backend+tier (opencode+zen→opencode/glm-5.2, opencode+go→opencode-go/glm-5.2, claude→opus).
+but defaults to the same OpenCode subscription as the upgrader (now OpenCode GO — the only tier
+with a key on this host; ZEN models are dead here, see launch-upgrader.py). The model default tracks
+the backend+tier (opencode+go→opencode-go/glm-5.3-flash, opencode+zen→opencode/glm-5.2, claude→opus).

 Usage:
  launch-report.py start [DATE]   use-or-create the session; runs /recipe-report [DATE]
@@ -12,7 +13,8 @@ Usage:
  launch-report.py status         show session state

 Env: REPORT_BACKEND (default opencode), REPORT_MODEL (default tracks tier),
-     LOOP_TIER / REPORT_TIER (default 'zen'; 'go' or 'zen'; only for opencode),
+     LOOP_TIER / REPORT_TIER (default 'go' — the only tier with a key on this host; ZEN
+     models are dead here; 'go' or 'zen'; only for opencode),
     OPENCODE_SHARE (1=attach web server + public --share link), REPORT_SESSION, REPORT_DIR.
 """
 import os, subprocess, sys, time
@@ -23,9 +25,9 @@ SESSION = os.environ.get("REPORT_SESSION", "cc-ci-report")
 WORKDIR = os.environ.get("REPORT_DIR", "/srv/cc-ci")
 LOG_DIR = os.environ.get("LOG_DIR", "/srv/cc-ci/.cc-ci-logs")
 BACKEND = os.environ.get("REPORT_BACKEND", "opencode")
-TIER = os.environ.get("LOOP_TIER", os.environ.get("REPORT_TIER", "zen"))
+TIER = os.environ.get("LOOP_TIER", os.environ.get("REPORT_TIER", "go"))
 _TIER_MODEL = {"go": "opencode-go/glm-5.2", "zen": "opencode/glm-5.2"}
-_DEFAULT_MODEL = _TIER_MODEL.get(TIER, _TIER_MODEL["zen"]) if BACKEND == "opencode" else "opus"
+_DEFAULT_MODEL = _TIER_MODEL.get(TIER, _TIER_MODEL["go"]) if BACKEND == "opencode" else "opus"
 MODEL   = os.environ.get("REPORT_MODEL", _DEFAULT_MODEL)
 CLAUDE_BIN   = os.environ.get("CLAUDE_BIN", "claude")
 CLAUDE_FLAGS = os.environ.get("CLAUDE_FLAGS", "--dangerously-skip-permissions")
@@ -15,14 +15,16 @@ Usage:

 Env:
  LOOP_BACKEND     opencode (default) | claude   — also accepts UPGRADER_BACKEND
-  LOOP_TIER       opencode subscription tier: "zen" (OpenCode ZEN, default) or "go"
-                   (OpenCode Go). Selects the default model + the usage-limit probe
-                   endpoint/key. Go hit a monthly limit; ZEN is the working alternative.
-                   Only affects the opencode backend (ignored for claude).
+  LOOP_TIER       opencode subscription tier: "go" (OpenCode Go, DEFAULT — the only tier
+                    with a key on this host) or "zen" (OpenCode ZEN, opt-in on a host that
+                    has a ZEN key). Selects the default model + the usage-limit probe
+                    endpoint/key. ZEN models are DEAD here (no key, gone from the
+                    provider catalogue — see the TIER block below).
+                    Only affects the opencode backend (ignored for claude).
  LOOP_MODEL       model flag (overrides UPGRADER_MODEL); default tracks backend+tier —
-                   opencode+zen→opencode/glm-5.2, opencode+go→opencode-go/glm-5.2, claude→sonnet
-  UPGRADER_MODEL   provider/model for opencode, e.g. opencode/glm-5.2 (OpenCode ZEN),
-                   opencode-go/glm-5.2 (OpenCode Go), or tinfoil/deepseek-v4-pro; sonnet for claude
+                    opencode+go→opencode-go/glm-5.3-flash, opencode+zen→opencode/glm-5.2, claude→sonnet
+  UPGRADER_MODEL   provider/model for opencode, e.g. opencode-go/glm-5.3-flash (OpenCode Go, default),
+                    opencode/glm-5.2 (OpenCode ZEN), or tinfoil/deepseek-v4-pro; sonnet for claude
  UPGRADER_ARGS    extra args passed to /upgrade-all (e.g. "n8n ghost", "--dry-run")

  claude backend:
@@ -42,11 +44,13 @@ WORKDIR = os.environ.get("UPGRADER_DIR",     "/srv/cc-ci")
 LOG_DIR = os.environ.get("LOG_DIR",          "/srv/cc-ci/.cc-ci-logs")

 # LOOP_BACKEND / LOOP_MODEL take precedence (unified control from the operator).
-# LOOP_TIER selects the OpenCode subscription: "zen" (default) or "go". Go hit a monthly
-# usage limit; ZEN is the working alternative. Set LOOP_TIER=go when the Go limit resets.
+# LOOP_TIER selects the OpenCode subscription: "go" (DEFAULT — the only tier with a key on
+# this host) or "zen" (opt-in on a host that has a ZEN key). ZEN models are DEAD here — the
+# endpoint errors server-side, and env loss in the tmux→agent→launch-report chain is exactly
+# how it silently ate the 2026-09-18 weekly report. Never default back to zen.
 # Only affects the opencode backend; claude ignores the tier.
 BACKEND = os.environ.get("LOOP_BACKEND", os.environ.get("UPGRADER_BACKEND", "opencode"))
-TIER = os.environ.get("LOOP_TIER", os.environ.get("UPGRADER_TIER", "zen"))
+TIER = os.environ.get("LOOP_TIER", os.environ.get("UPGRADER_TIER", "go"))
 _TIER_CFG = {
    "go":  {"model": "opencode-go/glm-5.2", "endpoint": "https://opencode.ai/zen/go/v1/chat/completions",
            "auth": "opencode-go", "label": "OpenCode Go"},
@@ -56,7 +60,7 @@ _TIER_CFG = {
 if BACKEND == "opencode" and TIER not in _TIER_CFG:
    print(f"[upgrader] ERROR: unknown LOOP_TIER '{TIER}' — use 'go' or 'zen'", flush=True)
    sys.exit(1)
-_tier = _TIER_CFG.get(TIER, _TIER_CFG["zen"])
+_tier = _TIER_CFG.get(TIER, _TIER_CFG["go"])
 _DEFAULT_MODEL = _tier["model"] if BACKEND == "opencode" else "sonnet"
 MODEL   = os.environ.get("LOOP_MODEL",   os.environ.get("UPGRADER_MODEL",   _DEFAULT_MODEL))

@@ -0,0 +1,172 @@
+# Plan: migrate cc-ci domains from `*.commoninternet.net` to `ci.autonomic.zone`
+
+**Date:** 2026-09-21 · **Status:** DRAFT — blocked only on operator DNS access (Gandi zone `autonomic.zone`)
+**Principle:** same host, same IP (195.201.88.249), same services — only names change. Dual-run new
+alongside old; old names retire last; rollback at any point is "change nothing further" because the
+old DNS zone keeps answering until Phase 4.
+
+---
+
+## 1. Inventory — what answers on what today
+
+| Name | Served by | Router/config source |
+|---|---|---|
+| `ci.commoninternet.net` | results dashboard + bridge `/hook` | `cc-ci/nix/modules/dashboard.nix`, `bridge.nix` (traefik Host rules) |
+| `report.ci.commoninternet.net` | weekly report static site | `cc-ci/nix/modules/reports.nix` |
+| `drone.ci.commoninternet.net` | Drone server | `cc-ci/nix/modules/drone.nix` (abra app; env `DOMAIN`, `DRONE_SERVER_HOST`); runner RPC: `drone-runner.nix` |
+| `oc.ci.commoninternet.net` | opencode UI (traefik → host nginx basic auth) | `cc-ci-orch/nix/modules/orchestrator-host.nix` (`opencodeUiHost`), set in `nix/hosts/cc-ci/configuration.nix` |
+| `traefik.ci.commoninternet.net` | traefik stack identity + deploy-proxy health gate | `cc-ci/nix/modules/proxy.nix` |
+| `warm-*.ci.commoninternet.net` | warm canonical recipe stacks | `warm-keycloak.nix`, warm-bluesky-pds (caddy labels) |
+| `<tag>-<6hex>.ci.commoninternet.net` | per-run recipe test deploys | `cc-ci/runner/harness/naming.py` `app_domain()` — **resolves via public DNS** (containers use 1.1.1.1/8.8.8.8) |
+| `backups.ci.commoninternet.net` | backup-bot-two — stack name only, no web route | `backupbot.nix` |
+| `acme.commoninternet.net` (NS `ns-acme`) | local acme-dns (systemd, port 53) | `cc-ci/nix/modules/acme-dns.nix` |
+
+**Cert chain today:** nixos `security.acme` (lego) issues `ci.commoninternet.net` + `*.ci.commoninternet.net`
+via DNS-01 through the local acme-dns; Gandi delegates `_acme-challenge.ci.commoninternet.net` CNAME →
+`9d3f0428-eb97-4b14-bb07-da2f0f6c2073.acme.commoninternet.net`. `postRun` fires
+`cc-ci-acme-traefik-handoff.service` → installs cert into `/var/lib/ci-certs/live` → restarts
+`deploy-proxy.service`, whose reconciler re-derives the `traefik_ci_commoninternet_net_ssl_{cert,key}_v<hash>`
+swarm secrets (traefik runs co-op-cloud wildcard/no-ACME mode off those secrets).
+**DNS:** both zones on Gandi. IPv4-only (no AAAA records anywhere; none needed).
+
+**External references:** Gitea OAuth app for Drone (redirect URI → drone.ci…); per-repo webhooks →
+`/hook` — **only `recipe-maintainers/ghost` and `recipe-maintainers/discourse` have any** (everything
+else is bridge-driven via the Drone API); ~60 doc references across cc-ci-orch (AGENTS.md, README,
+plan.md, skills).
+
+## 2. Target naming
+
+| Old | New |
+|---|---|
+| ci.commoninternet.net | ci.autonomic.zone |
+| report.ci.commoninternet.net | report.ci.autonomic.zone |
+| drone.ci.commoninternet.net | drone.ci.autonomic.zone |
+| oc.ci.commoninternet.net | oc.ci.autonomic.zone |
+| traefik.ci.commoninternet.net | traefik.ci.autonomic.zone |
+| *.ci.commoninternet.net (warm-\*, per-run apps, backups identity) | *.ci.autonomic.zone |
+| acme.commoninternet.net | **stays** through Phases 1–2 (infra, not a front door); optional Phase 3 |
+
+Swarm **stack names** (`traefik_ci_commoninternet_net`, `ccci-*`, warm stacks) are identifiers, not
+domains — keep them where possible to avoid churn in keep-lists/prune filters, EXCEPT abra-managed
+apps whose name **is** the domain (drone, backup-bot, per-run apps).
+
+## 3. DNS records to set (zone `autonomic.zone`, Gandi — operator)
+
+| # | Name | Type | Value | TTL | Purpose |
+|---|---|---|---|---|---|
+| 1 | `ci.autonomic.zone` | A | `195.201.88.249` | 300 | apex front door (wildcard does NOT cover the apex) |
+| 2 | `*.ci.autonomic.zone` | A | `195.201.88.249` | 300 | report/drone/oc/traefik/warm-\*/`<tag>-<6hex>` run apps |
+| 3 | `_acme-challenge.ci.autonomic.zone` | CNAME | `9d3f0428-eb97-4b14-bb07-da2f0f6c2073.acme.commoninternet.net.` | 300 | DNS-01 for the new wildcard — **reuses the existing restricted acme-dns account** (`disable_registration=true`; the lego *account* authenticates, not the FQDN, so one acme-dns subdomain serves both zones) |
+
+Also: lower TTL to 300 on the old `ci`/`*.ci` records a few days ahead (fast rollback later).
+Phase 3 (optional) adds: `acme.ci.autonomic.zone` + `ns-acme.ci.autonomic.zone` A records.
+
+**DNS verification (before touching the host):**
+```bash
+dig +short ci.autonomic.zone A @ns-187-a.gandi.net   # 195.201.88.249
+dig +short report.ci.autonomic.zone @1.1.1.1          # 195.201.88.249 (via wildcard)
+dig +short drone.ci.autonomic.zone @8.8.8.8           # 195.201.88.249
+dig +short _acme-challenge.ci.autonomic.zone CNAME    # 9d3f0428-….acme.commoninternet.net.
+```
+
+## 4. Phase 1a — certificate (issue BEFORE any routing change)
+
+1. `cc-ci/nix/modules/acme-dns.nix`: add a second `security.acme.certs."ci.autonomic.zone"` entry
+   (domain + `*.ci.autonomic.zone` SAN, `dnsProvider = "acmedns"`, `dnsResolver = "1.1.1.1:53"`).
+   Split the lego env per cert — today `ACME_DNS_STORAGE_PATH` is baked to
+   `/var/lib/acme/ci.commoninternet.net/…`; generate one env file per cert so each keeps its own
+   acme-dns account file.
+2. Extend `cc-ci-acme-traefik-handoff` to stage the new cert alongside the old
+   (`/var/lib/ci-certs/live` holds both, e.g. `fullchain.pem` = new + `fullchain-old.pem`).
+3. Extend the **deploy-proxy reconciler** (`proxy.nix`) for **dual-cert SNI**: new wildcard pair as
+   `ssl_cert/ssl_key` (default store) + old pair as a second secret pair wired through a small
+   file-provider `[[tls.certificates]]` so `*.ci.commoninternet.net` keeps serving during the bake
+   window. This is the one fiddly bit — validate against the **Let's Encrypt staging** endpoint
+   first (the module already has the staging-marker pattern).
+4. `nixos-rebuild test` → confirm both certs issued and served (`openssl s_client -servername` for a
+   name in each zone, check SANs) → `switch`.
+
+## 5. Phase 1b — routing + service config (one stack at a time, health-gated)
+
+During the bake window every router rule carries **both** hostnames
+(`Host(`ci.autonomic.zone`) || Host(`ci.commoninternet.net`)`); old names drop out only in Phase 4.
+
+- `dashboard.nix` + `bridge.nix` (share the `ci` host: dashboard `/` + bridge `/hook`), `reports.nix`,
+  and the traefik health-gate name in `proxy.nix`.
+- `bridge.py` / `dashboard.py` code defaults for `DRONE_URL` / `DASH_URL`.
+- `recipe-report.py` — footer, publish URL text and its verification curl → `report.ci.autonomic.zone`
+  (regenerate the index after, so the home page stops linking the old name).
+- Drone **abra app rename** (name = domain, so this is a new stack):
+  1. Export enabled-repo list from the old Drone API.
+  2. Deploy new abra app `drone.ci.autonomic.zone` (same module; secrets `rpc_secret`,
+     `client_secret` re-insert from `/run/secrets` as today). **Fresh DB** — Drone data volume is
+     stack-name-prefixed; repos re-sync on Gitea login, build history is lost (accepted; export the
+     old repo list and re-enable via API).
+  3. Point `drone-runner.nix` `DRONE_RPC_HOST` at the new name; restart runner.
+  4. Re-run `scripts/bootstrap-drone-oauth.sh` with `DRONE=https://drone.ci.autonomic.zone`
+     (updates the Gitea OAuth redirect URI; script already uses `--resolve` to hit loopback).
+  5. Update the two webhooks (ghost, discourse) to `https://drone.ci.autonomic.zone/hook?secret=<same>`
+     — read the secret off the existing hook config before deleting it.
+  6. `abra app undeploy` + remove the old drone app once green.
+- `configuration.nix`: `opencodeUiHost = "oc.ci.autonomic.zone"`; extend the `networking.hosts`
+  self-pin list with the new names (keep old pins during bake).
+- Gitea `GITEA_SERVER` stays `git.autonomic.zone` — unchanged.
+
+## 6. Phase 2 — harness / warm / per-run naming (the `_ci_commoninternet_net` blast radius)
+
+- `cc-ci/runner/harness/naming.py` `app_domain()` → `.ci.autonomic.zone` (single source of truth).
+- Regexes/patterns that match the derived names: `docker-prune.nix` run-app filter,
+  `tests/concurrency/concutil.py` (`DOMAIN` const + `RUN_APP_RE`), any separate `RUN_APP_RE` in
+  `lifecycle.py` / `warm.py` / `warm_reconcile.py` / `canonical.py` / `generic.py` / `http.py`,
+  `nightly-sweep` keep-list patterns, `tests/gitea/recipe_meta.py` comment.
+- Redeploy warm stacks under new names (abra `warm-<recipe>.ci.autonomic.zone`), reconcile, drain old.
+- `backupbot.nix` `DOMAIN` (stack-name-only change).
+- Verify: full deterministic test suite + one live `!testme` run end-to-end.
+
+## 7. Phase 3 (optional) — move acme-dns off commoninternet.net
+
+Only if `commoninternet.net` is being dropped entirely: new `domain`/`nsname` in acme-dns config
+(`acme.ci.autonomic.zone` / `ns-acme.ci.autonomic.zone` + Gandi records), briefly re-enable
+registration or hand-migrate the account DB, re-point `_acme-challenge` CNAMEs, re-issue both certs.
+
+## 8. Phase 4 — retirement (after ≥7 clean days)
+
+Remove old names from the dual router rules → remove `security.acme.certs."ci.commoninternet.net"`
+ its env file + old cert secret pair → drop old `networking.hosts` pins → delete old Gandi records
+(`ci`, `*.ci`, `_acme-challenge.ci`) → restore TTLs to 3600. Keep `acme.commoninternet.net` unless
+Phase 3 was done.
+
+## 9. Verification checklist
+
+- **DNS:** the dig table in §3 against a Gandi NS and a public resolver.
+- **Cert:** `openssl s_client -connect 195.201.88.249:443 -servername <name>` for one name per zone —
+  SANs correct for both, `ssl_verify_result=0`.
+- **Front doors:** `ci` → 200 · `report.ci` → 200 · `drone.ci` → 200/303 · `oc.ci` → **401**
+  (basic auth intact) · `traefik.ci/api/version` → 200.
+- **SSO:** Drone login through Gitea on the new redirect URI.
+- **Webhook:** empty commit to `recipe-maintainers/ghost` main → build fires on the new host.
+- **Bridge/!testme:** `!testme` on a scratch PR → build runs, `results.json` written, PR comment
+  contains **new** domain links.
+- **Report:** publish one report page and verify it at `https://report.ci.autonomic.zone`.
+- **Host:** `systemctl --failed` empty; `deploy-proxy` reconcile logs clean.
+
+## 10. Risks / gotchas
+
+- **Dual-cert SNI in the traefik reconciler** is the highest-risk change — stage-first, verify SNI
+  selection for both zones before switching.
+- **Drone DB** does not follow the abra rename — accept fresh build history; export/re-apply the
+  enabled-repo list; re-insert secrets from `/run/secrets` (module already does).
+- **Per-run recipe domains resolve via public DNS** — the `*.ci.autonomic.zone` wildcard A record is
+  load-bearing for every test deploy; never merge Phase 2 before it propagates.
+- Old PR comments/badges link old names — they keep working until Phase 4 removes the old records.
+- ~60 doc references (AGENTS.md, README, plan.md, skills, status skill) — one cosmetic doc PR at the
+  end so the docs don't lie.
+- No domain-bearing secret values were found (webhook secrets are per-repo and copied as-is).
+
+## 11. Sequencing
+
+1. **Operator:** TTLs down + 3 Gandi records (§3) — 5 minutes.
+2. **Agent:** cert PR (§4) → `test` → verify dual SNI → `switch`.
+3. **Agent:** routing/env PRs (§5) stack-by-stack, health-gated; OAuth + webhooks + drone rename.
+4. **Agent:** end-to-end verification (§9) + report republish under the new name.
+5. **Bake ≥7 days** → Phase 2 PR → full suite → bake → Phase 4 retirement.
@@ -303,7 +303,7 @@ def render(spec_path, out_path):
                       for p in lead.split("\n\n") if p.strip())
    body = (_mast() +
            f'<div class="dateline"><span>{_esc(sub)}</span>'
-            f'<span>report.ci.commoninternet.net</span><span>{gen}</span></div>'
+            f'<span>report.ci.autonomic.zone</span><span>{gen}</span></div>'
            f'<div class="lead">{lead}</div>')
    # 1) the full wire — every recipe, in the agent's recommended priority order (CVEs first); CVEs column.
    wire = ("The full wire — every recipe, in priority order" if kind == "week"
@@ -322,7 +322,7 @@ def render(spec_path, out_path):
    if s.get("changes"):
        body += f'<h2>What changed</h2>{_changes(s.get("changes"), repo_url)}'
    body += (f'<footer>{title} · generated {gen} · '
-             f'<a href="https://ci.commoninternet.net/">dashboard</a> · <a href="./">archive</a></footer>')
+             f'<a href="https://ci.autonomic.zone/">dashboard</a> · <a href="./">archive</a></footer>')
    open(out_path, "w").write(_page(f"{title} · " + s["date"], body))
    print("wrote", out_path)

@@ -347,10 +347,10 @@ def publish(html_path, date, kind="week"):
        for d, k in sorted(set(entries), reverse=True))
    idx = _page("The Recipe Report — Archive", _mast() +
                '<div class="dateline"><span>Weekly review of Co-op Cloud recipe upgrades &amp; CI</span>'
-                '<span>report.ci.commoninternet.net</span></div>'
+                '<span>report.ci.autonomic.zone</span></div>'
                f'<ul class="idx">{lis or "<li><em>No reports yet.</em></li>"}</ul>')
    subprocess.run(["ssh", "cc-ci", f"cat > {HOST_REPORTS}/index.html"], input=idx.encode(), check=True)
-    print(f"published https://report.ci.commoninternet.net/{page}  (+ index)")
+    print(f"published https://report.ci.autonomic.zone/{page}  (+ index)")


 def main():
@@ -2,7 +2,7 @@

 | service | image | source repo | releases / changelog |
 |---------|-------|-------------|----------------------|
-| app | ghcr.io/bluesky-social/pds | https://github.com/bluesky-social/pds | git tags `vX.Y.Z` (no GitHub Releases objects — `releases/latest` 404s) |
+| app | ghcr.io/bluesky-social/pds | https://github.com/bluesky-social/pds | git tags `vX.Y.Z` 1:1 with image tags (no GitHub Releases objects — `releases/latest` 404s); app-level changes live in the atproto monorepo changelog: https://github.com/bluesky-social/atproto/blob/main/packages/pds/CHANGELOG.md (image `0.4.5NNN` == `@atproto/pds` `0.5.NN`) |
 | caddy | docker.io/caddy:2 | https://github.com/caddyserver/caddy | https://github.com/caddyserver/caddy/releases |

 ## Standing notes (researched 2026-06-11, bsky phase)
@@ -9,6 +9,7 @@
 - nginx even-numbered minor versions (1.28.x, 1.30.x) are mainline; odd (1.27.x, 1.29.x) are stable. The recipe tracks mainline.
 - compose.git-pull.yml is an optional overlay for git-pull functionality; alpine/git version lives there.
 - compose.sftp.yml and compose.sso.yml are other optional overlays; linuxserver/openssh-server uses `latest` tag and is not version-pinned.
+- nginx 1.31.6 (15 Sep 2026) is a security patch release: CVE-2026-90439 (heap buffer overflow in a worker under certain configurations when using HTTP/3 with OpenSSL 3.5.0 and earlier). Change: QUIC transport parameters extension received in an SSL connection is now always ignored. Bugfixes: binary upgrade with control API socket + perl module; predicate-location evaluation error ignored; nested location lookup with regex/predicate locations; segfault reading config with `geo` … `ranges` + corrupted binary base file. None affect the custom-html static-file-serving use case (no HTTP/3/QUIC, no predicate locations, no perl, no `geo`).
 - Breaking change in nginx 1.31.0: HTTP/2 and HTTP/3 requests with Connection/Proxy-Connection/Keep-Alive/Transfer-Encoding/Upgrade/TE headers are now rejected.
 - Breaking change in nginx 1.29.7: keepalive in upstream block is enabled by default; proxy_http_version changed to 1.1; Connection proxy header no longer sent by default.
 - nginx 1.31.3 (15 Jul 2026) is a security patch release: CVE-2026-42533 (heap buffer overflow in `map` directive with regex matching / non-cacheable variable in string expression), CVE-2026-60005 (uninitialized memory access with unnamed regex captures + `slice` directive / background cache update → memory disclosure or worker crash), CVE-2026-56434 (use-after-free in `ngx_http_ssi_filter_module` processing crafted proxied backend response). Two non-security `Change`s: HTTP/2 response header/trailer size now limited by `proxy_buffer_size`/`grpc_buffer_size` in `ngx_http_proxy_v2_module`+`ngx_http_grpc_module`; external-entity loading disabled by default in `ngx_http_xslt_filter_module` (new `xml_external_entities` directive re-enables it). Plus bugfixes (HTTP/2 out-of-order pseudo-headers, flow control, Upgrade header in HTTP/2/3 responses). NONE of these changes affect the custom-html static-file-serving use case (no `map`, no `slice`/cache, no SSI, no xslt, no proxy_v2/grpc). No breaking config changes — no operator action required beyond the image pull.
@@ -10,8 +10,27 @@ This is where CVE-2026-60004 + CVE-2026-59774 (both CVSS 9.8, fixed in 1.27.1) w
 and why the 2026-08-03/07 reports under-counted gitea's CVEs. advisory-scan.py fetches every URL
 in this file, so keep vendor security pages listed here.
 | db      | postgres | https://github.com/postgres/postgres | https://www.postgresql.org/docs/release/ |
+| db (mariadb overlay) | mariadb | https://github.com/MariaDB/server | https://mariadb.com/docs/release-notes/community-server/10.11 (series index — the CVE tables live on the PER-RELEASE pages, e.g. https://mariadb.com/docs/release-notes/community-server/10.11/10.11.19) |

 ## Standing notes
+- **mariadb overlay** (compose.mariadb.yml): pinned `mariadb:10.11.x` (10.11 LTS, maintained until
+  Feb 2028). ⚠ BOTH db overlays (postgres + mariadb) define the same service name `db`, and
+  `abra recipe upgrade -m` machine output keeps only ONE `db` entry — the postgres one (files are
+  processed alphabetically; the mariadb result is clobbered). ALWAYS check the mariadb pin against
+  Docker Hub directly; the weekly survey's single "db:" line hides it. Per-release security notes
+  (CVE tables — GitHub release bodies only link out): 
+  https://mariadb.com/docs/release-notes/community-server/10.11/10.11.3.md
+  https://mariadb.com/docs/release-notes/community-server/10.11/10.11.6.md
+  https://mariadb.com/docs/release-notes/community-server/10.11/10.11.8.md
+  https://mariadb.com/docs/release-notes/community-server/10.11/10.11.11.md
+  https://mariadb.com/docs/release-notes/community-server/10.11/10.11.12.md
+  https://mariadb.com/docs/release-notes/community-server/10.11/10.11.15.md
+  https://mariadb.com/docs/release-notes/community-server/10.11/10.11.16.md
+  https://mariadb.com/docs/release-notes/community-server/10.11/10.11.17.md
+  https://mariadb.com/docs/release-notes/community-server/10.11/10.11.18.md
+  https://mariadb.com/docs/release-notes/community-server/10.11/10.11.19.md
+  (the 10 CVE-bearing releases of the 10.11.2→10.11.19 window, 2026-09-11; append newer pages for
+  later windows).
 - **postgres major version**: recipe uses plain postgres (not pgautoupgrade); pg_backup.sh uses pg_dump/psql (logical backup, not pg_upgrade). Major version bump (e.g. 15→16) requires operator to manually restore from logical backup — no auto-migration tooling. Bump only within same major (e.g. 15.13→15.18) unless recipe adds pg_upgrade support.
 - **compose.postgres.yml**: postgres is an optional overlay (not in the main compose.yml). The cc-ci tests determine whether the postgres overlay is used. `abra recipe upgrade` reads it too and will propose a db bump — hold db unless doing a migration-aware major bump.
 - **1.25.0**: Removed deprecated auth sources — legacy auth setups may need reconfiguration.
@@ -7,5 +7,7 @@

 ## Standing notes
 - hedgedoc 1.11.0 (2026): 4 security CVEs fixed (HTML injection, YAML DoS, CSRF via Gist export, rate-limit bypass). No breaking changes, no migrations, no schema changes. Optional new env var `CMD_RATE_LIMIT_USING_CLOUDFLARE` only needed if running behind Cloudflare — not required for standard deployments.
- pgautoupgrade: handles Postgres major-version upgrades automatically on container start. Bump ONE major at a time (16→17, then 17→18 on next cycle). The image tag is `<pg-major>-alpine`.
- cc-ci tests use the sqlite backend (default compose.yml), not the postgresql compose override — so pgautoupgrade bumps do not affect CI test coverage.
+- pgautoupgrade: handles Postgres major-version upgrades automatically on container start; multi-major in ONE deploy works (16.15→18.6 verified live 2026-09-18 — pg_upgrade ran directly on the v16 files to 18.6, data survived). The image tag is `<pg-major>-alpine`.
+- pgautoupgrade 18+ (RESOLVED on PR #3, 2026-09-18): PG 18 changes the data-directory layout (docker-library/postgres#1259; PGDATA → `/var/lib/postgresql/<MAJOR>/docker`, `/var/lib/postgresql/data` becomes a symlink → mount error or EMPTY instance if the mount stays at .../data). The fix carried by PR #3: move the volume mount to `postgres:/var/lib/postgresql` (NO PGDATA env pin needed) — the entrypoint's `MOVING_TO_NEW_STRUCTURE` detects `/var/lib/postgresql/PG_VERSION`, upgrades, and moves data into `<MAJOR>/docker` automatically. Verified live on cc-ci 2026-09-18: `Upgrade to PostgreSQL 18.6 complete.`, pad data byte-identical, app green; 16→17 separately verified 2026-09-11 (`Upgrade to PostgreSQL 17.11 complete.`). Upstream coopcloud renovate PR #22 bumps the tag WITHOUT this mount fix — do not merge that alone.
+- hedgedoc 1.12.0 (2026-08): requires Node 20.17+ (app image bundles its own node — not recipe-relevant); webp uploads; some highlight.js languages removed/changed. No breaking changes or migrations for the recipe.
+- cc-ci tests use the sqlite backend (default compose.yml), not the postgresql compose override — so pgautoupgrade bumps do not affect CI test coverage (verify pg bumps via a direct --chaos deploy with the postgresql override instead).
@@ -50,6 +50,35 @@
  Immich is now **up-to-date** (v3.1.0 is latest stable per GitHub releases; no v3.1.x patch / v3.2.x).
  PR #4 now diverges from upstream main only on `database` (pgvectors0.3.0 vs 0.2.0) + `redis`
  (valkey:9@8e8d64b4 vs @3acc0687) pins — operator decides merge-vs-close.
+- **immich-server v3.2.0 (2026-09-10) pins** (from its `docker/docker-compose.yml` at the v3.2.0 tag):
+  `valkey:9@sha256:70739f85ad2ee01a726a965584a0f94895f01b0c60b3cc8b0aeef11eaa6888cf`
+  (= valkey **9.1.1** per image env; a NEWER build than v3.1.0's `8e8d64b4…` = 9.1.0 — but OLDER
+  than the live `9` tag, which by 2026-09-11 moved to `c123e371…` = 9.1.2; pin immich's tested
+  `70739f85…`, NOT the live tag, per the 2026-07-17 precedent) and
+  `postgres:14-vectorchord0.4.3-pgvectors0.2.0@sha256:bcf63357…` (SAME combo as v3.1.0 — DB
+  unchanged). No breaking server-side changes in v3.2.0; `example.env` byte-identical to v3.1.0;
+  TypeORM migrations auto-run. Operator-info only: the new opt-in cluster-groups feature requires
+  a per-group facial-recognition reset (user-triggered button, not a deploy migration). **ghcr
+  pagination gotcha (hit 2026-09-11):** `ghcr.io/v2/immich-app/postgres/tags/list` pages are ~100
+  tags, NOT purely lexicographic, and the `Link:` header returns a RELATIVE url (`/v2/...`) —
+  prefix the host or pagination silently stops at page 1; the pinned pg tag `14-vectorchord0.4.3-
+  pgvectors0.2.0` is on **page 2** of 174 (a first-page probe wrongly suggests it's gone). Upstream
+  coopcloud has open renovate PRs #18 (v3.2.0 monorepo bump = server+ML only), #17 (valkey digest
+  → live c123e37 — not immich's tested pin), #16 (postgres → pg16 — unsupported by immich v3.2.0,
+  reject). Done in the 2026-09-11 upgrade (v3.1.0→v3.2.0, extending PR #4 from upstream main
+  794560f; PR #4's pgvectors0.3.0/valkey-8e8d64b4 divergences superseded — DB realigned to
+  immich's official combo, which upstream main itself chose 2026-08-07).
+- **immich-server v3.2.1 (2026-09-14) + v3.2.2 (2026-09-15) pins** (from `docker/docker-compose.yml`
+  @ the v3.2.2 tag): `postgres:14-vectorchord0.4.3-pgvectors0.2.0@sha256:bcf63357…` and
+  `valkey:9@sha256:70739f85ad…` — **BOTH identical to the recipe's v3.2.0-era pins** (DB and
+  redis stay put; live valkey `9` tag has drifted to `c123e371…` = 9.1.2 but immich's tested
+  pin `70739f85…` = 9.1.1 is kept per the 2026-07-17 precedent; live postgres tag digest
+  re-verified = the pin, not stale). `example.env` @ v3.2.2 byte-identical to v3.2.0. Both
+  releases are bug-fix-only (v3.2.1: sync connection-pool exhaustion, search modal, partner
+  assets on people page, person merge, server auto-VACUUM after TypeORM migrations; v3.2.2:
+  reassign-faces fix) — no breaking changes, no config additions, no operator action. Done in
+  the 2026-09-18 upgrade (v3.2.0→v3.2.2, app+ML only; fresh PR — PR #4 was closed when its
+  v3.2.0 content merged upstream as #19 / `1.11.0+v3.2.0`).
 - **2026-08-07 INFRA note: cc-ci runner's gitea clone-token is STALE (HTTP 401).** `!testme` build
  #1210 died at the recipe `git clone` step (~5s, before any deploy) with `could not read Username for
  'https://git.autonomic.zone'` — the runner's mounted gitea token `13e299f2…` is rejected (verified
@@ -6,10 +6,10 @@
 | backend | lasuite/impress-backend | https://github.com/suitenumerique/docs | https://github.com/suitenumerique/docs/releases |
 | celery | lasuite/impress-backend | https://github.com/suitenumerique/docs | https://github.com/suitenumerique/docs/releases |
 | y-provider | lasuite/impress-y-provider | https://github.com/suitenumerique/docs | https://github.com/suitenumerique/docs/releases |
-| docspec | ghcr.io/docspecio/api | https://github.com/docspecIO/api | https://github.com/docspecIO/api/releases |
+| docspec | ghcr.io/docspec/api | https://github.com/docspec/docspec | https://github.com/docspec/docspec/releases |
 | db | pgautoupgrade/pgautoupgrade | https://github.com/pgautoupgrade/docker-pgautoupgrade | https://github.com/pgautoupgrade/docker-pgautoupgrade/releases |
 | redis | redis | https://github.com/redis/redis | https://hub.docker.com/_/redis/tags |
-| minio | minio/minio | https://github.com/minio/minio | https://github.com/minio/minio/releases |
+| minio | quay.io/minio/minio | https://github.com/minio/minio | https://github.com/minio/minio/releases |
 | web | nginx | https://github.com/nginx/nginx | https://nginx.org/en/CHANGES |

 ## Standing notes
@@ -36,3 +36,17 @@
 - redis sidecar (cache/broker for celery+backend): prefer the patch line (8.8.1 over 8.10.0) unless a
  feature is needed — 8.8.1 is a security patch (RedisBloom/TDigest RESTORE RCE); 8.10.0 is a minor
  with many new features.
+- **2026-09-18: docspec image switched with impress v5.7.0** (upstream PR suitenumerique/docs#2553):
+  `ghcr.io/docspecio/api:3.0.x` (Elixir, repo ARCHIVED) → `ghcr.io/docspec/api:1.21.4` (Rust rewrite,
+  github.com/docspec/docspec). New image listens on **3000** (was 4000): recipe sets
+  `DOCSPEC_API_URL=http://docspec:3000/conversion` and healthcheck `http://127.0.0.1:3000/health`
+  (path `/conversion` and the raw-body request contract unchanged).
+- ⚠️ When checking GHCR tags by script: the `/tags/list` endpoint CAPS AT 100 tags — a missing tag
+  there means nothing. Verify the **manifest** endpoint instead (e.g. `ghcr.io/docspec/api:1.21.4`
+  is absent from tags/list but pulls fine).
+- **2026-09-13ish: upstream #25 moved the minio pin to `quay.io/minio/minio`** (same
+  RELEASE.2025-09-07T16-13-09Z tag — manifest-verified pullable on quay; quay's own tags/list is
+  paginated and misleading, use manifest checks there too).
+- **v5.7.0 moved the favorites API endpoint to `/documents/favorites/`** — breaking only for
+  external API consumers, not the recipe/cc-ci tests (which use /authenticate/, /users/me/,
+  /documents/).
@@ -9,8 +9,8 @@
 | db | pgautoupgrade/pgautoupgrade | https://github.com/pgautoupgrade/docker-pgautoupgrade | https://github.com/pgautoupgrade/docker-pgautoupgrade/releases |
 | redis | redis | https://github.com/redis/redis | https://github.com/redis/redis/releases |
 | mailcatcher | sj26/mailcatcher | https://github.com/sj26/mailcatcher | https://github.com/sj26/mailcatcher/releases |
-| minio | minio/minio | https://github.com/minio/minio | https://github.com/minio/minio/releases |
-| minio-createbuckets | minio/minio | https://github.com/minio/minio | https://github.com/minio/minio/releases |
+| minio | quay.io/minio/minio | https://github.com/minio/minio | https://github.com/minio/minio/releases |
+| minio-createbuckets | quay.io/minio/minio | https://github.com/minio/minio | https://github.com/minio/minio/releases |
 | collabora | collabora/code | https://github.com/CollaboraOnline/online | https://www.collaboraonline.com/release-notes/ (per-version: .../collabora-online-25-04-release-notes/) |
 | onlyoffice | onlyoffice/documentserver-de | https://github.com/ONLYOFFICE/DocumentServer | https://github.com/ONLYOFFICE/DocumentServer/blob/master/CHANGELOG.md |
 | web | nginx | https://github.com/nginx/nginx | https://nginx.org/en/CHANGES |
@@ -19,9 +19,12 @@
 - lasuite/drive-frontend and lasuite/drive-backend share the same version tag (drive monorepo).
 - minio and onlyoffice tags use non-semver formats; abra cannot auto-detect upgrades for them.
 - collabora/code uses a 5-part version scheme; abra cannot auto-detect upgrades for it.
- minio: Docker Hub `minio/minio` `latest` is frozen at `RELEASE.2025-09-07T16-13-09Z` (the
-  `RELEASE.2025-10-15T17-29-55Z` GitHub security release is NOT on Docker Hub `minio/minio`).
-  Newer minio releases moved to the **AIStor** product line (`quay.io/minio/aistor/minio`).
-  A future minio bump requires a registry+product switch — operator decision, not a routine tag bump.
+- minio: Docker Hub `minio/minio` was **REMOVED** (verified 2026-09-18: Hub repo API 404, registry
+  manifest 401 on anonymous pull; last successful docker.io pull was the 2026-09-11 run). The same
+  release tag lives on **`quay.io/minio/minio`** (MinIO's official distribution channel) — recipe
+  pins switched there in lasuite-drive PR #7 (2026-09-18), same release `RELEASE.2025-09-07T16-13-09Z`,
+  mirroring the identical fix merged upstream for lasuite-docs ("fix: pull MinIO image from Quay",
+  PR #25, 2026-09-15). A future minio **version** bump still requires the AIStor product-line
+  decision — quay.io only re-hosts the frozen community line.
 - nginx 1.31.x (1.31.0, 1.31.1) contains multiple security CVE fixes; upgrade is recommended.
 - nginx 1.31.0 breaking change: HTTP/2 and HTTP/3 requests with Connection/Proxy-Connection/Keep-Alive/Transfer-Encoding/Upgrade headers are now rejected (affects proxied HTTP/2 backends).
@@ -6,6 +6,37 @@
 | postgres | postgres                                  | https://github.com/postgres/postgres              | https://www.postgresql.org/docs/release/                          |

 ## Standing notes
+  - **2026-09-18 re-check** (Mattermost release-policy docs; GitHub release; Docker Hub; vendor
+    security feed): **11.7 ESR line UNCHANGED** — newest patch **11.7.11** (released 2026-09-15,
+    "Low to High severity level security fixes"; Docker Hub tag pushed 2026-09-15). v10.11 ESR
+    remains EXPIRED (2026-08-15); 10.11.24 sits on the dead line — do NOT target. This run
+    extended open PR #2 **11.7.10 → 11.7.11** (branch tip 5696f0b4). Direct `--chaos` deploy on
+    cc-ci exercised the full operator path (deploy at 10.11.22 → seed admin user via API →
+    upgrade to 11.7.11): schema migrations ran on boot, **seeded user data survived** (same
+    id/roles, login 200), ping 200 with x-version-id 11.7.11; `!testme` **GREEN**, drone 1373
+    (level 5: install/upgrade/backup/restore/custom all pass, clean teardown). Vendor feed: **6
+    server advisories fixed in 11.7.11** (MMSA-2026-00711 Low; -00761/-00763/-00764/-00765/-00773
+    Medium, all affecting 11.7.x <= 11.7.10; CVE ids pending 30-day disclosure, due 2026-10-15).
+    **CVE-2026-13426 re-adjudicated FIXED** for the 10.11.22 → 11.7.11 window (fresh GitHub
+    compare: fix commit 3321db82 is an ancestor of v11.7.11 — ahead/behind 0 — and diverged from
+    v10.11.22) → window CVE count **14** (13 deterministic + 1 adjudicated). postgres **15-alpine
+    HELD** (DB-major out of scope). Gotchas learned: `abra app new` RESETS the recipe checkout to
+    main (re-checkout the branch before a `--chaos` deploy), and `abra app new --domain` does NOT
+    write DOMAIN into the app .env (edit the .env + redeploy to fix traefik routing).
+  - **2026-09-11 re-check** (Mattermost Server Releases docs; endoflife.date/api/mattermost.json;
+    GitHub releases + module tags): **11.7 ESR line UNCHANGED** — newest patch still **11.7.10**
+    (2026-08-26; no 11.7.11; v11.11.0 is still a GitHub **prerelease**, innovation). **10.11 ESR
+    expired 2026-08-15; 10.11.23 (2026-08-13) was its FINAL patch** — the "10.x is LTS, never
+    11.x" survey hint appeared a THIRD time (2026-06-26, 2026-08-07, 2026-09-11) and was again
+    NOT followed: no supported 10.x LTS exists (10.12 = innovation, expired 2025-12-15). PR #2
+    re-verified unchanged (tree 59e8c2c0; direct `--chaos` deploy converged + ping 200;
+    `!testme` GREEN, drone 1355). **Adjudicated CVE-2026-13426 (MMSA-2025-00532, medium) FIXED by
+    the 10.11.22 → 11.7.10 window**: module fix `server/public` v0.1.22 (commit 3321db82, tag cut
+    2026-01-30 from the 11.4.0 cycle) is an ancestor of v11.7.10 but NOT of v10.11.22/v10.11.23
+    (GitHub compare evidence) → CVE count for the ESR move = **14** (13 deterministic + 1
+    adjudicated). postgres **15-alpine HELD** (recipe has no compose overlays beyond compose.yml;
+    no pg_upgrade/pgautoupgrade support in recipe or cc-ci tests; the floating 15-alpine tag picks
+    up 15.x patches on re-pull — pg 15.19 fixes CVE-2026-14662/14663/14664).
  - **2026-08-28 re-check** (endoflife.date/api/mattermost.json 2026-08-28; Docker Hub; GitHub
    releases): **11.7.10** (released 2026-08-26, "various bug fixes", not prerelease) is the newest
    11.7.x ESR patch — the ESR/LTS line has NOT moved, still **11.7** (EOL **2027-05-15**). This run
@@ -171,3 +171,71 @@
  flag (unchanged from 2.37.0): the two API behavior changes only affect HTTP API callers. Recommended
  release: `-y` (feature minor across 2.34.4 → 2.38.4). CVE-2026-73088 was adjudicated NOT-FIXED in the
  2.x window by the 2026-09-04 parent run; not blocking.
+- 2.38.5 (2026-09-09, patch): 1 core fix (keep reporting the original job error when the job key is
+  gone).
+- 2.38.6 (2026-09-10, patch; **Stable/Latest badge**): 1 perf fix (bound peak memory during source
+  control push).
+- 2.39.0 (2026-09-08, Pre-release): the 2.39 feature minor — large feature/bugfix release. Features:
+  source-control push/pull + status endpoints in the public API, **workflow version endpoint
+  (deprecates the old two-variable path — API-level deprecation)**, Instance reporting module, opt-in
+  concurrency limits for Instance AI runs, OTLP gRPC support for OTel, Confluence/Jira Atlassian
+  Service Account (2LO) auth, MS Teams channel-message + Online Meeting ops, Databricks partner UA +
+  hidden chat model, Instance AI folder exploration / past-conversation search / credential-selection
+  guidance, **encryption-key module rework** (always load the encryption module and seed keys safely
+  on startup; cache keys in memory; pluggable write path with rotation moved into the module; signing
+  deployment keys stored in the same wrapped form as data-encryption keys), deprecate + hide the Ask
+  AI tab in the Code node. Many core/editor/node fixes (webhook Set-Cookie response headers; OTel
+  header redaction in settings; Azure OpenAI classic/Foundry credential fix; OIDC failed-login
+  redirect; Qdrant client update for Node 26; AMQP trigger receiver reattach; Anthropic disabled
+  thinking; etc.). No breaking compose/env/migration changes; no `N8N_*` env renames.
+- 2.39.1 (2026-09-09, Pre-release): 1 core fix — **repair legacy-format data-encryption keys during
+  bootstrap** (upgrade-relevant: old-format keys are repaired, not rejected).
+- 2.39.2 (2026-09-10, Pre-release): 2 core fixes (memoize pairedItem ancestry traversal; reuse an
+  existing consent grant after the visitor authenticates mid-flow) + 1 perf (stop copying user
+  workflows into the Instance AI sandbox). Docker Hub `2.39.2` manifest verified multi-arch
+  (amd64 + arm64, active, 2026-09-10).
+- 2026-09-11 run: survey framed 2.34.4 → 2.39.2, but the morning reconcile closed PR #7 as
+  merged-upstream and upstream main released **3.5.0+2.38.4** — so the effective compose window is
+  **2.38.4 → 2.39.2** on a FRESH branch (no open upgrade PR to extend). 2.38.6 holds the
+  Stable/Latest badge; 2.39.x is Pre-release (consistent tracking-the-newest-tag precedent:
+  2.34.2/2.35.2/2.36.3/2.37.3/2.37.6/2.38.4 were all Pre-release when taken). Rolling upgrade safe
+  (sqlite mode, TypeORM auto-migrate on boot). Live-deploy watch item: the 2.39.0 encryption-key
+  module rework + 2.39.1 legacy-key repair (recipe injects N8N_ENCRYPTION_KEY as a secret).
+  Operator flags, all HTTP-API-caller-level only: 2.33.0 workflow activate/deactivate deprecation;
+  2.36.0 Array.merge→mergeIntoObject + workflow-tags API migration; 2.37.0 "Any workflow" caller-policy
+  deprecation + JSON content-type on decorator body routes + binary-data endpoint adapt; 2.39.0
+  workflow-version endpoint deprecation. Recommended release: `-y` (feature minor).
+- 2.38.7 (2026-09-11, patch): 3 core fixes (MCP toolkit members on workers; consent-grant reuse;
+  trust the CA from GIT_SSL_CAINFO for source-control HTTPS remotes).
+- 2.39.3 (2026-09-11, Pre-release): 1 core fix (GIT_SSL_CAINFO trust — 2.38.7 backport).
+- 2.39.4 (2026-09-11, Pre-release): 1 core fix (sandbox npm install flags by provider).
+- 2.39.5 (2026-09-14, stable line): editor fix (revoke end-user credentials) + feature
+  (instance-report log streaming events).
+- 2.39.6 (2026-09-16, stable line): core fix (Gateway credits eligibility) + features (**v3
+  breaking-change rule for the storage-directory rename**; **warn on deprecated N8N_DB_PING_TIMEOUT
+  env var** — recipe does not set it).
+- 2.39.7 (2026-09-17, stable line; **`stable` tag points here** — the stable badge moved to the
+  2.39.x line): core breaking-change-detection resilience; editor numeric credential inputs.
+- 2.40.0 (2026-09-15, Pre-release, feature minor — large): Microsoft Dataverse core node; AI Agent
+  forced first-iteration tool calls; Git-based promotion model (selective promote, timestamped
+  branches, promotion entities/DTOs); **workflow publication service enabled by default**;
+  ephemeral sandboxes; agent episodic memory; ai_preference table; HTTP Request
+  PROPFIND/MKCOL/MOVE/COPY/REPORT; Grist OAuth2; MS Teams online-meeting/channel-message ops; v3
+  migration-report rules (storage-directory rename, preExecute) + "chat hub goes away" warning;
+  N8N_DB_PING_TIMEOUT deprecation warning; secret-redaction consolidation; queue-mode execution
+  timeout for stuck jobs; ~150 core/editor/node bugfixes. **No breaking compose/env/migration
+  changes; no N8N_* renames.**
+- 2.40.1 (2026-09-16, Pre-release): GitHub release body EMPTY (maintenance republish, no listed
+  changes).
+- 2.40.2 (2026-09-17, Pre-release): core breaking-change-detection resilience + editor numeric
+  credential inputs (the 2.39.7 pair on the 2.40 line). Docker Hub manifest verified active.
+- 2.40.3 (2026-09-18, Pre-release): core "repair data-encryption keys stored as the raw instance
+  key" + editor blank-workflow-preview fix. Published the morning of the 2026-09-18 run — newer
+  than the operator's snapshot (which named 2.40.2 as newest); NOT taken that run (operator's
+  explicit target 2.40.2), flagged in the plan/PR as a trivial follow-up bump.
+- 2026-09-18 run: PR #8 (branch upgrade-ef0dd56, at 2.39.2 since 2026-09-11, !testme GREEN then)
+  extended **2.38.4 → 2.40.2** on upstream main tip 0b436ec (still 3.5.0+2.38.4 — upstream has not
+  started the bump). Stable badge on 2.39.7; 2.40.x pre-release per precedent. Rolling upgrade
+  safe (sqlite, TypeORM auto-migrate). Operator flags unchanged (API-caller-level deprecations,
+  encryption-key rework informational, N8N_DB_PING_TIMEOUT warn-only, recipe doesn't set it).
+  2.40.3 exists (see above) — flagged, not taken. Recommended release: `-y`.