ssh-keys: drop claude@claude-vm, relabel sandbox key cc-ci-root-ed25519, add notplants-orchestrator pub

# Conflicts:
#	cc-ci-plan/JOURNAL.md

cc-ci-plan/JOURNAL.md

@@ -1238,3 +1238,76 @@ the host: `opencode-go/deepseek-v4-flash` and `opencode-go/glm-5.3-flash` answer
   `upgrader.env` (`LOOP_TIER=go` maps to the `opencode-go` auth entry; `LOOP_MODEL` overrides the
   tier default). Next fire Fri 2026-09-11 02:00 UTC.
 - The steering orchestrator agent stays on `opencode-go/glm-5.2` (not asked to change).
+
+## Session 2026-09-21 — domain cutover to ci.autonomic.zone (orchestrator)
+
+**What happened.** Morning: hourly supervisor resumed the stalled 09-18 weekly run on the GO tier
+(ZEN endpoint dead server-side — `UnknownError`; run completed 13 green PRs, 0 failed). Published
+the missing week-2026-09-18 report (PR-finding #4; launcher defaults flipped to `go` in
+cc-ci-orchestrator PR #23). Then executed the full domain cutover per
+`cc-ci-plan/plan-domain-migration-ci-autonomic-zone.md` (PR #25).
+
+**Plan deviation (simplification).** No dual-cert SNI: ONE Let's Encrypt cert carries SANs for
+BOTH zones (`ci` + `*.ci` of autonomic.zone AND commoninternet.net) — the unchanged single-pair
+`ssl_cert/ssl_key` traefik reconciler keeps working; Phase 4 reissues without the legacy SANs.
+The new zone's DNS-01 challenge reuses the SAME acme-dns account: storage re-keyed by
+`cc-ci-acme-storage-seed.service` (jq clone of the legacy entry under `ci.autonomic.zone`),
+CNAME already delegated. Proven by a hand lego **staging** run before any production change.
+
+**Merged:** cc-ci #39 (front doors dual Host rules, bridge/dashboard env URLs, drone abra rename,
+runner RPC, naming.py → `*.ci.autonomic.zone`, dual-zone name regexes in lifecycle/warm/prune,
+recipe-report URLs) · cc-ci #40 (seed-unit nesting fix) · cc-ci #41 (have_secret stack-scope —
+caught live: the old stack's `*_rpc_secret_v1` satisfied the check post-rename) · cc-ci #42 (nix
+interpolation escape) · orchestrator #27 (oc.ci host + `opencodeUiExtraHosts`, host self-pins,
+flake bump).
+
+**Deployed** via `nixos-rebuild test` → verify → `switch` (generation `nn1vwiv7v1k…`, running ==
+boot). Cert SANs confirmed 4-name; all 5 front doors answer on BOTH zones
+(200/200/303/401 + traefik 200), TLS verify=0 from outside; zero failed units; disk dropped
+88%→45% after prune.
+
+**Drone migration.** New abra app `drone.ci.autonomic.zone`, FRESH DB (module's
+`DRONE_USER_CREATE` re-injected the sops bridge token). The Gitea OAuth app redirect now has both
+URIs; the client secret was rotated (each Gitea PATCH regenerates it) and synced through
+sops → `sops-install-secrets` → swarm secret v1 → drone. Bootstrapped OAuth
+(`drone login ok (admin=true)`), re-enabled cc-ci + discourse repos, build timeout 60m.
+Webhooks: ghost + discourse repointed (secrets preserved); cc-ci repo's bridge webhook →
+`ci.autonomic.zone/hook`, stale drone hook deleted, Drone's auto-created new-zone hook active.
+Old stack removed + orphaned secrets reaped.
+
+**E2E proof.** `!testme` on keycloak PR #9 → bridge → drone build #1 (new DB numbering) → runner →
+harness → `results.json` + PR card `✅ passed` linking `ci.autonomic.zone/runs/1/summary.png`.
+
+**Deferred / open.**
+- Warm stacks + backupbot stay on the legacy zone (data-warm volumes / restic password tied to
+  abra app names) — post-bake migration; harness regexes accept both zones meanwhile.
+- Docs sweep (~60 references: AGENTS.md, README, skills incl. cc-ci-status front-door list,
+  launcher printed URLs).
+- Phase 4 (after ≥7 clean days): drop legacy SANs (reissue), remove legacy Host arms + host
+  self-pins + Gandi records (`ci`, `*.ci`, `_acme-challenge.ci`), TTLs back to 3600.
+- Host auto-update was `failed` 2026-09-15 (health check) — `/cc-ci-orchestrator-update` still
+  pending; next auto-attempt Tue 09-22.
+
+## Session 2026-09-28 20:00 UTC — operator-broken cc-ci recovered by plain hard reset
+
+- Operator reported ci.autonomic.zone down after their own change, supplied a Hetzner API token
+  in chat (token is now in the transcript — SHOULD BE ROTATED). Staged at /tmp/opencode/hcloud-token
+  (0600) instead of echoing it.
+- Triage: SSH (port 22) timed out, ICMP 100% loss, tailscale 100.95.31.88 no reply — yet Hetzner
+  reported "running". Old recovery note's server id 134485294 is GONE; current cc-ci is id
+  165014541, public 195.201.88.249 (token project also holds 114514766 autonomic-cc-testing).
+  Last Hetzner action was 2026-09-07 (rescue cycles during the rebuild), so the outage was
+  OS-internal, not API-driven.
+- Fix: single hard reset via `POST /servers/165014541/actions/reset`. ICMP after ~60s, SSH after
+  ~90s. Box booted the default profile nixos-system-cc-ci-26.05.20260906.c257840 — no rescue/
+  GRUB generation-picking needed this time.
+- Post-checks: nginx + gitea active, drone-runner-exec active (NOT drone-runner-docker — wrong
+  guess), disk 41%, https://ci.autonomic.zone → 200. One failed unit:
+  acme-order-renew-ci.autonomic.zone.service — renewal itself fine (cert valid to 2026-12-20),
+  it died on `chmod: out/acme-dns-accounts.json: Operation not permitted` because the file was
+  root:root (touched today 19:54, likely by whatever the operator did) while the unit runs as
+  acme. chown acme:acme (matching the healthy ci.commoninternet.net dir) + restart → unit green,
+  zero failed units.
+- NOTE: no tailscale on this host (`tailscale: command not found`) — the AGENTS.md "ssh cc-ci"
+  alias + 100.90.116.4 peer notes are stale post-rebuild; public-IP SSH is the access path.
+  Recovery scripts in scripts/recovery/ still reference old server id 134485294 — worth updating.

cc-ci-plan/recipe-report.py

@@ -303,7 +303,7 @@ def render(spec_path, out_path):
                        for p in lead.split("\n\n") if p.strip())
     body = (_mast() +
             f'<div class="dateline"><span>{_esc(sub)}</span>'
-            f'<span>report.ci.commoninternet.net</span><span>{gen}</span></div>'
+            f'<span>report.ci.autonomic.zone</span><span>{gen}</span></div>'
             f'<div class="lead">{lead}</div>')
     # 1) the full wire — every recipe, in the agent's recommended priority order (CVEs first); CVEs column.
     wire = ("The full wire — every recipe, in priority order" if kind == "week"
@@ -322,7 +322,7 @@ def render(spec_path, out_path):
     if s.get("changes"):
         body += f'<h2>What changed</h2>{_changes(s.get("changes"), repo_url)}'
     body += (f'<footer>{title} · generated {gen} · '
-             f'<a href="https://ci.commoninternet.net/">dashboard</a> · <a href="./">archive</a></footer>')
+             f'<a href="https://ci.autonomic.zone/">dashboard</a> · <a href="./">archive</a></footer>')
     open(out_path, "w").write(_page(f"{title} · " + s["date"], body))
     print("wrote", out_path)
 
@@ -347,10 +347,10 @@ def publish(html_path, date, kind="week"):
         for d, k in sorted(set(entries), reverse=True))
     idx = _page("The Recipe Report — Archive", _mast() +
                 '<div class="dateline"><span>Weekly review of Co-op Cloud recipe upgrades &amp; CI</span>'
-                '<span>report.ci.commoninternet.net</span></div>'
+                '<span>report.ci.autonomic.zone</span></div>'
                 f'<ul class="idx">{lis or "<li><em>No reports yet.</em></li>"}</ul>')
     subprocess.run(["ssh", "cc-ci", f"cat > {HOST_REPORTS}/index.html"], input=idx.encode(), check=True)
-    print(f"published https://report.ci.commoninternet.net/{page}  (+ index)")
+    print(f"published https://report.ci.autonomic.zone/{page}  (+ index)")
 
 
 def main():

cc-ci-plan/upstream/mattermost-lts.md

@@ -134,6 +134,18 @@
     2026-08-15 (upstream main still pins 10.11.22 = EXPIRED ESR → the 10→11 ESR move PR #2 carries
     remains required; Mattermost docs: ESR→ESR is "fully supported and tested"). postgres 15-alpine
     still HELD (DB-major out of scope, operator dump/pg_upgrade).
+  - **2026-09-04 re-check** (endoflife.date/api/mattermost.json 2026-09-04; Mattermost release-policy
+    docs `https://docs.mattermost.com/product-overview/release-policy.html`; `mattermost-server-releases.html`;
+    GitHub releases `v11.7.10`): **11.7 ESR is STILL the current supported ESR/LTS line** — "v11.7 &
+    Desktop App v6.2 Extended Support: 2026-05-15 → 2027-05-15" (the chart on the release-policy page;
+    ESR cadence = every 9 months, supported 12 months). Latest 11.7.x patch **11.7.10** (2026-08-26,
+    "Mattermost Platform Extended Support Release 11.7.10 contains various bug fixes") — NOT a
+    prerelease; target confirmed. 11.8/11.9/11.10 remain Feature/innovation releases (EOL 2026-09-15 /
+    10-15 / 11-15, `lts:false`), NOT ESR — do NOT target; wait for the NEXT official ESR (expected
+    ~Feb 2027 on the 9-month cadence). No newer 11.7.x ESR patch exists as of this week, so PR #2's
+    head (`59e8c2c`, app image `11.7.10`) is still the correct target → this run RE-VERIFIES PR #2
+    (no new app bump). 11.11.0-rc1/rc2 seen on GitHub but innovation + pre-release — not a target.
+    postgres 15-alpine still HELD (DB-major out of scope, operator dump/pg_upgrade).
 
 ## NVD CPE fallback
 This project publishes nothing machine-readable we can reach — no GitHub advisory feed,

flake.lock

@@ -10,11 +10,11 @@
         ]
       },
       "locked": {
-        "lastModified": 1789404337,
+        "lastModified": 1790010051,
-        "narHash": "sha256-FY5oOz/C6i6Ct6Qe6DqN6nOq+TXCyiZq5Am1LKLpDss=",
+        "narHash": "sha256-0dz8fg4zqg54tk+1A7CHZAw+lK9vWxre6Tioi5B0ocU=",
         "ref": "refs/heads/main",
-        "rev": "eb5fb826114f2b141dffb270fd36f7a22d1bf343",
+        "rev": "2f6787f0433e7ff3053116039f10c00cde34128f",
-        "revCount": 1550,
+        "revCount": 1558,
         "type": "git",
         "url": "https://git.autonomic.zone/recipe-maintainers/cc-ci.git"
       },

nix/hosts/cc-ci/configuration.nix

@@ -30,8 +30,10 @@
   # Weekly self-update (Tue 03:00 UTC; skips itself while CI is busy; see nix/modules/auto-update.nix).
   cc-ci-orchestrator.autoUpdate.enable = true;
 
-  # The opencode UI: traefik (public 443, the *.ci.commoninternet.net cert) → nginx basic auth.
+  # The opencode UI: traefik (public 443, the dual-zone wildcard cert) → nginx basic auth.
-  cc-ci-orchestrator.opencodeUiHost = "oc.ci.commoninternet.net";
+  # Domain cutover 2026-09: new name primary; legacy name kept answering during the bake window.
+  cc-ci-orchestrator.opencodeUiHost = "oc.ci.autonomic.zone";
+  cc-ci-orchestrator.opencodeUiExtraHosts = [ "oc.ci.commoninternet.net" ];
   cc-ci-orchestrator.opencodeUiTraefikNetwork = "proxy";
 
   # ---- no tailscale on this host (operator 2026-09-07) --------------------------------------
@@ -124,7 +126,13 @@
   # This host's own public names resolve to itself regardless of external DNS state (host
   # processes: the drone runner, the harness, the orchestrator; containers use the resolvers
   # above). Per-run recipe domains are random and cannot be pinned — those follow public DNS.
+  # Domain cutover 2026-09: new names pinned; legacy names kept during the bake window.
   networking.hosts."195.201.88.249" = [
+    "ci.autonomic.zone"
+    "drone.ci.autonomic.zone"
+    "report.ci.autonomic.zone"
+    "traefik.ci.autonomic.zone"
+    "oc.ci.autonomic.zone"
     "ci.commoninternet.net"
     "drone.ci.commoninternet.net"
     "report.ci.commoninternet.net"

nix/hosts/cc-ci/ssh-keys

@@ -6,6 +6,6 @@ ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJVlfoLBPseQ9fA9534KmRg2KWcksKZGzAJIpHJ2JpsI
 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAQFuqUB2qNZSDNjDsjjhVA/WnnQNVAMmsUscW6OgMDN
 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHOcLo0YBa0UYi7i/l8K/Y/7cF2OclmDqSTlAsHM0dOS notplants-orchestrator
 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMniNzAzuI527bfk/EipqFILFayUCwYXDoZ3R7+QgYq6
-ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOk8NaeBdPbS2gfUvbny8h0AkZlVjGYHzx4QPXSJ38gd claude@claude-vm
+ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAcyTGb/wVgdhg5oBCZZvBaR1RuUQRY/3WHnOQpNDCsp cc-ci-root-ed25519@cc-ci-orchestrator-sandbox
-ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAcyTGb/wVgdhg5oBCZZvBaR1RuUQRY/3WHnOQpNDCsp claude-cc-ci-sandbox@20260526
 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKmGDZC6wrOQNJAW5PPDpxgEXXrcsnIU4b3QJLtq05RQ cc-ci-loops-to-root@cc-ci
+ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHIa8iunWtA4mqLKV6MuiTo3RkVj2ucsk3gLL4ArMEPO notplants-orchestrator

nix/modules/orchestrator-host.nix

@@ -62,6 +62,16 @@ in
       description = "nginx server_name for the opencode web UI (TLS + basic auth).";
     };
 
+    opencodeUiExtraHosts = lib.mkOption {
+      type = lib.types.listOf lib.types.str;
+      default = [ ];
+      description = ''
+        Extra hostnames routed to the opencode UI alongside opencodeUiHost, rendered as
+        `||`-joined Host conditions on the same router. Domain cutover 2026-09: the
+        legacy oc.ci.commoninternet.net keeps answering here during the bake window.
+      '';
+    };
+
     opencodeUiHtpasswdFile = lib.mkOption {
       type = lib.types.str;
       default = "/secrets/nginx/oc-htpasswd";
@@ -216,6 +226,7 @@ SSHCFG
       recommendedProxySettings = true;
       virtualHosts.${cfg.opencodeUiHost} = {
         listen = [ { addr = "0.0.0.0"; port = cfg.opencodeUiBackendPort; } ];
+        serverAliases = cfg.opencodeUiExtraHosts;
         basicAuthFile = cfg.opencodeUiHtpasswdFile;
         extraConfig = ''
           # traefik sits on the docker networks (ingress 10.0.0.0/24, gwbridge 172.18.0.0/16)
@@ -256,7 +267,7 @@ SSHCFG
                 replicas: 1
                 labels:
                   - "traefik.enable=true"
-                  - "traefik.http.routers.opencode-ui.rule=Host(`${cfg.opencodeUiHost}`)"
+                  - "traefik.http.routers.opencode-ui.rule=${lib.concatStringsSep " || " (map (h: "Host(`${h}`)") ([ cfg.opencodeUiHost ] ++ cfg.opencodeUiExtraHosts))}"
                   - "traefik.http.routers.opencode-ui.entrypoints=web-secure"
                   - "traefik.http.routers.opencode-ui.tls=true"
                   - "traefik.http.services.opencode-ui.loadbalancer.server.port=${toString cfg.opencodeUiBackendPort}"