recipe-maintainers/cc-ci-orchestrator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

opencode UI stack is ccci-opencode-ui, on the weekly sweep's keep-list; journal: first weekly run on the new host

Browse Source 
The 2026-09-07 weekly run's step-0 orphan sweep removed the `opencode-ui`
swarm stack (the traefik route to the opencode web UI) because it was not
on sweep-orphans.sh's keep-list. Renamed to ccci-opencode-ui alongside the
other control-plane stacks and added to KEEP_RE.

Journal: the run itself (lasuite-docs #8, n8n #7 GREEN on the new Drone,
report week-2026-09-07 published) ran entirely on the new host.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqkQq3CDmFWcQ7u1LzoyRz
This commit is contained in:
notplants
2026-09-07 22:13:43 +00:00
co-authored by Claude Fable 5.1
parent be9687962c
commit 48226dd78f
4 changed files with 20 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude/skills/upgrade-all/sweep-orphans.sh
+4 -2
View File
@@ -9,14 +9,16 @@
# SAFE BY ALLOWLIST. It removes ONLY things NOT on the keep-list, so it can never take down infra or # SAFE BY ALLOWLIST. It removes ONLY things NOT on the keep-list, so it can never take down infra or
# the warm canonicals. The keep-list (leading name prefix) is: # the warm canonicals. The keep-list (leading name prefix) is:
# - traefik, drone, backups : Swarm + CI infra # - traefik, drone, backups : Swarm + CI infra
# - ccci-bridge / -dashboard / -reports: the cc-ci control plane # - ccci-bridge / -dashboard / -reports / -opencode-ui: the cc-ci control plane (opencode-ui is
# the traefik route to the orchestrator's opencode web UI, deployed by the host's nix config —
# the 2026-09-07 run reaped it as an "orphan" before it was on this list)
# - warm-* : warm canonicals (idle persistent deps reused across runs; # - warm-* : warm canonicals (idle persistent deps reused across runs;
# their retained volumes are spared too) # their retained volumes are spared too)
# Everything else deployed on the Swarm is a per-run test stack and is fair game. # Everything else deployed on the Swarm is a per-run test stack and is fair game.
set -uo pipefail set -uo pipefail
export PATH=/run/current-system/sw/bin:$PATH export PATH=/run/current-system/sw/bin:$PATH
KEEP_RE='^(traefik|drone|backups|ccci-(bridge|dashboard|reports)|warm-)' KEEP_RE='^(traefik|drone|backups|ccci-(bridge|dashboard|reports|opencode-ui)|warm-)'
removed=0 removed=0
echo "== orphan sweep: scanning (keep-list: infra + warm-* canonicals) ==" echo "== orphan sweep: scanning (keep-list: infra + warm-* canonicals) =="
README.md
+1 -1
View File
@@ -160,7 +160,7 @@ Everything in this section is **outside git**. Do it as root over SSH, in this o
The combined host is NOT on the tailnet (operator decision 2026-09-07): ssh is key-only on the The combined host is NOT on the tailnet (operator decision 2026-09-07): ssh is key-only on the
public IP, the CI front doors are public via traefik, and the opencode UI is public on the same public IP, the CI front doors are public via traefik, and the opencode UI is public on the same
443 — traefik routes `oc.ci.commoninternet.net` to an nginx vhost on the docker bridge that 443 — traefik routes `oc.ci.commoninternet.net` to an nginx vhost on the docker bridge that
enforces basic auth, via traefik labels on a tiny swarm relay stack (`opencode-ui`) that enforces basic auth, via traefik labels on a tiny swarm relay stack (`ccci-opencode-ui`, on the weekly sweep's keep-list) that
`opencode-ui-route.service` deploys. (Not via traefik's file *directory* provider: enabling `opencode-ui-route.service` deploys. (Not via traefik's file *directory* provider: enabling
that in the coop-cloud traefik recipe REPLACES its provider file, which holds the wildcard cert that in the coop-cloud traefik recipe REPLACES its provider file, which holds the wildcard cert
and the `security` middleware, and every front door goes down — it did, for two minutes, on and the `security` middleware, and every front door goes down — it did, for two minutes, on
cc-ci-plan/JOURNAL.md
+10 -29
View File
@@ -1062,33 +1062,14 @@ Plan + live log: `cc-ci-plan/plan-cc-ci-combined-host.md` (on the branch; copy h
- Open: tailscale auth key revoked (`invalid key: API key does not exist`) → operator issues a - Open: tailscale auth key revoked (`invalid key: API key does not exist`) → operator issues a
new one. DNS cutover at Gandi (ci, *.ci, ns-acme → 195.201.88.249) → operator. new one. DNS cutover at Gandi (ci, *.ci, ns-acme → 195.201.88.249) → operator.
## Session 2026-09-07 21:10 UTC — CUTOVER DONE; orchestrator now on the new host ## 2026-09-07 22:15 UTC — first weekly upgrade run on the new host: GREEN, report published
CI is served from 195.201.88.249 (DNS repointed by the operator, no tailscale on the new box, Started by hand 21:23 UTC (`systemctl start cc-ci-upgrade-all` on 195.201.88.249, opencode /
opencode UI on 443 via traefik + basic auth, fail2ban on). `!testme` → Drone build 1326 on the deepseek-v4-flash); `UPGRADE RUN COMPLETE` 22:02 (39 min). Everything ran on the new host — old
new box. This session (on the old shared host) is the last one here: cc-ci units runtime-masked server's Drone/bridge at 0/0, no new run dirs or report there. 20 recipes surveyed, 2 upgrade PRs
on the old host; a fresh orchestrator session is up on the new box (`ssh root@195.201.88.249`, extended and `!testme` GREEN on the new Drone (lasuite-docs #8 → v5.6.1, build 1338; n8n #7 →
`sudo -iu loops tmux attach -t cc-ci-orchestrator`). Details + the two incidents: 2.38.4, build 1339), 1 PR closed as merged upstream (custom-html #7), 18 skipped as up-to-date or
`cc-ci-plan/plan-cc-ci-combined-host.md` (log) and README §2a/§7. Operator to merge covered. Summary: `.cc-ci-logs/upgrades/upgrade-all-2026-09-07.md`. Report agent published
notplants-nix `chore/drop-cc-ci`, delete the old CI server in a week, revoke the Hetzner token. https://report.ci.commoninternet.net/week-2026-09-07.html (200, 42 KB, indexed) at 22:11.
One side effect: the run's orphan sweep removed the `opencode-ui` swarm stack (traefik route to
## Session 2026-09-07 21:35 UTC — evening changes on the cc-ci host (from Claude on notplants-orchestrator) the opencode web UI) — redeployed, renamed `ccci-opencode-ui`, added to the sweep keep-list.
- notplants-nix `chore/drop-cc-ci` merged + applied on the old box (gen 57). cc-ci push self-test
green again (#33 ruff format, #34 statix). cc-ci input relocked on the host (NAR-hash note in
README day-2).
- Operator: NO Claude on the cc-ci host. The orchestrator there is an opencode agent
(agents.toml: opencode/glm-5.2) steered from https://oc.ci.commoninternet.net; claude-install,
CLAUDE_BIN and all Claude state removed from the host. Claude sessions stay on
notplants-orchestrator, whose loops `ssh cc-ci` now points at 195.201.88.249.
- Operator: only strictly necessary secrets on the cc-ci host, none of this machine's keys.
`/secrets/files` there now holds exactly: cc-ci.testenv (GITEA_PASSWORD, DOCKERHUB_*),
cc-ci-local-ed25519 (loops→root loopback, generated there, pub in nix/hosts/cc-ci/ssh-keys),
autonomic-bot-cc-ci-ed25519 (generated there, Gitea key id 156, root uses it for cc-ci-secrets),
opencode-auth.json. Gone: tailscale key, tangled key, copied cc-ci-root + bot keys, htpasswd
plaintext, TINFOIL/OPENAI keys, the master age key (cc-ci-secrets re-keyed to the host's ssh
host key age1tmvg…, cc-ci PR #35; /var/lib/sops-nix/key.txt = host-derived identity). /old-root
deleted. Inventory: /secrets/README.txt on the host.
- Weekly upgrade run started by hand on the new host at 21:23 UTC (`systemctl start
cc-ci-upgrade-all`; opencode/deepseek-v4-flash, session ses_f823e7a18ffejcu0) as the post-move
proof; result + report recorded below when done.
nix/modules/orchestrator-host.nix
+5 -4
View File
@@ -47,7 +47,7 @@ in
example = "proxy"; example = "proxy";
description = '' description = ''
Name of the swarm overlay network the cc-ci traefik watches (cc-ci's swarm.nix creates Name of the swarm overlay network the cc-ci traefik watches (cc-ci's swarm.nix creates
`proxy`). When set, a one-container swarm stack `opencode-ui` (a socat TCP relay to `proxy`). When set, a one-container swarm stack `ccci-opencode-ui` (a socat TCP relay to
nginx on the docker bridge) is deployed with traefik labels routing opencodeUiHost on nginx on the docker bridge) is deployed with traefik labels routing opencodeUiHost on
the `web-secure` entrypoint — the same label mechanism every cc-ci service and recipe the `web-secure` entrypoint — the same label mechanism every cc-ci service and recipe
uses, so it coexists with the traefik recipe's own file provider (the wildcard cert and uses, so it coexists with the traefik recipe's own file provider (the wildcard cert and
@@ -208,11 +208,12 @@ SSHCFG
# Only docker's bridge may reach the plain-HTTP backend; the public interface stays closed. # Only docker's bridge may reach the plain-HTTP backend; the public interface stays closed.
networking.firewall.interfaces.docker_gwbridge.allowedTCPPorts = [ cfg.opencodeUiBackendPort ]; networking.firewall.interfaces.docker_gwbridge.allowedTCPPorts = [ cfg.opencodeUiBackendPort ];
# The traefik side of the route: a swarm service carrying the router labels. It is a plain # The traefik side of the route: a swarm service carrying the router labels. Named ccci-* so
# the weekly run's orphan sweep (skills/upgrade-all/sweep-orphans.sh keep-list) leaves it be. A plain
# TCP relay (socat) from the overlay network to nginx on the docker bridge; traefik's # TCP relay (socat) from the overlay network to nginx on the docker bridge; traefik's
# X-Forwarded-For passes through untouched, which is what nginx's real_ip reads. # X-Forwarded-For passes through untouched, which is what nginx's real_ip reads.
systemd.services.opencode-ui-route = lib.mkIf (cfg.opencodeUiTraefikNetwork != null) { systemd.services.opencode-ui-route = lib.mkIf (cfg.opencodeUiTraefikNetwork != null) {
description = "swarm stack opencode-ui: traefik labels ${cfg.opencodeUiHost} -> nginx basic auth"; description = "swarm stack ccci-opencode-ui: traefik labels ${cfg.opencodeUiHost} -> nginx basic auth";
wantedBy = [ "multi-user.target" ]; wantedBy = [ "multi-user.target" ];
after = [ "deploy-proxy.service" "docker.service" "nginx.service" ]; after = [ "deploy-proxy.service" "docker.service" "nginx.service" ];
wants = [ "deploy-proxy.service" ]; wants = [ "deploy-proxy.service" ];
@@ -238,7 +239,7 @@ SSHCFG
networks: networks:
${cfg.opencodeUiTraefikNetwork}: ${cfg.opencodeUiTraefikNetwork}:
external: true external: true
''} opencode-ui ''} ccci-opencode-ui
''; '';
}; };
}; };