recipe-maintainers/cc-ci-orchestrator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
726a4febff556eaa32f5ae73b477f9d8c9708614
cc-ci-orchestrator/cc-ci-plan/plan-domain-migration-ci-autonomic-zone.md
T

11 KiB
Raw Blame History

Plan: migrate cc-ci domains from *.commoninternet.net to ci.autonomic.zone

Date: 2026-09-21 · Status: DRAFT — blocked only on operator DNS access (Gandi zone autonomic.zone) Principle: same host, same IP (195.201.88.249), same services — only names change. Dual-run new alongside old; old names retire last; rollback at any point is "change nothing further" because the old DNS zone keeps answering until Phase 4.

1. Inventory — what answers on what today

Name Served by Router/config source
ci.commoninternet.net results dashboard + bridge /hook cc-ci/nix/modules/dashboard.nix, bridge.nix (traefik Host rules)
report.ci.commoninternet.net weekly report static site cc-ci/nix/modules/reports.nix
drone.ci.commoninternet.net Drone server cc-ci/nix/modules/drone.nix (abra app; env DOMAIN, DRONE_SERVER_HOST); runner RPC: drone-runner.nix
oc.ci.commoninternet.net opencode UI (traefik → host nginx basic auth) cc-ci-orch/nix/modules/orchestrator-host.nix (opencodeUiHost), set in nix/hosts/cc-ci/configuration.nix
traefik.ci.commoninternet.net traefik stack identity + deploy-proxy health gate cc-ci/nix/modules/proxy.nix
warm-*.ci.commoninternet.net warm canonical recipe stacks warm-keycloak.nix, warm-bluesky-pds (caddy labels)
<tag>-<6hex>.ci.commoninternet.net per-run recipe test deploys cc-ci/runner/harness/naming.py app_domain() — resolves via public DNS (containers use 1.1.1.1/8.8.8.8)
backups.ci.commoninternet.net backup-bot-two — stack name only, no web route backupbot.nix
acme.commoninternet.net (NS ns-acme) local acme-dns (systemd, port 53) cc-ci/nix/modules/acme-dns.nix

Cert chain today: nixos security.acme (lego) issues ci.commoninternet.net + *.ci.commoninternet.net via DNS-01 through the local acme-dns; Gandi delegates _acme-challenge.ci.commoninternet.net CNAME → 9d3f0428-eb97-4b14-bb07-da2f0f6c2073.acme.commoninternet.net. postRun fires cc-ci-acme-traefik-handoff.service → installs cert into /var/lib/ci-certs/live → restarts deploy-proxy.service, whose reconciler re-derives the traefik_ci_commoninternet_net_ssl_{cert,key}_v<hash> swarm secrets (traefik runs co-op-cloud wildcard/no-ACME mode off those secrets). DNS: both zones on Gandi. IPv4-only (no AAAA records anywhere; none needed).

External references: Gitea OAuth app for Drone (redirect URI → drone.ci…); per-repo webhooks → /hook — only recipe-maintainers/ghost and recipe-maintainers/discourse have any (everything else is bridge-driven via the Drone API); ~60 doc references across cc-ci-orch (AGENTS.md, README, plan.md, skills).

2. Target naming

Old New
ci.commoninternet.net ci.autonomic.zone
report.ci.commoninternet.net report.ci.autonomic.zone
drone.ci.commoninternet.net drone.ci.autonomic.zone
oc.ci.commoninternet.net oc.ci.autonomic.zone
traefik.ci.commoninternet.net traefik.ci.autonomic.zone
*.ci.commoninternet.net (warm-*, per-run apps, backups identity) *.ci.autonomic.zone
acme.commoninternet.net stays through Phases 1–2 (infra, not a front door); optional Phase 3

Swarm stack names (traefik_ci_commoninternet_net, ccci-*, warm stacks) are identifiers, not domains — keep them where possible to avoid churn in keep-lists/prune filters, EXCEPT abra-managed apps whose name is the domain (drone, backup-bot, per-run apps).

3. DNS records to set (zone autonomic.zone, Gandi — operator)

# Name Type Value TTL Purpose
1 ci.autonomic.zone A 195.201.88.249 300 apex front door (wildcard does NOT cover the apex)
2 *.ci.autonomic.zone A 195.201.88.249 300 report/drone/oc/traefik/warm-*/<tag>-<6hex> run apps
3 _acme-challenge.ci.autonomic.zone CNAME 9d3f0428-eb97-4b14-bb07-da2f0f6c2073.acme.commoninternet.net. 300 DNS-01 for the new wildcard — reuses the existing restricted acme-dns account (disable_registration=true; the lego account authenticates, not the FQDN, so one acme-dns subdomain serves both zones)

Also: lower TTL to 300 on the old ci/*.ci records a few days ahead (fast rollback later). Phase 3 (optional) adds: acme.ci.autonomic.zone + ns-acme.ci.autonomic.zone A records.

DNS verification (before touching the host):

dig +short ci.autonomic.zone A @ns-187-a.gandi.net   # 195.201.88.249
dig +short report.ci.autonomic.zone @1.1.1.1          # 195.201.88.249 (via wildcard)
dig +short drone.ci.autonomic.zone @8.8.8.8           # 195.201.88.249
dig +short _acme-challenge.ci.autonomic.zone CNAME    # 9d3f0428-….acme.commoninternet.net.

4. Phase 1a — certificate (issue BEFORE any routing change)

  1. cc-ci/nix/modules/acme-dns.nix: add a second security.acme.certs."ci.autonomic.zone" entry (domain + *.ci.autonomic.zone SAN, dnsProvider = "acmedns", dnsResolver = "1.1.1.1:53"). Split the lego env per cert — today ACME_DNS_STORAGE_PATH is baked to /var/lib/acme/ci.commoninternet.net/…; generate one env file per cert so each keeps its own acme-dns account file.
  2. Extend cc-ci-acme-traefik-handoff to stage the new cert alongside the old (/var/lib/ci-certs/live holds both, e.g. fullchain.pem = new + fullchain-old.pem).
  3. Extend the deploy-proxy reconciler (proxy.nix) for dual-cert SNI: new wildcard pair as ssl_cert/ssl_key (default store) + old pair as a second secret pair wired through a small file-provider [[tls.certificates]] so *.ci.commoninternet.net keeps serving during the bake window. This is the one fiddly bit — validate against the Let's Encrypt staging endpoint first (the module already has the staging-marker pattern).
  4. nixos-rebuild test → confirm both certs issued and served (openssl s_client -servername for a name in each zone, check SANs) → switch.

5. Phase 1b — routing + service config (one stack at a time, health-gated)

During the bake window every router rule carries both hostnames (Host(ci.autonomic.zone) || Host(ci.commoninternet.net)); old names drop out only in Phase 4.

  • dashboard.nix + bridge.nix (share the ci host: dashboard / + bridge /hook), reports.nix, and the traefik health-gate name in proxy.nix.
  • bridge.py / dashboard.py code defaults for DRONE_URL / DASH_URL.
  • recipe-report.py — footer, publish URL text and its verification curl → report.ci.autonomic.zone (regenerate the index after, so the home page stops linking the old name).
  • Drone abra app rename (name = domain, so this is a new stack):
    1. Export enabled-repo list from the old Drone API.
    2. Deploy new abra app drone.ci.autonomic.zone (same module; secrets rpc_secret, client_secret re-insert from /run/secrets as today). Fresh DB — Drone data volume is stack-name-prefixed; repos re-sync on Gitea login, build history is lost (accepted; export the old repo list and re-enable via API).
    3. Point drone-runner.nix DRONE_RPC_HOST at the new name; restart runner.
    4. Re-run scripts/bootstrap-drone-oauth.sh with DRONE=https://drone.ci.autonomic.zone (updates the Gitea OAuth redirect URI; script already uses --resolve to hit loopback).
    5. Update the two webhooks (ghost, discourse) to https://drone.ci.autonomic.zone/hook?secret=<same> — read the secret off the existing hook config before deleting it.
    6. abra app undeploy + remove the old drone app once green.
  • configuration.nix: opencodeUiHost = "oc.ci.autonomic.zone"; extend the networking.hosts self-pin list with the new names (keep old pins during bake).
  • Gitea GITEA_SERVER stays git.autonomic.zone — unchanged.

6. Phase 2 — harness / warm / per-run naming (the _ci_commoninternet_net blast radius)

  • cc-ci/runner/harness/naming.py app_domain() → .ci.autonomic.zone (single source of truth).
  • Regexes/patterns that match the derived names: docker-prune.nix run-app filter, tests/concurrency/concutil.py (DOMAIN const + RUN_APP_RE), any separate RUN_APP_RE in lifecycle.py / warm.py / warm_reconcile.py / canonical.py / generic.py / http.py, nightly-sweep keep-list patterns, tests/gitea/recipe_meta.py comment.
  • Redeploy warm stacks under new names (abra warm-<recipe>.ci.autonomic.zone), reconcile, drain old.
  • backupbot.nix DOMAIN (stack-name-only change).
  • Verify: full deterministic test suite + one live !testme run end-to-end.

7. Phase 3 (optional) — move acme-dns off commoninternet.net

Only if commoninternet.net is being dropped entirely: new domain/nsname in acme-dns config (acme.ci.autonomic.zone / ns-acme.ci.autonomic.zone + Gandi records), briefly re-enable registration or hand-migrate the account DB, re-point _acme-challenge CNAMEs, re-issue both certs.

8. Phase 4 — retirement (after ≥7 clean days)

Remove old names from the dual router rules → remove security.acme.certs."ci.commoninternet.net"

  • its env file + old cert secret pair → drop old networking.hosts pins → delete old Gandi records (ci, *.ci, _acme-challenge.ci) → restore TTLs to 3600. Keep acme.commoninternet.net unless Phase 3 was done.

9. Verification checklist

  • DNS: the dig table in §3 against a Gandi NS and a public resolver.
  • Cert: openssl s_client -connect 195.201.88.249:443 -servername <name> for one name per zone — SANs correct for both, ssl_verify_result=0.
  • Front doors: ci → 200 · report.ci → 200 · drone.ci → 200/303 · oc.ci → 401 (basic auth intact) · traefik.ci/api/version → 200.
  • SSO: Drone login through Gitea on the new redirect URI.
  • Webhook: empty commit to recipe-maintainers/ghost main → build fires on the new host.
  • Bridge/!testme: !testme on a scratch PR → build runs, results.json written, PR comment contains new domain links.
  • Report: publish one report page and verify it at https://report.ci.autonomic.zone.
  • Host: systemctl --failed empty; deploy-proxy reconcile logs clean.

10. Risks / gotchas

  • Dual-cert SNI in the traefik reconciler is the highest-risk change — stage-first, verify SNI selection for both zones before switching.
  • Drone DB does not follow the abra rename — accept fresh build history; export/re-apply the enabled-repo list; re-insert secrets from /run/secrets (module already does).
  • Per-run recipe domains resolve via public DNS — the *.ci.autonomic.zone wildcard A record is load-bearing for every test deploy; never merge Phase 2 before it propagates.
  • Old PR comments/badges link old names — they keep working until Phase 4 removes the old records.
  • ~60 doc references (AGENTS.md, README, plan.md, skills, status skill) — one cosmetic doc PR at the end so the docs don't lie.
  • No domain-bearing secret values were found (webhook secrets are per-repo and copied as-is).

11. Sequencing

  1. Operator: TTLs down + 3 Gandi records (§3) — 5 minutes.
  2. Agent: cert PR (§4) → test → verify dual SNI → switch.
  3. Agent: routing/env PRs (§5) stack-by-stack, health-gated; OAuth + webhooks + drone rename.
  4. Agent: end-to-end verification (§9) + report republish under the new name.
  5. Bake ≥7 days → Phase 2 PR → full suite → bake → Phase 4 retirement.
Reference in New Issue View Git Blame Copy Permalink