terraform: provision cc-ci on Hetzner Cloud (cpx32, nixos-infect, debian-12) #2

2026-05-31T01:12:44Z

autonomic-bot commented

2026-05-31 01:12:44 +00:00

Summary

Adds terraform/ and nix/hosts/cc-ci-hetzner/ to provision cc-ci on Hetzner Cloud as an alternative to the Incus cc-nix-test VM (see cc-ci-plan/plan-cc-ci-hetzner-terraform.md).

What is in this PR

terraform/ — 7 files:

versions.tf — hcloud provider pinned at 1.64.0 (OpenTofu/Terraform compatible)
variables.tf — server type, location, image, SSH key, name
main.tf — hcloud_ssh_key + hcloud_server (cpx32, debian-12, nbg1, user-data)
outputs.tf — server IPv4, ID, SSH connect command
user-data.sh — nixos-infect pinned at 40f62a680bb0e8f2f607d79abfaaecd99d59401c (2026-03-22)
.gitignore — tfstate + auto.tfvars + .terraform/ (secrets/state never committed)
.terraform.lock.hcl — provider SHA pins (analogous to flake.lock)
README.md — full apply + Stage 2 instructions

nix/hosts/cc-ci-hetzner/ — 3 files:

hardware.nix — nixos-infect-generated hardware (GRUB+EFI, qemu-guest, /dev/sda1 ext4)
networking.nix — infect-generated static networking (update IP per README when re-provisioning)
configuration.nix — imports shared cc-ci modules + hardware; adds ports 80+443 for public IP

flake.nix — adds nixosConfigurations.cc-ci-hetzner (parallel to existing cc-ci Incus host).

Verification (throwaway run 2026-05-31)

terraform apply → cpx32 server in nbg1 created in 17 s (Hetzner ID 134464512, IP 168.119.126.100)
nixos-infect → NixOS 24.11.719113.50ab793786d9 (same nixpkgs pin as the flake)
nixos-rebuild build --flake .#cc-ci-hetzner → exit 0 on the server (131 derivations built; all cc-ci modules: tailscale, drone, drone-runner, bridge, dashboard, harness, swarm, abra, proxy, secrets, …)
terraform plan → No changes (idempotent)
terraform destroy → server + SSH key deleted; billing stopped

Operator-pending step

Full nixos-rebuild switch (Stage 2) needs the bootstrap age key at /var/lib/sops-nix/key.txt. The flake evaluates and builds without it; activation of sops-managed secrets requires it. See terraform/README.md §Stage 2.

Decisions recorded (plan §7)

v1: keep sops wildcard cert (no ACME change); evaluate ACME-on-public-IP as follow-up
cpx31 retired in nbg1; using cpx32 (same spec: AMD 4 vCPU / 8 GB x86)
Debian 12 preferred over Ubuntu 24.04 as nixos-infect base
Parallel cc-ci-hetzner host during transition; make it canonical cc-ci after cutover

Do not merge — throwaway verification only; real cutover is a separate operator step.

## Summary Adds `terraform/` and `nix/hosts/cc-ci-hetzner/` to provision cc-ci on Hetzner Cloud as an alternative to the Incus `cc-nix-test` VM (see `cc-ci-plan/plan-cc-ci-hetzner-terraform.md`). ## What is in this PR **`terraform/`** — 7 files: - `versions.tf` — hcloud provider pinned at `1.64.0` (OpenTofu/Terraform compatible) - `variables.tf` — server type, location, image, SSH key, name - `main.tf` — `hcloud_ssh_key` + `hcloud_server` (cpx32, debian-12, nbg1, user-data) - `outputs.tf` — server IPv4, ID, SSH connect command - `user-data.sh` — nixos-infect pinned at `40f62a680bb0e8f2f607d79abfaaecd99d59401c` (2026-03-22) - `.gitignore` — tfstate + auto.tfvars + .terraform/ (secrets/state never committed) - `.terraform.lock.hcl` — provider SHA pins (analogous to flake.lock) - `README.md` — full apply + Stage 2 instructions **`nix/hosts/cc-ci-hetzner/`** — 3 files: - `hardware.nix` — nixos-infect-generated hardware (GRUB+EFI, qemu-guest, /dev/sda1 ext4) - `networking.nix` — infect-generated static networking (update IP per README when re-provisioning) - `configuration.nix` — imports shared cc-ci modules + hardware; adds ports 80+443 for public IP **`flake.nix`** — adds `nixosConfigurations.cc-ci-hetzner` (parallel to existing `cc-ci` Incus host). ## Verification (throwaway run 2026-05-31) - `terraform apply` → cpx32 server in nbg1 created in 17 s (Hetzner ID 134464512, IP 168.119.126.100) - nixos-infect → NixOS `24.11.719113.50ab793786d9` (same nixpkgs pin as the flake) - `nixos-rebuild build --flake .#cc-ci-hetzner` → **exit 0** on the server (131 derivations built; all cc-ci modules: tailscale, drone, drone-runner, bridge, dashboard, harness, swarm, abra, proxy, secrets, …) - `terraform plan` → **No changes** (idempotent) - `terraform destroy` → server + SSH key deleted; billing stopped ## Operator-pending step Full `nixos-rebuild switch` (Stage 2) needs the bootstrap age key at `/var/lib/sops-nix/key.txt`. The flake evaluates and builds without it; activation of sops-managed secrets requires it. See `terraform/README.md` §Stage 2. ## Decisions recorded (plan §7) - v1: keep sops wildcard cert (no ACME change); evaluate ACME-on-public-IP as follow-up - `cpx31` retired in nbg1; using `cpx32` (same spec: AMD 4 vCPU / 8 GB x86) - Debian 12 preferred over Ubuntu 24.04 as nixos-infect base - Parallel `cc-ci-hetzner` host during transition; make it canonical `cc-ci` after cutover **Do not merge** — throwaway verification only; real cutover is a separate operator step.

autonomic-bot added 1 commit 2026-05-31 01:12:44 +00:00

terraform: provision cc-ci on Hetzner Cloud via nixos-infect 4c7150d502

Adds terraform/ (hcloud provider, cpx32/nbg1/debian-12) and a new
nix/hosts/cc-ci-hetzner/ flake host to provision the cc-ci server on
Hetzner Cloud as an alternative to the Incus cc-nix-test VM.

Stage 1 (Terraform): creates a cpx32 server (4 vCPU / 8 GB / x86 AMD,
Nuremberg), runs nixos-infect (pinned rev 40f62a6, 2026-03-22) to convert
Debian 12 → NixOS 24.11, and reboots into bare NixOS.

Stage 2 (manual, per terraform/README.md): clone cc-ci --recursive,
provision the bootstrap age key, then `nixos-rebuild switch --flake
.#cc-ci-hetzner`.

Verified (throwaway run 2026-05-31, server 134464512, 168.119.126.100):
- terraform apply: cpx32 in nbg1 created in 17 s
- nixos-infect: NixOS 24.11.719113.50ab793786d9 (same nixpkgs pin as flake)
- nixos-rebuild build --flake .#cc-ci-hetzner: exit 0 on server
  (131 derivations; all cc-ci modules: tailscale, drone, drone-runner,
  bridge, dashboard, harness, swarm, abra, proxy, secrets)
- terraform plan: no changes (idempotent)
- terraform destroy: server + SSH key removed

Age key step (plan §4 Stage 2): operator-pending. Full switch/convergence
requires bootstrap age key at /var/lib/sops-nix/key.txt. Flake builds
without it; activation needs it.

No secrets committed: HCLOUD_TOKEN via env, tfstate gitignored,
networking.nix contains throwaway IP (update per README for production).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

autonomic-bot added 1 commit 2026-05-31 03:00:05 +00:00

nix: add cc-ci-hetzner host (cpx32, nixos-infect hardware) + root SSH keys b08ebea876

Created by assistant + patched with root authorized keys so nixos-rebuild
does not lock out SSH access.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

autonomic-bot referenced this issue from a commit

2026-06-09 22:10:56 +00:00

fix(harness): convergence must span stop-first rolling updates (immich 238 backup 409)

autonomic-bot referenced this issue from a commit

2026-06-11 00:33:09 +00:00

fix(harness): merge fix/converged-oneshot @ be2026a — services_converged completed-one-shot rule (rcust M2 fix-forward #2, Adversary-approved a531746)

autonomic-bot referenced this issue from a commit

2026-06-11 00:42:56 +00:00

review(rcust): be2026a merge cold-verified — merged lifecycle.py + test file byte-identical to branch (condition #2 met); m2p-lasuite-drive L0 = diagnosed pre-fix symptom; awaiting discourse A/B + post-fix L5

autonomic-bot referenced this issue from a commit

2026-06-11 18:32:26 +00:00

claim(kuma): M2 — build #462 LEVEL 5 PASS (flake #2); DEFERRED closed; PARITY updated

autonomic-bot commented

2026-06-11 21:09:17 +00:00

Closing as superseded. The Hetzner migration is complete — the cc-ci server runs on Hetzner now, and nix/hosts/cc-ci-hetzner landed directly on main 2026-05-31 (4237cc0 + later fixes), newer than this branch. The one unmerged piece — the terraform/ provisioning scaffolding — has been salvaged onto current main as IaC-of-record in commit e37a7df. This branch is hundreds of commits behind and unmergeable; nothing left to merge.

Closing as **superseded**. The Hetzner migration is complete — the cc-ci server runs on Hetzner now, and nix/hosts/cc-ci-hetzner landed directly on main 2026-05-31 (4237cc0 + later fixes), newer than this branch. The one unmerged piece — the terraform/ provisioning scaffolding — has been salvaged onto current main as IaC-of-record in commit e37a7df. This branch is hundreds of commits behind and unmergeable; nothing left to merge.

autonomic-bot closed this pull request

2026-06-11 21:09:17 +00:00

autonomic-bot referenced this issue from a commit

2026-06-17 06:45:44 +00:00

feat(canon): M1.2 release-tag trigger + faithful mirror-sync in the weekly sweep (§2.C/§2.D)

autonomic-bot referenced this issue from a commit

2026-06-17 07:07:46 +00:00

claim(M1): canonical sweep machinery built + live-proven on custom-html

autonomic-bot referenced this issue from a commit

2026-06-18 02:00:07 +00:00

journal(redfix): M2 progress — gitea PR #2 + advance verifying; bluesky rename PR #4; harness branch redfix-m2-harness pushed (keycloak/mumble/bluesky-exec)

autonomic-bot referenced this issue from a commit