notplantsandClaude Fable 5.1
31af820079
The orchestrator's flake now builds the machine it shares with the cc-ci CI
server: `nixosConfigurations.cc-ci` composes cc-ci's nixosModules.cc-ci-server
(new flake input, nixpkgs + sops-nix follow ours), this repo's orchestrator
module (nix/modules/cc-ci.nix, exported as cc-ci-orchestrator, `cc-ci` kept
as an alias for notplants-nix) and the new nix/modules/orchestrator-host.nix
— the host contract those units always assumed (loops user, claude/opencode
CLIs, opencode web server + tailnet-only UI on 8443 since traefik owns
80/443, nix-ld, tool set, `ssh cc-ci` → loopback).
nix/hosts/cc-ci/{hardware,networking}.nix are PROVISIONAL copies of the old
server's layout so the flake evaluates; they get replaced by the
nixos-infect output of 195.201.88.249.
README.md is the deploy guide (Hetzner Debian → nixos-infect → this flake →
staging → data restore → cutover). archive/ holds the retired Incus/Hetzner
orchestrator host configs, the old terraform and the migration plans;
references updated. cc-ci-plan/plan-cc-ci-combined-host.md is the working
plan for the move.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqkQq3CDmFWcQ7u1LzoyRz
124 lines
3.4 KiB
Markdown
124 lines
3.4 KiB
Markdown
# terraform — Hetzner cc-ci-orchestrator server
|
|
|
|
Provisions a Hetzner **cpx11** (2 vCPU / 2 GB dedicated AMD / 40 GB NVMe) for the cc-ci loops
|
|
runtime (Builder + Adversary + Watchdog + Orchestrator sessions), replacing the slow b1 Incus VM.
|
|
Uses nixos-infect to convert Debian → NixOS, then converges via the cc-ci-orchestrator flake.
|
|
|
|
---
|
|
|
|
## Stage 1 — provision the server
|
|
|
|
```bash
|
|
# from /srv/cc-ci/terraform/
|
|
source /srv/cc-ci/.testenv # loads HCLOUD_TOKEN
|
|
export TF_VAR_ssh_public_key="$(cat /home/loops/.ssh/cc-ci-root-ed25519.pub)"
|
|
|
|
tofu init
|
|
tofu plan
|
|
tofu apply
|
|
```
|
|
|
|
Note the `server_ipv4` output. nixos-infect runs on first boot — wait ~5 min, then:
|
|
|
|
```bash
|
|
# confirm NixOS is up (may need to retry while infect reboots)
|
|
ssh root@<server_ipv4> 'nixos-version'
|
|
```
|
|
|
|
---
|
|
|
|
## Stage 2 — converge to cc-ci-orchestrator-hetzner
|
|
|
|
### 2a. Capture hardware config
|
|
|
|
```bash
|
|
ssh root@<server_ipv4> 'cat /etc/nixos/hardware-configuration.nix'
|
|
```
|
|
|
|
Copy the output to `nix/hosts/cc-ci-orchestrator-hetzner/hardware.nix` in this repo, commit, push.
|
|
|
|
### 2b. Stage workspace on the new server
|
|
|
|
```bash
|
|
ssh root@<server_ipv4>
|
|
|
|
# Install Tailscale auth key (from .testenv TS_AUTH_KEY)
|
|
echo "<TS_AUTH_KEY>" > /etc/ts-auth-key && chmod 600 /etc/ts-auth-key
|
|
|
|
# Clone this repo as the loops user workspace
|
|
git clone --recursive \
|
|
https://autonomic-bot:<token>@git.autonomic.zone/recipe-maintainers/cc-ci-orchestrator.git \
|
|
/srv/cc-ci-orch
|
|
ln -sfn /srv/cc-ci-orch /srv/cc-ci # loops expect /srv/cc-ci
|
|
|
|
# Place master age key (copied from current VM .sops/master-age.txt)
|
|
mkdir -p /srv/cc-ci/.sops
|
|
scp loops@<old-vm>:/srv/cc-ci/.sops/master-age.txt /srv/cc-ci/.sops/master-age.txt
|
|
chmod 600 /srv/cc-ci/.sops/master-age.txt
|
|
```
|
|
|
|
### 2c. Run nixos-rebuild
|
|
|
|
```bash
|
|
# on the new server
|
|
cd /srv/cc-ci
|
|
nixos-rebuild switch --flake .#cc-ci-orchestrator-hetzner
|
|
```
|
|
|
|
### 2d. Stage credentials (not in git — placed once)
|
|
|
|
```bash
|
|
# SSH key for reaching cc-ci
|
|
mkdir -p /home/loops/.ssh && chmod 700 /home/loops/.ssh
|
|
# scp cc-ci-root-ed25519 from current VM or copy content
|
|
chmod 600 /home/loops/.ssh/cc-ci-root-ed25519
|
|
|
|
# .testenv (GITEA creds, etc.)
|
|
cp /path/to/.testenv /srv/cc-ci/.testenv && chmod 600 /srv/cc-ci/.testenv
|
|
```
|
|
|
|
### 2e. Auth claude and start loops
|
|
|
|
```bash
|
|
# as loops user on new server
|
|
sudo -u loops /home/loops/.local/bin/claude auth login # device code — operator step
|
|
|
|
# start the loops
|
|
cd /srv/cc-ci && sudo -u loops ./cc-ci-plan/launch.sh start
|
|
```
|
|
|
|
### 2f. Verify
|
|
|
|
```bash
|
|
tmux ls # should show cc-ci-builder, cc-ci-adv, cc-ci-watchdog
|
|
```
|
|
|
|
---
|
|
|
|
## Cutover
|
|
|
|
Once the new server is running and the loops are verified:
|
|
|
|
1. Update the `Host cc-ci` entry in the current VM's `/home/loops/.ssh/config` if needed
|
|
2. Stop the old Incus VM (or just leave it idle — it costs nothing in disk)
|
|
|
|
---
|
|
|
|
## Variables
|
|
|
|
| Variable | Default | Notes |
|
|
|---|---|---|
|
|
| `location` | `nbg1` | Nuremberg |
|
|
| `server_type` | `cpx11` | 2 vCPU / 2 GB dedicated AMD. Upgrade to `cpx21` (4 GB) if OOM. |
|
|
| `image` | `debian-12` | nixos-infect base |
|
|
| `server_name` | `cc-ci-orchestrator` | |
|
|
| `ssh_public_key` | required | Pass via `TF_VAR_ssh_public_key` |
|
|
|
|
---
|
|
|
|
## State
|
|
|
|
`terraform.tfstate` and `terraform.tfstate.backup` are gitignored. Keep the state file locally or
|
|
in a remote backend — losing it means `tofu destroy` can't find the server (use `tofu import` to
|
|
recover, or delete directly via the Hetzner console).
|