Files
cc-ci-orchestrator/cc-ci-plan/REBOOTS.md
autonomic-bot 21e7a79f50 orchestrator-hetzner: enable reboot-resilience + record migration
Now the workspace is staged on the Hetzner cpx22 (server 134487234, public
91.98.47.73, tailnet cc-ci-orchestrator-1 @ 100.84.190.30):

- configuration.nix: enable cc-ci-loops.service (wantedBy multi-user.target) so the
  loops + watchdog auto-resume on boot; wire reboot-log.sh as ExecStartPre so reboots
  auto-log to REBOOTS.md (boot_id-gated).
- plan-orchestrator-hetzner-migration.md: full migration record.
- REBOOTS.md / AGENTS.md: point the orchestrator host at Hetzner; first auto-logged
  reboot line.
- launch-orchestrator.sh: default session id -> the Hetzner orchestrator session.
- flake.lock: pin inputs.

Verified: nixos-rebuild switch applied; systemctl is-enabled cc-ci-loops.service =
enabled; ExecStartPre logged this boot to REBOOTS.md; loops healthy on phase 2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-31 03:54:17 +00:00

25 lines
1.8 KiB
Markdown

# Reboot log — cc-ci orchestrator VM
**Note:** the orchestrator Pi (`raspberrypi`) was decommissioned 2026-05-31. The agents then ran on
the `cc-ci-orchestrator` Incus NixOS VM (tailnet `100.116.55.106`), and **as of 2026-05-31 run on a
Hetzner `cpx22` cloud server** (`cc-ci-orchestrator-1`, tailnet `100.84.190.30`, public
`168.119.126.100`, flake host `cc-ci-orchestrator-hetzner`, Hetzner server 134487234) — see
`plan-orchestrator-hetzner-migration.md`. The three Pi reboot entries below are historical; the
2026-05-30 entry is an Incus-VM reboot. Hetzner-host reboots are logged from now on (auto-logged once
`cc-ci-loops.service` is enabled — wired at the Hetzner cutover).
One line per genuine reboot of the orchestrator host, appended automatically by
`reboot-log.sh` (ExecStartPre of `cc-ci-loops.service`, boot_id-gated so manual service restarts are
NOT counted). A reboot drops the tmux sessions (and the orchestrator session); `cc-ci-loops.service`
restarts the loops on boot. Count the lines below to see how often it's happening.
## Reboots
- 2026-05-28 (~19:?? BST) — reboot (backfilled from memory; mid-Phase-2). Orchestrator + loops were
down until manually relaunched. This pre-dates the systemd auto-restart service.
- 2026-05-28 (~20:02 BST) — reboot (backfilled from memory; uptime showed 5 min at 20:07). Loops
manually relaunched at phase 2; this is what prompted adding `cc-ci-loops.service` +
auto-logging. Auto-logging is live from the next reboot onward.
- 2026-05-30 17:03:05 BST — reboot detected; loops auto-started by systemd (resuming phase index 6). boot_id=f565f752-0463-42db-b787-9e0db35a5e3f
- 2026-05-31 03:38:29 UTC — reboot detected; loops auto-started by systemd (resuming phase index 5). boot_id=51c17fc3-8391-4109-bce2-413fbee6f26d