cc-ci/JOURNAL.md

JOURNAL — cc-ci Builder (append-only)

2026-05-26 — Bootstrap (§1)

Access verification (all pass):

• ssh cc-ci 'hostname && whoami && nixos-version' → nixos / root / 24.11.719113.50ab793786d9 (Vicuna)
• curl https://git.autonomic.zone/api/v1/version → {"version":"1.24.2"}
• Gitea bot auth (curl -u $GITEA_USERNAME:$GITEA_PASSWORD .../api/v1/user) → login: autonomic-bot, id 64
• getent hosts probe-$RANDOM.ci.commoninternet.net → 143.244.213.108 (the gateway IP, as expected — TLS passthrough)
• Cert present: ls /var/lib/ci-certs/live/ → fullchain.pem (2909 b), privkey.pem (227 b, mode 640)
• recipe-maintainers org exists (private); recipe-maintainers/cc-ci → 404 (created below)
• Mirrored recipes already present: bluesky-pds, lasuite-docs, custom-html, custom-html-tiny, n8n, keycloak, lasuite-meet, matrix-synapse, cryptpad

Baseline (docs/baseline.md): fresh NixOS 24.11 Incus VM, 2 vCPU, 3.5 GiB RAM, 8.9 GiB disk (3.8 GiB free). No docker/swarm/abra. Channel-based /etc/nixos/configuration.nix (no flake).

Actions:

• Created repo recipe-maintainers/cc-ci (private) via Gitea API.
• git init in /srv/cc-ci/cc-ci; credential helper reads creds from /srv/cc-ci/.testenv (no secrets stored in git config).
• Seeded skeleton layout (§3) + loop-state files + docs/baseline.md.

Next: commit + push bootstrap, then M0 (flake + base config + sops test secret).

2026-05-26 — M0: flake + base config rebuilt from repo

Authored flake.nix (pins nixpkgs rev 50ab793786d9…, the exact rev cc-ci ran), hosts/cc-ci/hardware.nix (incus VM module + cloud-init + DHCP/nameservers) and hosts/cc-ci/configuration.nix (faithful baseline repro: tailscale w/ hardcoded --hostname= cc-nix-test since builtins.readFile /etc/ts-hostname is impure under flakes; sshd root; firewall trust tailscale0 + tcp/22; base pkgs).

Disk/inode hiccup → resolved: first nix flake lock/build hit No space left on device — diagnosed as inode exhaustion (df -i → 6005 free of 586336; old 8.9 GiB fs). Operator grew the VM to 28 GiB while I was measuring; ext4 auto-resized → 22 GiB free, 1.21M inodes free. Retried.

Build + switch (commands + output):

• ssh cc-ci 'cd /root/cc-ci && nix flake lock && nixos-rebuild build --flake .#cc-ci' → BUILD EXIT 0, produced nixos-system-nixos-24.11.20250630.50ab793.
• ssh cc-ci 'systemd-run --unit=ccci-rebuild --collect --property=Type=oneshot nixos-rebuild switch --flake /root/cc-ci#cc-ci' (detached so it survives ssh drop) → unit Result=success ExecMainStatus=0.

Gate verification:

• systemctl is-system-running → running
• readlink /run/current-system → …-nixos-system-nixos-24.11.20250630.50ab793 (gen 3, from flake)
• systemctl is-active tailscaled → active; sshd.socket → active (sshd is socket-activated, so sshd.service reads inactive — live ssh proves it works)
• systemctl --failed → none
• nixos-rebuild list-generations → gen 3 current @20:23, prior channel gen 2 retained for rollback.

Known warning (tracked, non-blocking): incus module enables systemd.network while we keep networking.useDHCP=true (scripted dhcpcd); Nix warns both may manage interfaces. Inherited from baseline; networking is up. Clean up by choosing one stack later.

Deploy mechanism settled (DECISIONS.md): switch --flake on-host, repo synced via tar | ssh.

Next: sops-nix wiring (host age key from ssh host key + a decrypt-a-test-secret proof), then CLAIM the M0 gate for the Adversary.

2026-05-26 — M0: sops-nix wiring + decrypt-a-test-secret (M0 COMPLETE, gate CLAIMED)

Keys:

• Host age recipient from ssh host key: ssh cc-ci 'nix run nixpkgs#ssh-to-age -- -i /etc/ssh/ssh_host_ed25519_key.pub' → age1h90utdztfc23kx8ewrtrtk80mnddvrf8pg4ppej55rwwwupzhfvqhmp3qa.
• Master recovery key generated on host (age-keygen), public age1cmk26t…; private moved off-box to /srv/cc-ci/.sops/master-age.txt (mode 600) and shred-ded from the host. Never in repo.

Files: .sops.yaml (both recipients, rule secrets/.*\.(yaml|json|env)$); modules/secrets.nix (sops.age.sshKeyPaths=[/etc/ssh/ssh_host_ed25519_key], secrets.test_secret={}); flake gains sops-nix input + sops-nix.nixosModules.sops; configuration.nix imports the module.

sops-nix version pin (dead-end avoided): master sops-nix wants buildGo125Module (Go 1.25), absent in pinned nixpkgs 24.11 → eval error. Pinned sops-nix to 77c423a… (2025-06-17, last using plain buildGoModule). Verified the file at that rev uses buildGoModule. Build then OK.

Encrypt test secret: on host, printf 'test_secret: cc-ci-m0-<rand>' > secrets/secrets.yaml then nix run nixpkgs#sops -- --encrypt --in-place secrets/secrets.yaml (run inside repo so .sops.yaml resolves) → rc=0, two age recipients in the file.

Build + switch (commands + output):

• nixos-rebuild build --flake .#cc-ci → BUILD EXIT 0 (built sops-install-secrets w/ Go 1.23.8).
• systemd-run --unit=ccci-rebuild2 ... nixos-rebuild switch --flake /root/cc-ci#cc-ci → Result=success ExecMainStatus=0.

Gate verification (M0):

• systemctl is-system-running → running; systemctl --failed → none.
• ls -la /run/secrets/test_secret → -r-------- 1 root root 41 ; stat → root:root 400.
• head -c9 → cc-ci-m0- (matches generated value), wc -c → 41 (9 + 32 hex). Decrypt path proven.
• Pulled encrypted secrets/secrets.yaml + flake.lock back to clone; grep cc-ci-m0 secrets.yaml → no plaintext leak; lock inputs = nixpkgs, sops-nix.

Gate handshake: set Gate: M0 — CLAIMED, awaiting Adversary in STATUS.md. REVIEW.md still empty (no Adversary activity yet). Per §6.1 liveness I won't idle-block: I keep M0 claimed and proceed with M1 (independent infra build), without advancing to M2 until M0 shows PASS.

Next: M1 — Docker + single-node swarm via Nix (modules/swarm.nix), then Traefik (file provider → /var/lib/ci-certs/live/) + abra, then a by-hand HTTPS deploy/teardown of a trivial recipe.