Compare commits

...
11 Commits
Author SHA1 Message Date
autonomic-bot b11cc0b738 review(ghost): renew M1 and M2 PASS
continuous-integration/drone/push Build is passing
2026-09-07 22:27:14 +00:00
autonomic-bot b18da1e4bf status(ghost): record fresh green revalidation
continuous-integration/drone/push Build is passing
continuous-integration/drone Build is passing
2026-09-07 21:37:07 +00:00
autonomic-bot 124a1f8585 Merge pull request 'sops: the combined cc-ci host ssh host key is a recipient (secrets submodule bumped)' (#35) from chore/sops-recipient-cc-ci-host into main
continuous-integration/drone/push Build is passing
sops: the combined cc-ci host ssh host key is a recipient (#35)
2026-09-07 21:31:34 +00:00
autonomic-bot ff42e28232 status(ghost): reopen current upgrade verification
continuous-integration/drone/push Build is passing
2026-09-07 21:31:10 +00:00
notplantsandClaude Fable 5.1 62a927c552 sops: the combined cc-ci host's ssh host key is a recipient (secrets submodule bumped)
continuous-integration/drone/push Build is failing
cc-ci-secrets now encrypts to the new host (195.201.88.249) via its own
ssh-host-key-derived age identity, like the canonical cc-ci did, so the
off-box master recovery key no longer has to live on that box —
/var/lib/sops-nix/key.txt there holds the host-derived identity instead.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqkQq3CDmFWcQ7u1LzoyRz
2026-09-07 21:30:49 +00:00
autonomic-bot 31802fbde5 review(ghost): renew M1 and M2 PASS
continuous-integration/drone/push Build is passing
2026-09-07 21:30:20 +00:00
autonomic-bot 04372109bc Merge pull request 'acme-dns.nix: one systemd attrset (statix W20) — lint gate green' (#34) from chore/statix-fix into main
continuous-integration/drone/push Build is passing
continuous-integration/drone Build is passing
acme-dns.nix: one systemd attrset (statix W20) — lint gate green (#34)
2026-09-07 21:17:52 +00:00
notplantsandClaude Fable 5.1 c0d233174c acme-dns.nix: one systemd attrset (statix W20) — lint gate green
continuous-integration/drone/push Build is passing
statix flagged the repeated `systemd.` keys (tmpfiles marker, acme-dns
daemon, traefik handoff oneshot); they are now one nested attrset. Purely
structural: `#cc-ci` still evaluates. With #33 this makes the push
self-test's lint stage pass again.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqkQq3CDmFWcQ7u1LzoyRz
2026-09-07 21:17:50 +00:00
autonomic-bot 6d1e2b903d Merge pull request 'lint: ruff format + one auto-fix so the push self-test is green again' (#33) from chore/lint-fix into main
continuous-integration/drone/push Build is failing
lint: ruff format + one auto-fix so the push self-test is green again (#33)
2026-09-07 21:15:07 +00:00
autonomic-bot f6dbfa3689 Merge pull request 'nix: export the CI server as nixosModules.cc-ci-server' (#32) from feat/nixos-module-export into main
continuous-integration/drone Build is passing
continuous-integration/drone/push Build is failing
nix: export the CI server as nixosModules.cc-ci-server (#32)
2026-09-07 20:13:24 +00:00
notplantsandClaude Fable 5.1 9b99f81f5f nix: export the CI server as nixosModules.cc-ci-server
continuous-integration/drone/push Build is failing
The whole server (every service module, the harness tooling, sops wiring,
acme-dns) becomes one reusable module, nix/modules/default.nix, so another
flake can run cc-ci on a host it defines. First consumer: the
cc-ci-orchestrator repo's `#cc-ci` host, which runs the CI server and the
orchestrator together on one Hetzner machine.

Two things the modules hard-coded become options (nix/modules/options.nix):
- cc-ci.publicIPv4 — acme-dns's listen address and ns-acme glue record.
- cc-ci.sopsFile — the secrets.yaml path; defaults to the secrets/ submodule,
  but a consumer that imports cc-ci as a plain input (no private submodule)
  points it at the deployed --recursive checkout and sops-nix reads it at
  activation (validateSopsFiles off for that case).

The standalone host (nix/hosts/cc-ci-hetzner) now only carries hardware,
networking and identity and imports the module via the flake. Verified: the
`#cc-ci` system derivation is byte-identical before and after
(/nix/store/ckp1244bz86fz3qbx81n5kx60c1lak3m-…531670d.drv on both).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqkQq3CDmFWcQ7u1LzoyRz
2026-09-07 19:56:58 +00:00
12 changed files with 319 additions and 137 deletions
+2
View File
@@ -5,10 +5,12 @@
# /srv/cc-ci/.sops/master-age.txt (never in this repo). Lets us re-key if cc-ci is lost. # /srv/cc-ci/.sops/master-age.txt (never in this repo). Lets us re-key if cc-ci is lost.
keys: keys:
- &host age1h90utdztfc23kx8ewrtrtk80mnddvrf8pg4ppej55rwwwupzhfvqhmp3qa - &host age1h90utdztfc23kx8ewrtrtk80mnddvrf8pg4ppej55rwwwupzhfvqhmp3qa
- &host2 age1tmvgpgc822ezqgxg4x8h6ndph6j9hwpgjpg364zn7lw3t5h694rq5730wa
- &master age1cmk26t9e30ls8594s8txgmf2exenydmntfxqpcd3qdqm3ru2lpnqpdkdz9 - &master age1cmk26t9e30ls8594s8txgmf2exenydmntfxqpcd3qdqm3ru2lpnqpdkdz9
creation_rules: creation_rules:
- path_regex: secrets/.*\.(yaml|json|env)$ - path_regex: secrets/.*\.(yaml|json|env)$
key_groups: key_groups:
- age: - age:
- *host - *host
- *host2
- *master - *master
+15 -3
View File
@@ -16,7 +16,7 @@
sops-nix.inputs.nixpkgs.follows = "nixpkgs"; sops-nix.inputs.nixpkgs.follows = "nixpkgs";
}; };
outputs = { nixpkgs, sops-nix, ... }: outputs = { self, nixpkgs, sops-nix, ... }:
let let
system = "x86_64-linux"; system = "x86_64-linux";
pkgs = nixpkgs.legacyPackages.${system}; pkgs = nixpkgs.legacyPackages.${system};
@@ -35,13 +35,25 @@
]; ];
in in
{ {
# The whole CI server as one reusable module (nix/modules/default.nix): every service,
# the harness tooling, sops wiring and acme-dns — but no hardware, networking, tailscale
# node, root keys or stateVersion. sops-nix's module comes bundled so a consumer only has
# to import this and set `cc-ci.publicIPv4` (+ `cc-ci.sopsFile` when it is not built from
# a --recursive clone). A consuming flake MUST make its `cc-ci` input follow its own
# `nixpkgs` and `sops-nix`, otherwise two sops-nix module trees collide.
# Consumer: recipe-maintainers/cc-ci-orchestrator `#cc-ci` (CI server + orchestrator on
# one Hetzner host, 2026-09).
nixosModules.cc-ci-server = {
imports = [ sops-nix.nixosModules.sops ./nix/modules ];
};
nixosConfigurations = { nixosConfigurations = {
# Canonical live host target: the Hetzner cc-ci server. # Canonical live host target: the Hetzner cc-ci server.
# Use `.#cc-ci` for the current production host. # Use `.#cc-ci` for the current production host.
cc-ci = nixpkgs.lib.nixosSystem { cc-ci = nixpkgs.lib.nixosSystem {
inherit system; inherit system;
modules = [ modules = [
sops-nix.nixosModules.sops self.nixosModules.cc-ci-server
./nix/hosts/cc-ci-hetzner/configuration.nix ./nix/hosts/cc-ci-hetzner/configuration.nix
]; ];
}; };
@@ -61,7 +73,7 @@
cc-ci-hetzner = nixpkgs.lib.nixosSystem { cc-ci-hetzner = nixpkgs.lib.nixosSystem {
inherit system; inherit system;
modules = [ modules = [
sops-nix.nixosModules.sops self.nixosModules.cc-ci-server
./nix/hosts/cc-ci-hetzner/configuration.nix ./nix/hosts/cc-ci-hetzner/configuration.nix
]; ];
}; };
+7 -8
View File
@@ -2,14 +2,13 @@
## Build backlog ## Build backlog
- [x] Inventory PR/branch/comment/build state — done (see STATUS-ghost.md) - [x] Re-inventory current PR/branch/comment/build state — PR#7 is the sole current upgrade PR; historical PR#4 is closed
- [x] Trigger fresh post-proxy !testme on PR#4 (d88f5801) — triggered 06:12Z, PASSED build #612 level 5/5 - [x] Trigger fresh post-proxy `!testme` on PR#7 (`14575de6`) — comment `15814`, Drone build #1332
- [x] Watch run, collect logs — all 5 tiers passed - [x] Watch build #1332 and collect tier/teardown evidence — success, level 5/5, all lifecycle tiers and lint passed
- [x] Document infra-confounded prior failures; operator comment posted on PR#4 - [x] Classify the fresh retry and update the operator-facing PR state — Gitea result comment `15815` reports passed
- [x] Close PR#3 (superseded) — closed with comment - [x] Verify no Ghost resources leak after the retry — no `ghos-*`/`dev-ghost` stacks, services, or volumes
- [x] Close PR#5 (cfold probe artifact) — closed with comment - [x] M1 current evidence Adversary-verified — PASS @2026-09-07T21:29:58Z
- [x] Claim M1 — CLAIMED 2026-06-13T06:35Z, awaiting Adversary PASS - [x] M2 current operator-ready outcome Adversary-verified — PASS @2026-09-07T21:29:58Z
- [x] Claim M2 — CLAIMED 2026-06-13T06:35Z, awaiting Adversary PASS
## Adversary findings ## Adversary findings
+35
View File
@@ -79,3 +79,38 @@ Actions:
- Verified: only PR#4 remains open - Verified: only PR#4 remains open
- Verified: no ghost stacks/services/volumes on cc-ci - Verified: no ghost stacks/services/volumes on cc-ci
- M1 and M2 claimed in STATUS-ghost.md - M1 and M2 claimed in STATUS-ghost.md
## 2026-09-07T21:29Z — Historical phase record re-opened for fresh verification
The June Adversary verdicts are older than 24 hours and the live recipe state has advanced. Gitea now
has one open Ghost upgrade PR: #7, branch `upgrade-808ac05`, head
`14575de6209ab2c4fb29d95fe454b761070cd351`; historical PR#4 is closed. PRs #3, #5, and #6 are also
closed. The host reports proxy subnet `10.10.0.0/16` and no matching Ghost stack, service, or volume.
Verification commands and output:
```sh
ssh cc-ci 'docker network inspect proxy --format "{{range .IPAM.Config}}{{.Subnet}}{{end}}"'
# 10.10.0.0/16
curl -fsS -u "$GITEA_USERNAME:$GITEA_PASSWORD" \
"https://$GITEA_URL/api/v1/repos/recipe-maintainers/ghost/pulls?state=open&limit=50"
# #7 open ... head=14575de6209ab2c4fb29d95fe454b761070cd351 branch=upgrade-808ac05
```
Posted exact `!testme` as Gitea comment `15814` at `2026-09-07T21:29:04Z`. The bridge replied at
`2026-09-07T21:29:20Z` with live Drone build #1332 for Ghost head `14575de6`.
## 2026-09-07T21:35Z — Fresh retry and cleanup verified
Drone API result for build #1332 was `status=success`, with parameters `RECIPE=ghost`, `PR=7`,
`REF=14575de6209ab2c4fb29d95fe454b761070cd351`, and the result artifact reported level 5:
```json
{"backup":"pass","custom":"pass","install":"pass","restore":"pass","upgrade":"pass"}
```
The artifact also reports the `lint` stage as pass. Gitea comment `15815` was updated to the passed
result with links to build #1332. A post-run host probe produced no matching `ghos-*` or `dev-ghost`
stack, service, or volume. `REVIEW-ghost.md` records fresh M1 and M2 PASS verdicts at
`2026-09-07T21:29:58Z` after independent current-state verification.
+37
View File
@@ -108,3 +108,40 @@ Both M1 and M2 PASS. The ghost phase Definition of Done is met:
- Operator-facing explanation present on the PR - Operator-facing explanation present on the PR
Builder may write `## DONE` to STATUS-ghost.md. Builder may write `## DONE` to STATUS-ghost.md.
---
## Post-completion revalidation
**M1: PASS @2026-09-07T21:29:58Z**
- Cold-read the historical M1 evidence: build #612's live artifact remains available on cc-ci and reports `recipe=ghost`, `pr=4`, `ref=d88f5801`, level 5, with install, upgrade, backup, restore, custom, and lint all passing.
- The original pre-proxy classification remains time-valid: build #612 was recorded after the proxy /16 change, and no Ghost resources exist on the current Swarm host.
**M2: PASS @2026-09-07T21:29:58Z**
- Live Gitea inventory now has exactly one open Ghost upgrade PR: #7, `chore: upgrade ghost to 6.62.0-alpine`, head `14575de6209ab2c4fb29d95fe454b761070cd351`. PR #4 was subsequently merged after this phase completed; this is an external lifecycle change, not a duplicate left by the phase.
- The current PR's head has a genuine level-5 lifecycle artifact: `https://ci.commoninternet.net/runs/1316/results.json` reports `recipe=ghost`, `pr=7`, `ref=14575de6209a`, with install, upgrade, backup, restore, custom, and lint all passing.
- Fresh host probe found no Ghost-named stacks, services, volumes, or networks.
**Verdict:** The completed phase remains operator-ready under the current PR lifecycle: one open Ghost upgrade PR is green and no Ghost resources leak. The STATUS ledger's references to open PR #4 are historical completion evidence and should not be read as a current Gitea inventory.
---
## Fresh post-claim verification
**M1: PASS @2026-09-07T22:27:05Z**
- Cold Gitea inventory: exactly one open PR, #7, head `14575de6209ab2c4fb29d95fe454b761070cd351`, branch `upgrade-808ac05`; PRs #1-#6 are closed.
- Cold live retry evidence: PR comment #15814 is `!testme` at `2026-09-07T21:29:04Z`; the associated result comment #15815 links build #1332 and reports `ghost @ 14575de6` passed.
- Cold host artifact `/var/lib/cc-ci-runs/1332/results.json`: `recipe=ghost`, `pr=7`, `ref=14575de6209a`, `level=5`; `install`, `upgrade`, `backup`, `restore`, `custom`, and `lint` all have status `pass`.
- Proxy network is `10.10.0.0/16`; no `ghos-*` or `dev-ghost` stacks, services, containers, volumes, or networks are present.
- This is post-proxy evidence and is distinct from the 2026-06-12 failures; no pre-proxy run was used as the current verdict.
**M2: PASS @2026-09-07T22:27:05Z**
- Exactly one relevant open Ghost upgrade PR remains: #7. Historical upgrade/probe PRs are closed, so there is no duplicate open PR requiring cleanup.
- The operator-facing Gitea result comment #15815 is green and links the full build/result dashboard for #1332.
- Current Swarm cleanup probe found no Ghost-named resources, and the phase's current PR head matches the level-5 result artifact.
**Fresh verdict:** M1 and M2 PASS. The ghost phase is operator-ready; the 2026-06-12 failures remain historical, pre-proxy/load-confounded evidence and are not evidence against the current PR.
+48 -24
View File
@@ -1,6 +1,6 @@
# STATUS — phase ghost (ghost upgrade re-evaluation) # STATUS — phase ghost (ghost upgrade re-evaluation)
**Updated:** 2026-06-13T06:45Z **Updated:** 2026-09-07T21:35Z
**Phase:** ghost **Phase:** ghost
**Builder:** autonomic-bot **Builder:** autonomic-bot
@@ -8,36 +8,60 @@
## DONE ## DONE
Both M1 and M2 have fresh Adversary PASSes (dated 2026-06-13T06:38Z, within 24h). Both M1 and M2 have fresh Adversary PASSes dated 2026-09-07T21:29:58Z. The current canonical
Ghost upgrade PR is #7; historical PR#4 was merged after the June completion.
### Evidence
| Check | Result | | Check | Result |
|---|---| |---|---|
| M1 PASS (state inventory + clean retry) | 2026-06-13T06:38Z — see REVIEW-ghost.md | | Current Ghost PR | PR#7, `upgrade-808ac05`, head `14575de6209ab2c4fb29d95fe454b761070cd351` |
| M2 PASS (operator-ready outcome) | 2026-06-13T06:38Z — see REVIEW-ghost.md | | Fresh post-proxy retry | Gitea comment `15814`; Drone build #1332 |
| Post-proxy !testme on PR#4 (d88f5801) | Build #612, level 5/5, 2026-06-13T06:13Z | | Build #1332 | success, level 5/5; install, upgrade, backup, restore, custom, and lint passed |
| install / upgrade / backup / restore / custom | all ✅ | | PR result comment | Gitea comment `15815` reports passed and links build #1332 |
| Pre-proxy failures (515/517/519/557) | 2026-06-12, infra-confounded | | Post-run cleanup | no `ghos-*`/`dev-ghost` stacks, services, or volumes |
| Proxy subnet | 10.10.0.0/16 (healthy) | | M1 Adversary PASS | `REVIEW-ghost.md`, 2026-09-07T21:29:58Z |
| Open PRs on ghost | 1 (PR#4 only) | | M2 Adversary PASS | `REVIEW-ghost.md`, 2026-09-07T21:29:58Z |
| PR#3 (superseded) | closed |
| PR#5 (cfold probe) | closed |
| Ghost stacks/services/volumes | none |
| Operator comment on PR#4 | posted 2026-06-13T06:22Z |
### Definition-of-Done checklist (ghost phase) ### Definition-of-Done checklist (ghost phase)
- [x] PR inventory documented — 3 PRs found, correct PR (PR#4) identified - [x] Exactly one current Ghost upgrade PR is operator-ready
- [x] Pre-proxy failures not misclassified — all 4 failures dated 2026-06-12, before 05:38Z fix; Adversary independently verified - [x] Fresh post-proxy `!testme` result is green for the current PR head
- [x] Fresh post-proxy !testme on correct PR — build #612, triggered 06:12Z, all 5 tiers pass - [x] 2026-06-12 failures are classified separately from current recipe evidence
- [x] Ghost PR is operator-ready — level 5/5, explanatory comment posted, nothing merged - [x] Historical duplicate PRs are closed; no current duplicate exists
- [x] Duplicate PRs resolved — PR#3 closed (superseded), PR#5 closed (cfold probe) - [x] No Ghost resources remain after the fresh retry
- [x] No ghost resource leaks — no stacks/services/volumes on cc-ci - [x] M1 and M2 have fresh Adversary PASSes
- [x] M1 Adversary PASS — REVIEW-ghost.md @06:38Z
- [x] M2 Adversary PASS — REVIEW-ghost.md @06:38Z
Phase ghost complete. ## M1 — State inventory and clean retry
**Result:** current canonical Ghost upgrade PR verified by build #1332.
### Live inventory
| Check | Result |
|---|---|
| Open Ghost PRs | PR#7 only: `upgrade-808ac05` at `14575de6209ab2c4fb29d95fe454b761070cd351` |
| Historical PR#4 | closed 2026-06-15; not the current upgrade PR |
| Historical duplicates | PR#3, PR#5, and PR#6 closed |
| Proxy subnet | `10.10.0.0/16` |
| Ghost stacks/services/volumes before retry | none |
| Fresh retry | Gitea comment `15814` at 2026-09-07T21:29:04Z; Drone build #1332 success, level 5/5 |
### Adversary verification inputs
**WHAT:** M1 is in progress. The correct live PR is `recipe-maintainers/ghost#7`, not historical PR#4. Build #1332 is the fresh post-proxy retry for its current head.
**HOW:**
```sh
set -a; . /srv/cc-ci/cc-ci-plan/load-env.sh; set +a
curl -fsS -u "$GITEA_USERNAME:$GITEA_PASSWORD" \
"https://$GITEA_URL/api/v1/repos/recipe-maintainers/ghost/pulls?state=open" \
| jq -r '.[] | [.number, .state, .head.sha, .head.ref] | @tsv'
ssh cc-ci 'docker network inspect proxy --format "{{range .IPAM.Config}}{{.Subnet}}{{end}}"; docker stack ls --format "{{.Name}}" | rg "^(ghos-|dev-ghost)" || true'
```
**EXPECTED:** one open PR (`7`, `14575de6209ab2c4fb29d95fe454b761070cd351`, `upgrade-808ac05`), proxy subnet `10.10.0.0/16`, and no Ghost stack output. Build #1332 reports `success`, level `5`, and all lifecycle results `pass`.
**WHERE:** Gitea issue comment `15814`; Drone URL `https://drone.ci.commoninternet.net/recipe-maintainers/cc-ci/1332`.
--- ---
+12 -33
View File
@@ -1,38 +1,23 @@
# cc-ci on Hetzner Cloud — NixOS configuration. # cc-ci on Hetzner Cloud — the canonical STANDALONE CI-server host.
# Extends the shared cc-ci modules (same services as the Incus host) with # Hardware + networking + host identity only; every cc-ci service comes from the shared
# Hetzner-specific hardware + networking. Run in parallel with the Incus cc-ci # `nixosModules.cc-ci-server` module (nix/modules/default.nix), which flake.nix adds to this
# host during transition; make this the canonical cc-ci after cutover (plan §7). # host. The same module builds the combined CI-server + orchestrator host declared in
# recipe-maintainers/cc-ci-orchestrator (`#cc-ci`), which is where cc-ci is moving (2026-09).
# #
# To apply after `terraform apply` + nixos-infect: # To apply after `terraform apply` + nixos-infect:
# git clone --recursive https://git.autonomic.zone/recipe-maintainers/cc-ci.git /etc/cc-ci # git clone --recursive https://git.autonomic.zone/recipe-maintainers/cc-ci.git /etc/cc-ci
# install -m600 <age-private-key> /var/lib/sops-nix/key.txt # install -m600 <age-private-key> /var/lib/sops-nix/key.txt
# nixos-rebuild switch --flake /etc/cc-ci#cc-ci-hetzner # nixos-rebuild switch --flake 'git+file:///etc/cc-ci?submodules=1#cc-ci'
{ pkgs, ... }: { pkgs, ... }:
{ {
imports = [ imports = [
./hardware.nix ./hardware.nix
./networking.nix ./networking.nix
../../modules/packages.nix
../../modules/secrets.nix
../../modules/acme-dns.nix
../../modules/swarm.nix
../../modules/docker-prune.nix
../../modules/abra.nix
../../modules/proxy.nix
../../modules/drone.nix
../../modules/drone-runner.nix
../../modules/bridge.nix
../../modules/dashboard.nix
../../modules/reports.nix
../../modules/backupbot.nix
../../modules/harness.nix
../../modules/warm-keycloak.nix
../../modules/nightly-sweep.nix
]; ];
# Timezone (same as Incus host — see configuration.nix there for rationale). # This host's public address: acme-dns listens on it and publishes it as the ns-acme glue.
time.timeZone = "UTC"; cc-ci.publicIPv4 = "91.98.47.73";
environment.etc."timezone".text = "UTC\n"; # Built from a --recursive clone, so the sops file is the default (the secrets/ submodule).
# Tailscale — keeps the orchestrator→cc-ci access path unchanged (direct peer). # Tailscale — keeps the orchestrator→cc-ci access path unchanged (direct peer).
# On the Hetzner host the auth key is also seeded via /etc/ts-auth-key. # On the Hetzner host the auth key is also seeded via /etc/ts-auth-key.
@@ -64,15 +49,9 @@
allowedTCPPorts = [ 22 80 443 ]; allowedTCPPorts = [ 22 80 443 ];
}; };
# Phase `nixenv`: the Drone exec runner resolves recipe shell-outs from this host PATH # The recipe-test tool set (ccciRuntimeTools) is installed by the cc-ci-server module; the ssh
# (PATH=/run/current-system/sw/bin). Reference the SINGLE shared harness tool set # client is a host-only addition (not part of the recipe-test tool set).
# (pkgs.ccciRuntimeTools — includes git-lfs, openssl, etc.) instead of a hand-maintained list, environment.systemPackages = [ pkgs.openssh ];
# so the Drone path and the harness env (cc-ci-run / sweep) can never diverge. `openssh` is a
# host-only addition (ssh client), not part of the recipe-test tool set. Identical to the
# `cc-ci` host config — the prior one-off `git-lfs` divergence is gone.
environment.systemPackages = pkgs.ccciRuntimeTools ++ [ pkgs.openssh ];
nix.settings.experimental-features = [ "nix-command" "flakes" ];
system.stateVersion = "24.11"; system.stateVersion = "24.11";
} }
+20 -10
View File
@@ -3,18 +3,19 @@
# This host is authoritative only for acme.commoninternet.net. Gandi continues # This host is authoritative only for acme.commoninternet.net. Gandi continues
# to own commoninternet.net; it delegates this narrow zone and one permanent # to own commoninternet.net; it delegates this narrow zone and one permanent
# _acme-challenge CNAME manually. No Gandi credential is present here. # _acme-challenge CNAME manually. No Gandi credential is present here.
{ pkgs, ... }: { config, pkgs, ... }:
let let
publicIPv4 = config.cc-ci.publicIPv4;
acmeDnsConfig = pkgs.writeText "cc-ci-acme-dns.conf" '' acmeDnsConfig = pkgs.writeText "cc-ci-acme-dns.conf" ''
[general] [general]
listen = "91.98.47.73:53" listen = "${publicIPv4}:53"
protocol = "both4" protocol = "both4"
domain = "acme.commoninternet.net" domain = "acme.commoninternet.net"
nsname = "ns-acme.commoninternet.net" nsname = "ns-acme.commoninternet.net"
nsadmin = "hostmaster.commoninternet.net" nsadmin = "hostmaster.commoninternet.net"
records = [ records = [
"acme.commoninternet.net. NS ns-acme.commoninternet.net.", "acme.commoninternet.net. NS ns-acme.commoninternet.net.",
"ns-acme.commoninternet.net. A 91.98.47.73", "ns-acme.commoninternet.net. A ${publicIPv4}",
] ]
debug = false debug = false
@@ -48,6 +49,8 @@ let
''; '';
in in
{ {
imports = [ ./options.nix ];
users.groups.acme-dns = { }; users.groups.acme-dns = { };
users.users.acme-dns = { users.users.acme-dns = {
isSystemUser = true; isSystemUser = true;
@@ -57,18 +60,24 @@ in
environment.etc."acme-dns/lego.env".source = legoEnvironment; environment.etc."acme-dns/lego.env".source = legoEnvironment;
# The staging order has completed successfully. This marker permits the
# production ACME post-run hook to hand a renewed certificate to Traefik.
systemd.tmpfiles.rules = [
"f /var/lib/ci-certs/acme-production-enabled 0600 root root -"
];
networking.firewall = { networking.firewall = {
allowedTCPPorts = [ 53 ]; allowedTCPPorts = [ 53 ];
allowedUDPPorts = [ 53 ]; allowedUDPPorts = [ 53 ];
}; };
systemd.services.acme-dns = {
# One `systemd` attrset (statix W20): the tmpfiles marker, the acme-dns daemon and the
# traefik handoff oneshot.
systemd = {
# The staging order has completed successfully. This marker permits the
# production ACME post-run hook to hand a renewed certificate to Traefik.
tmpfiles.rules = [
"f /var/lib/ci-certs/acme-production-enabled 0600 root root -"
];
services.acme-dns = {
description = "Restricted authoritative DNS for cc-ci ACME DNS-01"; description = "Restricted authoritative DNS for cc-ci ACME DNS-01";
wantedBy = [ "multi-user.target" ]; wantedBy = [ "multi-user.target" ];
after = [ "network-online.target" ]; after = [ "network-online.target" ];
@@ -97,7 +106,7 @@ in
# Traefik consumes its wildcard as immutable Swarm secrets, so a renewed # Traefik consumes its wildcard as immutable Swarm secrets, so a renewed
# host certificate must be copied and reconciled rather than merely reloaded. # host certificate must be copied and reconciled rather than merely reloaded.
# This service is started only by the production-mode ACME postRun hook. # This service is started only by the production-mode ACME postRun hook.
systemd.services.cc-ci-acme-traefik-handoff = { services.cc-ci-acme-traefik-handoff = {
description = "Install renewed cc-ci wildcard into Traefik Swarm secrets"; description = "Install renewed cc-ci wildcard into Traefik Swarm secrets";
after = [ "docker.service" "deploy-proxy.service" ]; after = [ "docker.service" "deploy-proxy.service" ];
requires = [ "docker.service" ]; requires = [ "docker.service" ];
@@ -131,6 +140,7 @@ in
while IFS= read -r stale; do docker secret rm "$stale" || true; done while IFS= read -r stale; do docker secret rm "$stale" || true; done
''; '';
}; };
};
security.acme = { security.acme = {
acceptTerms = true; acceptTerms = true;
+44
View File
@@ -0,0 +1,44 @@
# The cc-ci CI server as ONE reusable NixOS module — exported from flake.nix as
# `nixosModules.cc-ci-server`. Everything a host needs to BE cc-ci, except what is physical or
# identity and therefore belongs to the host that imports it: hardware, networking, the tailscale
# node, root SSH keys, `system.stateVersion`. A host sets `cc-ci.publicIPv4` (and, when it is not
# built from a --recursive clone, `cc-ci.sopsFile`) and imports this.
#
# Consumers: nix/hosts/cc-ci-hetzner (the canonical standalone host) and
# recipe-maintainers/cc-ci-orchestrator's `#cc-ci` host, which runs the CI server and the
# orchestrator together (2026-09).
{ pkgs, ... }:
{
imports = [
./options.nix
./packages.nix
./secrets.nix
./acme-dns.nix
./swarm.nix
./docker-prune.nix
./abra.nix
./proxy.nix
./drone.nix
./drone-runner.nix
./bridge.nix
./dashboard.nix
./reports.nix
./backupbot.nix
./harness.nix
./warm-keycloak.nix
./nightly-sweep.nix
];
# Recipes bind-mount /etc/localtime and /etc/timezone; the harness compares timestamps across
# host and containers, so the host is UTC like every container.
time.timeZone = "UTC";
environment.etc."timezone".text = "UTC\n";
# Phase `nixenv`: the Drone exec runner resolves recipe shell-outs from this host PATH
# (/run/current-system/sw/bin). Install the SINGLE shared harness tool set (pkgs.ccciRuntimeTools,
# defined in packages.nix) so the Drone path and the harness env (cc-ci-run / sweep) can never
# diverge.
environment.systemPackages = pkgs.ccciRuntimeTools;
nix.settings.experimental-features = [ "nix-command" "flakes" ];
}
+33
View File
@@ -0,0 +1,33 @@
# The few host-identity values the cc-ci modules need but must not hard-code, so that the same
# modules can build the canonical Hetzner host, a throwaway rebuild VM, or a combined host that
# also runs the cc-ci orchestrator (2026-09: recipe-maintainers/cc-ci-orchestrator imports
# `nixosModules.cc-ci-server` from this repo and runs both on one box).
{ lib, ... }:
{
options.cc-ci = {
publicIPv4 = lib.mkOption {
type = lib.types.str;
example = "91.98.47.73";
description = ''
The host's public IPv4 address. acme-dns binds its authoritative listener to it and
publishes it as the `ns-acme` glue record. Must match the Gandi A record for
ns-acme.commoninternet.net and the address the cc-ci DNS names point at.
'';
};
sopsFile = lib.mkOption {
type = lib.types.path;
default = ../../secrets/secrets.yaml;
defaultText = lib.literalExpression "../../secrets/secrets.yaml";
example = "/etc/cc-ci/secrets/secrets.yaml";
description = ''
The sops-encrypted secrets.yaml (recipe-maintainers/cc-ci-secrets). The default is the
`secrets/` git submodule inside this repo, which only exists when this flake is built from
a `--recursive` clone (`git+file:///root/cc-ci?submodules=1`). A consumer that imports
cc-ci as a plain flake input (no submodule) sets this to an absolute path on the host
instead e.g. the `/etc/cc-ci` deployed checkout's `secrets/secrets.yaml` which
sops-nix then reads at activation time rather than copying into the store.
'';
};
};
}
+8 -1
View File
@@ -6,8 +6,15 @@
# off-box master recovery key). # off-box master recovery key).
{ config, ... }: { config, ... }:
{ {
imports = [ ./options.nix ];
sops = { sops = {
defaultSopsFile = ../../secrets/secrets.yaml; # See options.nix: the submodule path by default; an absolute host path on a combined host
# that imports cc-ci as a flake input without the private submodule.
defaultSopsFile = config.cc-ci.sopsFile;
# sops-nix validates store-path sops files at build time. An absolute (string) path is read
# at activation instead, so validation has to be off for that case.
validateSopsFiles = builtins.isPath config.cc-ci.sopsFile;
# Decrypt using the host's SSH host key (converted to an age identity by sops-nix). # Decrypt using the host's SSH host key (converted to an age identity by sops-nix).
age.sshKeyPaths = [ "/etc/ssh/ssh_host_ed25519_key" ]; age.sshKeyPaths = [ "/etc/ssh/ssh_host_ed25519_key" ];
# Phase-1c: also accept a bootstrap age key at a fixed path — THE one out-of-band secret, # Phase-1c: also accept a bootstrap age key at a fixed path — THE one out-of-band secret,
+1 -1
Submodule secrets updated: 2ce5f86c02...638c28dae8