M2: Drone server + exec runner up; infra as idempotent-reconcile oneshots
Convert proxy+drone bring-up to writeShellApplication systemd oneshots that reconcile every activation (orchestrator steer). pkgs.abra overlay. Runner connected via RPC (polling, capacity=2). install.md = clone + nixos-rebuild switch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
16
DECISIONS.md
16
DECISIONS.md
@ -32,6 +32,22 @@ Architecture decisions and dead-ends. One line of rationale each. (§0, §8)
|
||||
- **abra teardown syntax** (for harness, §4.3): `abra app undeploy <d> -n`,
|
||||
`abra app volume remove <d> -f -n`, `abra app secret remove <d> --all -n`. None take `--chaos`.
|
||||
|
||||
- **Infra bring-up = idempotent-reconcile systemd oneshots — SETTLED (M2, orchestrator steer
|
||||
2026-05-26).** Every piece of swarm infra that abra deploys (traefik `modules/proxy.nix`, Drone
|
||||
`modules/drone.nix`, later comment-bridge + dashboard) is a `systemd.services.<x>` with
|
||||
`Type=oneshot` + `RemainAfterExit`, `after`/`requires` swarm-init + docker, `wants`
|
||||
network-online, `wantedBy` multi-user, embedding its script via **`pkgs.writeShellApplication`**
|
||||
(self-contained in the store, not a `/root/cc-ci` path). The script **reconciles** (inspect →
|
||||
converge → no-op if correct) on *every* activation/boot — **no run-once sentinel** — so it
|
||||
self-heals drift (stack gone → redeploy; secret missing → re-insert). Fails visibly (failed unit)
|
||||
on missing preconditions (e.g. cert absent). Result: a from-scratch install (D8) collapses to
|
||||
`git clone` + `nixos-rebuild switch` + operator preconditions, no manual post-steps. The old
|
||||
`scripts/deploy-*.sh` were folded into these modules and removed. `pkgs.abra` is provided via an
|
||||
overlay (`modules/packages.nix`) so all modules share the one pinned build.
|
||||
- *Cert rotation note:* the proxy reconcile inserts ssl_cert/ssl_key only if absent; rotating the
|
||||
wildcard means bumping `SECRET_WILDCARD_*_VERSION` (operator) so the next reconcile re-inserts.
|
||||
Documented in docs/secrets.md at M7.
|
||||
|
||||
## Open (defaults from §8, to confirm as reality lands)
|
||||
|
||||
- **Deploy mechanism — SETTLED (M0):** `nixos-rebuild switch --flake /root/cc-ci#cc-ci` run *on
|
||||
|
||||
36
JOURNAL.md
36
JOURNAL.md
@ -209,3 +209,39 @@ Server env: DRONE_SERVER_HOST/PROTO, DRONE_USER_CREATE.
|
||||
**Next:** scripts/deploy-drone.sh (abra deploy of drone server w/ Gitea SSO + rpc/client secrets),
|
||||
modules/drone-runner.nix (exec runner systemd unit, rpc secret from sops), wire sops secrets for the
|
||||
runner, then push a hello-world .drone.yml and confirm a green build (M2 gate).
|
||||
|
||||
## 2026-05-26 — M2: Drone server + exec runner up; infra as idempotent-reconcile oneshots
|
||||
|
||||
**Orchestrator steer (2×):** collapse install to a single `nixos-rebuild switch` — convert the
|
||||
manual deploy scripts into **idempotent-reconcile systemd oneshots** (writeShellApplication, embedded
|
||||
in store; after swarm-init+docker; wants network-online; wantedBy multi-user; reconcile every
|
||||
activation/boot, NO run-once sentinel; fail visibly on missing cert). Applied to proxy + drone.
|
||||
|
||||
**Refactor done:**
|
||||
- `modules/packages.nix`: `pkgs.abra` overlay (shared pinned build).
|
||||
- `modules/proxy.nix`: `deploy-proxy` oneshot — reconciles coop-cloud traefik (wildcard/no-ACME).
|
||||
- `modules/drone.nix`: `deploy-drone` oneshot — reconciles coop-cloud drone (Gitea SSO, secrets from
|
||||
/run/secrets), after deploy-proxy.
|
||||
- `modules/drone-runner.nix`: exec runner (fixed PATH conflict via `lib.mkForce`; allowUnfree for
|
||||
drone-runner-exec — Polyform license).
|
||||
- `modules/secrets.nix`: declared drone_rpc_secret + drone_gitea_client_secret + a sops *template*
|
||||
`drone-runner.env` (DRONE_RPC_SECRET) as the runner's EnvironmentFile (shared secret).
|
||||
- Removed `scripts/deploy-*.sh`. install.md now = clone + nixos-rebuild switch + preconditions.
|
||||
|
||||
**Build/switch:** build EXIT 0 (shellcheck clean via writeShellApplication; runner pkg unfree-allowed).
|
||||
`nixos-rebuild switch` → all three units `active`/`success`:
|
||||
- `deploy-proxy` success (reconciled traefik), `deploy-drone` → `deploy succeeded 🟢` (drone/drone
|
||||
2.26.0, secrets client_secret+rpc_secret v1, drone_env config), `drone-runner-exec` active.
|
||||
|
||||
**Verify (commands + output):**
|
||||
- `docker service ls` → `drone_ci_commoninternet_net_app 1/1`, traefik app+socket-proxy 1/1.
|
||||
- Via gateway: `…/healthz` → **200**; `/` → **303** (login redirect, correct).
|
||||
- Runner: journal shows a few startup `cannot ping the remote server (404)` (drone RPC not ready
|
||||
yet) then `successfully pinged the remote server` + `polling the remote server capacity=2
|
||||
endpoint=https://drone.ci.commoninternet.net kind=pipeline type=exec`. **Runner connected via RPC.**
|
||||
|
||||
**Remaining for M2 gate:** push a hello-world `.drone.yml` to cc-ci + get a green build. Needs the
|
||||
cc-ci repo activated in Drone, which requires the bot's Gitea OAuth login (browser flow) to grant
|
||||
Drone a Gitea token (to sync repos + set the push webhook). Next tick: script the OAuth login to mint
|
||||
a Drone token, activate cc-ci, push .drone.yml, confirm green. (DRONE_USER_CREATE made autonomic-bot
|
||||
the admin.)
|
||||
|
||||
@ -2,8 +2,12 @@
|
||||
|
||||
> WORK IN PROGRESS — grows with each milestone; the full from-scratch rebuild is verified at M9 (D8).
|
||||
|
||||
cc-ci is declared as a NixOS flake (this repo) plus a reproducible proxy-deploy step. Target:
|
||||
a NixOS 24.11 host reachable as `cc-ci` over SSH (root), with the operator preconditions in place.
|
||||
cc-ci is declared **entirely** as a NixOS flake (this repo). Bringing up the box is just
|
||||
**clone + `nixos-rebuild switch`** + the operator preconditions — no manual post-steps. The proxy
|
||||
(traefik) and Drone server are deployed by **idempotent-reconcile systemd oneshots** (`modules/
|
||||
proxy.nix`, `modules/drone.nix`) that converge the swarm to the desired state on every activation
|
||||
and boot (and self-heal drift), mirroring `swarm-init`. Target: a NixOS 24.11 host reachable as
|
||||
`cc-ci` over SSH (root).
|
||||
|
||||
## Operator preconditions (class-A1, see DECISIONS.md / docs/baseline.md)
|
||||
|
||||
@ -12,43 +16,40 @@ a NixOS 24.11 host reachable as `cc-ci` over SSH (root), with the operator preco
|
||||
- DNS: `*.ci.commoninternet.net` (+ bare) → the **gateway**, which TLS-passthroughs (SNI) to cc-ci.
|
||||
- Firewall path: gateway reaches cc-ci on tcp/80+443 (opened by `modules/swarm.nix`).
|
||||
|
||||
## 1. Apply the NixOS flake
|
||||
## 1. Apply the NixOS flake (this is the whole install)
|
||||
|
||||
The flake (`flake.nix`, `hosts/cc-ci/`, `modules/`) declares: base host, sops-nix (decrypts via the
|
||||
host SSH key), Docker + single-node Swarm + the `proxy` overlay (`modules/swarm.nix`), and abra
|
||||
(`modules/abra.nix`).
|
||||
host SSH key), Docker + single-node Swarm + the `proxy` overlay + firewall 80/443
|
||||
(`modules/swarm.nix`), abra (`modules/abra.nix` / `packages.nix`), the **traefik reconcile oneshot**
|
||||
(`modules/proxy.nix`), the **Drone server reconcile oneshot** (`modules/drone.nix`), and the
|
||||
**Drone exec runner** (`modules/drone-runner.nix`).
|
||||
|
||||
```sh
|
||||
# materialise the repo on the host (the build runs on cc-ci itself — see DECISIONS.md deploy mech)
|
||||
# e.g. git clone <repo> /root/cc-ci (or sync it)
|
||||
nixos-rebuild switch --flake /root/cc-ci#cc-ci
|
||||
# verify
|
||||
systemctl is-system-running # -> running
|
||||
docker info --format '{{.Swarm.LocalNodeState}}' # -> active
|
||||
docker network ls | grep proxy # -> proxy ... overlay swarm
|
||||
```
|
||||
|
||||
On activation, the reconcile oneshots (`deploy-proxy`, `deploy-drone`) run automatically and converge
|
||||
the swarm. Verify:
|
||||
|
||||
```sh
|
||||
systemctl is-system-running # -> running
|
||||
docker info --format '{{.Swarm.LocalNodeState}}' # -> active
|
||||
docker service ls # traefik (app+socket-proxy) + drone, all 1/1
|
||||
systemctl is-active deploy-proxy deploy-drone drone-runner-exec # -> active x3
|
||||
# wildcard cert served end-to-end via the gateway:
|
||||
curl -ksv --resolve probe.ci.commoninternet.net:443:<gateway-ip> https://probe.ci.commoninternet.net/ \
|
||||
2>&1 | grep -E 'subject:|HTTP/' # -> CN=*.ci.commoninternet.net, HTTP 404 (no app router yet)
|
||||
curl -ks --resolve drone.ci.commoninternet.net:443:<gateway-ip> \
|
||||
-o /dev/null -w '%{http_code}\n' https://drone.ci.commoninternet.net/healthz # -> 200
|
||||
```
|
||||
|
||||
> Tip: when driving the switch over an SSH session that rides Tailscale, run it as a detached unit so
|
||||
> it survives a momentary drop, and **use the absolute flake path** (systemd units run with cwd `/`):
|
||||
> `systemd-run --unit=ccci-sw --property=Type=oneshot nixos-rebuild switch --flake /root/cc-ci#cc-ci`
|
||||
|
||||
## 2. Deploy the reverse proxy (coop-cloud traefik, wildcard/file-provider, no ACME)
|
||||
## 2. (later milestones) comment-bridge, dashboard, recipe enrollment
|
||||
|
||||
```sh
|
||||
bash /root/cc-ci/scripts/deploy-proxy.sh
|
||||
```
|
||||
|
||||
This idempotently deploys the canonical Co-op Cloud `traefik` recipe via abra in wildcard mode,
|
||||
serving the pre-issued cert as the `ssl_cert`/`ssl_key` swarm secrets, with `LETS_ENCRYPT_ENV` empty
|
||||
so no ACME ever runs (see DECISIONS.md "Proxy: real coop-cloud/traefik via abra"). Verify:
|
||||
|
||||
```sh
|
||||
docker service ls | grep traefik # app + socket-proxy, 1/1
|
||||
# wildcard cert served end-to-end via the gateway:
|
||||
curl -ksv --resolve probe.ci.commoninternet.net:443:<gateway-ip> https://probe.ci.commoninternet.net/ \
|
||||
2>&1 | grep -E 'subject:|HTTP/' # -> CN=*.ci.commoninternet.net, HTTP 404 (no app router yet)
|
||||
```
|
||||
|
||||
## 3. (later milestones) Drone, comment-bridge, dashboard, recipe enrollment
|
||||
|
||||
See `docs/enroll-recipe.md` (D5), `docs/secrets.md` (D6), `docs/runbook.md`. Added as those land.
|
||||
See `docs/enroll-recipe.md` (D5), `docs/secrets.md` (D6), `docs/runbook.md`. Each new piece of infra
|
||||
is added as another idempotent reconcile oneshot, so this install stays a single `nixos-rebuild`.
|
||||
|
||||
@ -5,9 +5,13 @@
|
||||
{
|
||||
imports = [
|
||||
./hardware.nix
|
||||
../../modules/packages.nix
|
||||
../../modules/secrets.nix
|
||||
../../modules/swarm.nix
|
||||
../../modules/abra.nix
|
||||
../../modules/proxy.nix
|
||||
../../modules/drone.nix
|
||||
../../modules/drone-runner.nix
|
||||
];
|
||||
|
||||
# --- Tailscale (ACCESS-CRITICAL: do not break, this is the only route in) ---
|
||||
|
||||
@ -1,25 +1,6 @@
|
||||
# abra — the Co-op Cloud CLI used by the harness to deploy/upgrade/backup recipes (M1+).
|
||||
# Packaged from the upstream release binary, pinned by version + hash for reproducibility (D8).
|
||||
# abra — the Co-op Cloud CLI used by the harness and the proxy/drone reconcile oneshots.
|
||||
# The package is defined as an overlay in modules/packages.nix (pkgs.abra), pinned by hash (D8).
|
||||
{ pkgs, ... }:
|
||||
let
|
||||
abra = pkgs.stdenv.mkDerivation rec {
|
||||
pname = "abra";
|
||||
version = "0.13.0-beta";
|
||||
src = pkgs.fetchurl {
|
||||
url = "https://git.coopcloud.tech/toolshed/abra/releases/download/${version}/abra_${version}_linux_amd64.tar.gz";
|
||||
sha256 = "12csk6wp1pk9cspzqfl4a6h5jdz8p055sf0ggxw9k7ljhpd5qvc6";
|
||||
};
|
||||
# Tarball has files at the root (LICENSE, README.md, abra), no common subdir.
|
||||
sourceRoot = ".";
|
||||
nativeBuildInputs = [ pkgs.autoPatchelfHook ];
|
||||
buildInputs = [ pkgs.stdenv.cc.cc.lib ];
|
||||
installPhase = ''
|
||||
runHook preInstall
|
||||
install -Dm755 abra "$out/bin/abra"
|
||||
runHook postInstall
|
||||
'';
|
||||
};
|
||||
in
|
||||
{
|
||||
environment.systemPackages = [ abra ];
|
||||
environment.systemPackages = [ pkgs.abra ];
|
||||
}
|
||||
|
||||
42
modules/drone-runner.nix
Normal file
42
modules/drone-runner.nix
Normal file
@ -0,0 +1,42 @@
|
||||
# Drone exec runner (M2). Runs on cc-ci itself (not in a container) so CI pipelines can drive
|
||||
# host `abra` to deploy real recipes onto the swarm (plan §4.2, §8: exec runner). The Drone
|
||||
# *server* is deployed separately via abra (scripts/deploy-drone.sh) as a swarm service.
|
||||
#
|
||||
# The exec runner is drone-runner-exec (the only exec runner upstream ever shipped; see
|
||||
# DECISIONS.md "CI engine"). It connects to the server over RPC at drone.ci.commoninternet.net,
|
||||
# sharing DRONE_RPC_SECRET with the server via the sops-rendered EnvironmentFile.
|
||||
{ pkgs, config, lib, ... }:
|
||||
{
|
||||
# Drone ships under the Polyform Small Business license (nixpkgs marks it unfree);
|
||||
# permitted for our internal CI use. Allow only this package.
|
||||
nixpkgs.config.allowUnfreePredicate = pkg:
|
||||
builtins.elem (lib.getName pkg) [ "drone-runner-exec" ];
|
||||
|
||||
systemd.services.drone-runner-exec = {
|
||||
description = "Drone exec runner (drives host abra/swarm)";
|
||||
after = [ "network-online.target" ];
|
||||
wants = [ "network-online.target" ];
|
||||
wantedBy = [ "multi-user.target" ];
|
||||
environment = {
|
||||
DRONE_RPC_PROTO = "https";
|
||||
DRONE_RPC_HOST = "drone.ci.commoninternet.net";
|
||||
DRONE_RUNNER_CAPACITY = "2"; # concurrency cap (plan §4.2)
|
||||
DRONE_RUNNER_NAME = "cc-ci-exec";
|
||||
# exec runner needs a writable root for build workspaces
|
||||
DRONE_RUNNER_ROOT = "/var/lib/drone-runner";
|
||||
# Pipeline commands shell out to abra/docker/git — all live in the system path.
|
||||
PATH = lib.mkForce "/run/current-system/sw/bin:/run/wrappers/bin";
|
||||
};
|
||||
serviceConfig = {
|
||||
# DRONE_RPC_SECRET comes from the sops-rendered env file (shared with the server).
|
||||
EnvironmentFile = config.sops.templates."drone-runner.env".path;
|
||||
ExecStart = "${pkgs.drone-runner-exec}/bin/drone-runner-exec";
|
||||
Restart = "always";
|
||||
RestartSec = "5s";
|
||||
StateDirectory = "drone-runner";
|
||||
# exec runner runs pipelines as this service's user; root is needed to drive docker/abra
|
||||
# and to read the abra config under /root/.abra (same as manual deploys).
|
||||
User = "root";
|
||||
};
|
||||
};
|
||||
}
|
||||
64
modules/drone.nix
Normal file
64
modules/drone.nix
Normal file
@ -0,0 +1,64 @@
|
||||
# Drone CI server = coop-cloud `drone` recipe via abra (swarm, traefik-routed at
|
||||
# drone.ci.commoninternet.net, Gitea SSO, wildcard cert / no ACME). The exec *runner* is a
|
||||
# separate host systemd service (modules/drone-runner.nix). See DECISIONS.md "CI engine"/"Drone
|
||||
# deployment shape".
|
||||
#
|
||||
# Idempotent-RECONCILE oneshot (same pattern as proxy/swarm-init): converges every boot/activation.
|
||||
# RPC + OAuth-client secrets come from sops (/run/secrets), inserted as swarm secrets here.
|
||||
{ pkgs, ... }:
|
||||
let
|
||||
giteaClientId = "ab4cdb9d-ee96-4867-875f-87384505fc52";
|
||||
reconcile = pkgs.writeShellApplication {
|
||||
name = "cc-ci-reconcile-drone";
|
||||
runtimeInputs = with pkgs; [ abra docker jq gnused gnugrep coreutils git ];
|
||||
text = ''
|
||||
DRONE_DOMAIN="drone.ci.commoninternet.net"
|
||||
ENV_FILE="$HOME/.abra/servers/default/$DRONE_DOMAIN.env"
|
||||
|
||||
if [ ! -r /run/secrets/drone_rpc_secret ] || [ ! -r /run/secrets/drone_gitea_client_secret ]; then
|
||||
echo "FATAL: drone sops secrets missing at /run/secrets (rebuild ordering?)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
abra server ls -m -n >/dev/null 2>&1 || abra server add --local -n || true
|
||||
abra recipe fetch drone -n >/dev/null
|
||||
|
||||
[ -f "$ENV_FILE" ] || abra app new drone -s default -D "$DRONE_DOMAIN" -n
|
||||
|
||||
set_env() {
|
||||
sed -i -E "/^[[:space:]]*#?[[:space:]]*$1=/d" "$ENV_FILE"
|
||||
printf '%s=%s\n' "$1" "$2" >> "$ENV_FILE"
|
||||
}
|
||||
set_env LETS_ENCRYPT_ENV ""
|
||||
set_env EXTRA_DOMAINS ""
|
||||
set_env DRONE_USER_CREATE "username:autonomic-bot,admin:true"
|
||||
set_env GITEA_DOMAIN "git.autonomic.zone"
|
||||
set_env GITEA_CLIENT_ID "${giteaClientId}"
|
||||
set_env RPC_SECRET_VERSION "v1"
|
||||
set_env CLIENT_SECRET_VERSION "v1"
|
||||
set_env DRONE_ENV_VERSION "v1"
|
||||
set_env COMPOSE_FILE '"compose.yml:compose.gitea.yml"'
|
||||
|
||||
have_secret() { docker secret ls --format '{{.Name}}' | grep -q "_$1_v1$"; }
|
||||
have_secret rpc_secret || abra app secret insert "$DRONE_DOMAIN" rpc_secret v1 /run/secrets/drone_rpc_secret -f -n
|
||||
have_secret client_secret || abra app secret insert "$DRONE_DOMAIN" client_secret v1 /run/secrets/drone_gitea_client_secret -f -n
|
||||
|
||||
abra app deploy "$DRONE_DOMAIN" -n -C
|
||||
'';
|
||||
};
|
||||
in
|
||||
{
|
||||
systemd.services.deploy-drone = {
|
||||
description = "Reconcile the Drone CI server (coop-cloud recipe, Gitea SSO) via abra";
|
||||
after = [ "deploy-proxy.service" "swarm-init.service" "docker.service" "network-online.target" ];
|
||||
requires = [ "swarm-init.service" "docker.service" ];
|
||||
wants = [ "network-online.target" ];
|
||||
wantedBy = [ "multi-user.target" ];
|
||||
environment.HOME = "/root";
|
||||
serviceConfig = {
|
||||
Type = "oneshot";
|
||||
RemainAfterExit = true;
|
||||
ExecStart = "${reconcile}/bin/cc-ci-reconcile-drone";
|
||||
};
|
||||
};
|
||||
}
|
||||
25
modules/packages.nix
Normal file
25
modules/packages.nix
Normal file
@ -0,0 +1,25 @@
|
||||
# Project package overlay. `abra` (the Co-op Cloud CLI) is exposed as `pkgs.abra` so every
|
||||
# module (systemPackages, the proxy/drone reconcile oneshots) can use the same pinned build.
|
||||
{ ... }:
|
||||
{
|
||||
nixpkgs.overlays = [
|
||||
(final: prev: {
|
||||
abra = prev.stdenv.mkDerivation rec {
|
||||
pname = "abra";
|
||||
version = "0.13.0-beta";
|
||||
src = prev.fetchurl {
|
||||
url = "https://git.coopcloud.tech/toolshed/abra/releases/download/${version}/abra_${version}_linux_amd64.tar.gz";
|
||||
sha256 = "12csk6wp1pk9cspzqfl4a6h5jdz8p055sf0ggxw9k7ljhpd5qvc6";
|
||||
};
|
||||
sourceRoot = ".";
|
||||
nativeBuildInputs = [ prev.autoPatchelfHook ];
|
||||
buildInputs = [ prev.stdenv.cc.cc.lib ];
|
||||
installPhase = ''
|
||||
runHook preInstall
|
||||
install -Dm755 abra "$out/bin/abra"
|
||||
runHook postInstall
|
||||
'';
|
||||
};
|
||||
})
|
||||
];
|
||||
}
|
||||
63
modules/proxy.nix
Normal file
63
modules/proxy.nix
Normal file
@ -0,0 +1,63 @@
|
||||
# Reverse proxy = the canonical Co-op Cloud `traefik` recipe, deployed via abra in
|
||||
# wildcard / file-provider mode (operator's pre-issued cert as ssl_cert/ssl_key swarm secrets,
|
||||
# LETS_ENCRYPT_ENV empty => NO ACME, no DNS token). See DECISIONS.md "Proxy: real coop-cloud/traefik".
|
||||
#
|
||||
# Declared as an idempotent-RECONCILE systemd oneshot (like swarm-init): it inspects current
|
||||
# state and converges every activation/boot, self-healing drift (redeploys if the stack is gone,
|
||||
# re-inserts secrets if missing). No run-once sentinel. So a from-scratch install is just
|
||||
# `nixos-rebuild switch` + operator preconditions (D8) — no manual post-steps.
|
||||
{ pkgs, ... }:
|
||||
let
|
||||
reconcile = pkgs.writeShellApplication {
|
||||
name = "cc-ci-reconcile-proxy";
|
||||
runtimeInputs = with pkgs; [ abra docker jq gnused gnugrep coreutils git ];
|
||||
text = ''
|
||||
PROXY_DOMAIN="traefik.ci.commoninternet.net"
|
||||
CERT_DIR="/var/lib/ci-certs/live"
|
||||
ENV_FILE="$HOME/.abra/servers/default/$PROXY_DOMAIN.env"
|
||||
|
||||
# Fail visibly (failed unit) if the operator cert is missing — do NOT silently skip.
|
||||
if [ ! -r "$CERT_DIR/fullchain.pem" ] || [ ! -r "$CERT_DIR/privkey.pem" ]; then
|
||||
echo "FATAL: wildcard cert missing at $CERT_DIR (operator precondition)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
abra server ls -m -n >/dev/null 2>&1 || abra server add --local -n || true
|
||||
abra recipe fetch traefik -n >/dev/null
|
||||
|
||||
[ -f "$ENV_FILE" ] || abra app new traefik -s default -D "$PROXY_DOMAIN" -n
|
||||
|
||||
set_env() {
|
||||
sed -i -E "/^[[:space:]]*#?[[:space:]]*$1=/d" "$ENV_FILE"
|
||||
printf '%s=%s\n' "$1" "$2" >> "$ENV_FILE"
|
||||
}
|
||||
set_env LETS_ENCRYPT_ENV ""
|
||||
set_env WILDCARDS_ENABLED "1"
|
||||
set_env SECRET_WILDCARD_CERT_VERSION "v1"
|
||||
set_env SECRET_WILDCARD_KEY_VERSION "v1"
|
||||
set_env COMPOSE_FILE '"compose.yml:compose.wildcard.yml"'
|
||||
|
||||
have_secret() { docker secret ls --format '{{.Name}}' | grep -q "_$1_v1$"; }
|
||||
have_secret ssl_cert || abra app secret insert "$PROXY_DOMAIN" ssl_cert v1 "$CERT_DIR/fullchain.pem" -f -n
|
||||
have_secret ssl_key || abra app secret insert "$PROXY_DOMAIN" ssl_key v1 "$CERT_DIR/privkey.pem" -f -n
|
||||
|
||||
# Converge the stack (idempotent: no-op if already at desired state).
|
||||
abra app deploy "$PROXY_DOMAIN" -n -C
|
||||
'';
|
||||
};
|
||||
in
|
||||
{
|
||||
systemd.services.deploy-proxy = {
|
||||
description = "Reconcile the Co-op Cloud traefik proxy (wildcard/no-ACME) via abra";
|
||||
after = [ "swarm-init.service" "docker.service" "network-online.target" ];
|
||||
requires = [ "swarm-init.service" "docker.service" ];
|
||||
wants = [ "network-online.target" ];
|
||||
wantedBy = [ "multi-user.target" ];
|
||||
environment.HOME = "/root";
|
||||
serviceConfig = {
|
||||
Type = "oneshot";
|
||||
RemainAfterExit = true;
|
||||
ExecStart = "${reconcile}/bin/cc-ci-reconcile-proxy";
|
||||
};
|
||||
};
|
||||
}
|
||||
@ -2,7 +2,7 @@
|
||||
# ed25519 SSH host key as the age identity (no separate key file to manage on the box).
|
||||
# Encrypted material lives in ../secrets/*.yaml, committed and readable only by recipients
|
||||
# listed in /.sops.yaml (host key + off-box master recovery key).
|
||||
{ ... }:
|
||||
{ config, ... }:
|
||||
{
|
||||
sops = {
|
||||
defaultSopsFile = ../secrets/secrets.yaml;
|
||||
@ -11,8 +11,19 @@
|
||||
# Do not also look for a GPG key.
|
||||
gnupg.sshKeyPaths = [ ];
|
||||
|
||||
# M0 proof secret — confirms the decrypt path works end to end. Real infra secrets
|
||||
# (Drone RPC, webhook HMAC, OAuth, registry creds) are added in their milestones.
|
||||
# M0 proof secret — confirms the decrypt path works end to end.
|
||||
secrets.test_secret = { };
|
||||
|
||||
# M2 Drone (A2 internal secrets). drone_rpc_secret is shared between the swarm-deployed
|
||||
# Drone server (inserted as the `rpc_secret` swarm secret by scripts/deploy-drone.sh) and
|
||||
# the host exec runner (read via the env template below). drone_gitea_client_secret is the
|
||||
# Gitea OAuth app secret, inserted as the server's `client_secret` swarm secret.
|
||||
secrets.drone_rpc_secret = { };
|
||||
secrets.drone_gitea_client_secret = { };
|
||||
|
||||
# EnvironmentFile for the host exec runner: DRONE_RPC_SECRET rendered from the sops secret.
|
||||
templates."drone-runner.env".content = ''
|
||||
DRONE_RPC_SECRET=${config.sops.placeholder.drone_rpc_secret}
|
||||
'';
|
||||
};
|
||||
}
|
||||
|
||||
@ -1,60 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Reproducibly deploy the canonical Co-op Cloud `traefik` recipe as cc-ci's reverse proxy,
|
||||
# in wildcard / file-provider mode — serving the operator's pre-issued wildcard cert, with
|
||||
# NO ACME and NO DNS token on the box (see DECISIONS.md "Proxy: real coop-cloud/traefik").
|
||||
#
|
||||
# Idempotent: safe to re-run. Run as root on cc-ci (abra drives the local Docker swarm).
|
||||
# ssh cc-ci 'bash /root/cc-ci/scripts/deploy-proxy.sh'
|
||||
#
|
||||
# Prereqs (declared in the flake): docker + single-node swarm + `proxy` overlay (modules/swarm.nix),
|
||||
# abra (modules/abra.nix), and the wildcard cert at /var/lib/ci-certs/live/ (operator-provided).
|
||||
set -euo pipefail
|
||||
|
||||
PROXY_DOMAIN="${PROXY_DOMAIN:-traefik.ci.commoninternet.net}"
|
||||
CERT_DIR="${CERT_DIR:-/var/lib/ci-certs/live}"
|
||||
ENV_FILE="$HOME/.abra/servers/default/${PROXY_DOMAIN}.env"
|
||||
|
||||
export PATH=/run/current-system/sw/bin:"$PATH"
|
||||
|
||||
echo "==> ensure local abra server"
|
||||
abra server ls -m -n >/dev/null 2>&1 || abra server add --local -n || true
|
||||
|
||||
echo "==> fetch traefik recipe"
|
||||
abra recipe fetch traefik -n >/dev/null
|
||||
|
||||
if [ ! -f "$ENV_FILE" ]; then
|
||||
echo "==> create traefik app ($PROXY_DOMAIN)"
|
||||
abra app new traefik -s default -D "$PROXY_DOMAIN" -n
|
||||
fi
|
||||
|
||||
echo "==> configure wildcard / no-ACME env"
|
||||
# Set each var deterministically: drop any existing (commented or not) line, then append.
|
||||
# Empty LETS_ENCRYPT_ENV => the traefik router uses no cert resolver => no ACME ever fires.
|
||||
set_env() {
|
||||
local key="$1" val="$2"
|
||||
sed -i -E "/^[[:space:]]*#?[[:space:]]*${key}=/d" "$ENV_FILE"
|
||||
printf '%s=%s\n' "$key" "$val" >> "$ENV_FILE"
|
||||
}
|
||||
set_env LETS_ENCRYPT_ENV ""
|
||||
set_env WILDCARDS_ENABLED "1"
|
||||
set_env SECRET_WILDCARD_CERT_VERSION "v1"
|
||||
set_env SECRET_WILDCARD_KEY_VERSION "v1"
|
||||
set_env COMPOSE_FILE '"compose.yml:compose.wildcard.yml"'
|
||||
echo " env written: $ENV_FILE"
|
||||
|
||||
echo "==> insert wildcard cert secrets (v1) from $CERT_DIR (idempotent)"
|
||||
# Check the actual swarm secret (generated name ${STACK_NAME}_<name>_v1), not abra's
|
||||
# recipe-defined list (which always shows the names with "created on server":"false").
|
||||
have_secret() { docker secret ls --format '{{.Name}}' | grep -q "_${1}_v1\$"; }
|
||||
# Insert from file (-f) so the multi-line PEM is read verbatim, not arg-parsed.
|
||||
if ! have_secret ssl_cert; then
|
||||
abra app secret insert "$PROXY_DOMAIN" ssl_cert v1 "$CERT_DIR/fullchain.pem" -f -n
|
||||
fi
|
||||
if ! have_secret ssl_key; then
|
||||
abra app secret insert "$PROXY_DOMAIN" ssl_key v1 "$CERT_DIR/privkey.pem" -f -n
|
||||
fi
|
||||
|
||||
echo "==> deploy traefik"
|
||||
abra app deploy "$PROXY_DOMAIN" -n -C
|
||||
|
||||
echo "==> done"
|
||||
Reference in New Issue
Block a user