cc-ci/docs/secrets.md

# Secrets model & rotation (D6)

cc-ci handles three classes of secret in deliberately different ways (plan §4.4). **No plaintext
secret ever lives in git, logs, or the results UI** — only sops-encrypted ciphertext and
references-by-location. The Adversary's leak test greps published Drone logs + the dashboard for
known secret patterns and any generated app password; it must find nothing.

## Decryption chain (sops-nix)

- Infra secrets live sops-encrypted in `secrets/secrets.yaml` (committed). `/.sops.yaml` lists two
  age recipients: the **host key** (`age1h90ut…`, derived from cc-ci's SSH host key via ssh-to-age)
  and an off-box **master recovery key** (`age1cmk26t…`; its private half is kept only at
  `/srv/cc-ci/.sops/master-age.txt` on the build host, never in the repo).
- On cc-ci, `sops-nix` decrypts at activation using the host's ed25519 SSH host key as the age
  identity (`sops.age.sshKeyPaths = ["/etc/ssh/ssh_host_ed25519_key"]`), materialising each secret at
  `/run/secrets/<name>` (mode 0400 root). No extra key file to manage on the box.
- Swarm services don't read `/run/secrets` directly; the reconcile oneshots copy each into a **docker
  swarm secret** (`docker secret create … /run/secrets/<name>`) which the service mounts. abra-managed
  apps use `abra app secret …`.

## Class A1 — external inputs (operator-provided; the loop CANNOT create them)

| Secret | Location | Rotation |
|---|---|---|
| Tailscale auth key | `/srv/cc-ci/.testenv` (sandbox) | operator re-issues; re-run `tailscale up` |
| cc-ci SSH root key | `~/.ssh/cc-ci-root-ed25519` (sandbox) | operator re-keys `authorized_keys` |
| Gitea bot creds | `/srv/cc-ci/.testenv` (`GITEA_USERNAME/PASSWORD`) | operator resets; update `.testenv` |
| **Wildcard TLS cert** | cc-ci `/var/lib/ci-certs/live/{fullchain,privkey}.pem` | **operator** re-issues (LE DNS-01/Gandi, ~90d, next ~2026-08-24) — see below |
| Registry pull creds (if needed) | sops `secrets/secrets.yaml` | operator-provided |

A missing/invalid A1 secret is a `## Blocked` condition — the agent never invents or works around it,
and **never** runs ACME/DNS-01 for commoninternet.net.

**Wildcard cert rotation (manual, operator + agent):**
1. Operator re-issues the SAN cert (`*.ci.commoninternet.net` + `ci.commoninternet.net`) out-of-band
   and writes it to `/var/lib/ci-certs/live/{fullchain,privkey}.pem` on cc-ci.
2. Bump `SECRET_WILDCARD_CERT_VERSION` / `SECRET_WILDCARD_KEY_VERSION` on the traefik app env
   (modules/proxy.nix) so the next reconcile inserts the new cert as a fresh swarm secret version.
3. `nixos-rebuild switch` (re-runs the proxy reconcile → re-inserts + redeploys traefik). One cert
   covers every per-run subdomain (SNI), so no per-app cert work.

## Class A2 — internal infra secrets (the loop GENERATES + manages; never a blocker)

All sops-encrypted in `secrets/secrets.yaml`, decrypted to `/run/secrets/<name>`:

| Secret | Used by | Generate |
|---|---|---|
| `drone_rpc_secret` | Drone server ↔ exec runner RPC | `openssl rand -hex 32` |
| `drone_gitea_client_secret` | Drone↔Gitea OAuth app | from the Gitea OAuth app creation |
| `bridge_webhook_hmac` | comment-bridge webhook HMAC | `openssl rand -hex 32` |
| `bridge_drone_token` | bridge + dashboard → Drone API | minted Drone user token |
| `bridge_gitea_token` | bridge → Gitea API (poll/comment) | minted Gitea token (bot) |
| `restic_password` | backup-bot-two restic repo | **abra-generated** (`abra app secret generate`, kept stable across reconciles) |

**Rotate an A2 secret** (e.g. `bridge_webhook_hmac`):
1. `set -a; . /srv/cc-ci/.testenv; set +a` (for the editor key, not echoed).
2. In the repo: `sops secrets/secrets.yaml` → replace the value (or `openssl rand -hex 32 | …`),
   save. (Re-encrypts to both recipients automatically per `.sops.yaml`.)
3. For swarm-secret-backed values, **bump the consuming app's secret version** so the reconcile
   re-creates the swarm secret (docker swarm secrets are immutable): e.g. drone `RPC_SECRET_VERSION`
   v1→v2 (modules/drone.nix), bridge `cc_ci_bridge_*_v<n>` (modules/bridge.nix). Update both ends
   (server + runner share `drone_rpc_secret`).
4. `git commit` + push, sync to host, `nixos-rebuild switch` → reconcile re-inserts + redeploys.
5. Verify: the consuming service is healthy and re-auth works (e.g. a fresh build triggers).

**Re-key sops recipients** (e.g. cc-ci host re-provisioned → new host age key): add the new
`age1…` to `/.sops.yaml`, `sops updatekeys secrets/secrets.yaml` (run from the build host, which
holds the master identity), commit. The master recovery key lets you re-encrypt even if the host key
is lost.

## Class B — recipe app secrets (the harness generates per run; NEVER a blocker)

- **Generated at install:** `abra app secret generate <app> --all` (+ any deterministic test fixtures
  the harness chooses) when the recipe deploys.
- **Persisted for the run:** the same generated values survive install → upgrade → backup/restore
  because abra/swarm holds them keyed by the per-run app name (`<recipe[:4]>-<6hex>`); the harness
  re-reads them between stages. Concurrent runs are isolated by the unique per-run app name (and
  MAX_TESTS=1 means no concurrency anyway).
- **Destroyed at teardown:** the same teardown that removes the app/volumes runs
  `abra app secret remove <app> --all` (+ docker-secret cleanup by stack name as a fallback). Nothing
  generated for a run outlives it.

## No-plaintext guarantees

- Secrets are referenced by `/run/secrets/<name>` path or read inline (e.g.
  `PGPASSWORD=$(cat /run/secrets/…)` *inside* the app container), never printed by the harness.
- abra does not echo generated secret values; reconciles redirect secret-generate stdout to
  `/dev/null`.
- The results dashboard renders run status only (no log bodies); per-run logs live in Drone's UI.
- Adversary leak test: greps published Drone logs + the dashboard for the known infra-secret values
  and any generated app password → must be zero. (Baseline + recipe-CI log scans: clean.)