/secrets is the authoritative location for every secret, incl. the ssh host keys

Operator rule: secrets live in /secrets and consumers reach them from there.
Three subdirectories with the ownership each consumer needs — files/ (loops),
host/ (root: ssh host keys + sops age identity), nginx/ (root:nginx: the
opencode UI htpasswd) — under a 0711 /secrets so nginx can traverse to its own
without the directory being listable.

sshd's hostKeys and sops-nix's sshKeyPaths/keyFile are pointed at /secrets
DIRECTLY rather than through symlinks: the ed25519 host key is load-bearing
beyond ssh, since its age identity (age1tmvg…) is a recipient of cc-ci-secrets,
and a dangling symlink would let sshd write a NEW key and silently make every
cc-ci secret undecryptable. The /etc/ssh symlinks are added for discoverability
only, so nothing depends on activation ordering.

nginx's htpasswd path becomes an option (opencodeUiHtpasswdFile) defaulting
under /secrets, rather than a hard-coded /etc/nginx path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqkQq3CDmFWcQ7u1LzoyRz

nix/hosts/cc-ci/configuration.nix

@@ -38,10 +38,47 @@
   # Everything is reached over the public IP: ssh (keys only), the CI front doors via traefik,
   # and the opencode UI on 443 (traefik → nginx basic auth). fail2ban below guards the two logins.
 
+  # ---- /secrets is THE authoritative location for this host's secret material ---------------
+  # Operator rule (2026-09-08): every secret lives under /secrets; anything that needs one either
+  # reads it from there directly (where we own the path) or reaches it by a symlink (where the
+  # consuming software fixes the path). One directory to audit, back up, and reason about.
+  #
+  #   /secrets/files   loops:users 0700  the agent's secrets (testenv, opencode auth, its ssh keys)
+  #   /secrets/host    root:root   0700  host identity: ssh host keys + the sops age identity
+  #   /secrets/nginx   root:nginx  0750  the opencode UI htpasswd (nginx must read it)
+  #
+  # /secrets itself is 0711: traversable so nginx can reach its own subdirectory, not listable.
+  systemd.tmpfiles.rules = [
+    "d /secrets       0711 root root  -"
+    "d /secrets/host  0700 root root  -"
+    "d /secrets/nginx 0750 root nginx -"
+    # Convenience symlinks at the conventional paths, so an operator (or a tool that assumes the
+    # usual location) still finds the host keys. NOT load-bearing: sshd and sops-nix below are
+    # pointed at /secrets directly, precisely so nothing depends on symlink/activation ordering.
+    "L+ /etc/ssh/ssh_host_ed25519_key     - - - - /secrets/host/ssh_host_ed25519_key"
+    "L+ /etc/ssh/ssh_host_ed25519_key.pub - - - - /secrets/host/ssh_host_ed25519_key.pub"
+    "L+ /etc/ssh/ssh_host_rsa_key         - - - - /secrets/host/ssh_host_rsa_key"
+    "L+ /etc/ssh/ssh_host_rsa_key.pub     - - - - /secrets/host/ssh_host_rsa_key.pub"
+  ];
+
+  # sops-nix: the cc-ci server module hard-codes /etc/ssh/... and /var/lib/sops-nix/key.txt.
+  # Override both to the authoritative copies. THE ED25519 HOST KEY IS LOAD-BEARING BEYOND SSH:
+  # its age identity (age1tmvg…) is a recipient of cc-ci-secrets, so replacing or regenerating it
+  # makes every cc-ci secret undecryptable. Move it, never re-create it.
+  sops.age.sshKeyPaths = lib.mkForce [ "/secrets/host/ssh_host_ed25519_key" ];
+  sops.age.keyFile = lib.mkForce "/secrets/host/sops-age-key.txt";
+
   # ---- ssh ----------------------------------------------------------------------------------
   services.openssh = {
     enable = true;
     settings.PermitRootLogin = "yes";
+    # Host keys live in /secrets (above). sshd is pointed here directly rather than through the
+    # /etc/ssh symlinks, so it can never write a NEW key through a dangling link — that would
+    # silently rotate the age identity that decrypts cc-ci-secrets.
+    hostKeys = [
+      { path = "/secrets/host/ssh_host_ed25519_key"; type = "ed25519"; }
+      { path = "/secrets/host/ssh_host_rsa_key"; type = "rsa"; bits = 4096; }
+    ];
   };
   # Root keys: PUBLIC keys, tracked deliberately in ./ssh-keys (one per line, blank lines ok).
   users.users.root.openssh.authorizedKeys.keys =

nix/modules/orchestrator-host.nix

@@ -62,6 +62,18 @@ in
       description = "nginx server_name for the opencode web UI (TLS + basic auth).";
     };
 
+    opencodeUiHtpasswdFile = lib.mkOption {
+      type = lib.types.str;
+      default = "/secrets/nginx/oc-htpasswd";
+      description = ''
+        htpasswd file for the opencode UI (`oc:<bcrypt>`), created out of band — a store path
+        would be world-readable. Default is under /secrets, the authoritative location for this
+        host's secrets; it must be readable by the `nginx` user (root:nginx 0640 in a directory
+        nginx can traverse). **nginx refuses to start if it is missing**, and its config check
+        runs as the nginx user, so a root-only file fails the check even though the path exists.
+      '';
+    };
+
   };
 
   config = {
@@ -204,7 +216,7 @@ SSHCFG
       recommendedProxySettings = true;
       virtualHosts.${cfg.opencodeUiHost} = {
         listen = [ { addr = "0.0.0.0"; port = cfg.opencodeUiBackendPort; } ];
-        basicAuthFile = "/etc/nginx/oc-htpasswd";
+        basicAuthFile = cfg.opencodeUiHtpasswdFile;
         extraConfig = ''
           # traefik sits on the docker networks (ingress 10.0.0.0/24, gwbridge 172.18.0.0/16)
           set_real_ip_from 172.16.0.0/12;