recipe-maintainers/cc-ci-orchestrator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

opencode UI on 443 via traefik (Host routing), nginx basic auth on the docker bridge

Browse Source 
Operator: standard 443, routed by domain. Traefik already owns 443 with the
*.ci.commoninternet.net cert, so opencode-ui-route.service drops a dynamic
config into the traefik stack's watched file-providers volume routing
oc.ci.commoninternet.net -> http://172.18.0.1:8080, where nginx (reachable
only on docker_gwbridge) enforces the basic auth and logs 401s with the
real client IP (real_ip from X-Forwarded-For). The fail2ban nginx jail bans
in DOCKER-USER, since that traffic is forwarded, not INPUT. 8443 and the
ACME-host variant are gone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqkQq3CDmFWcQ7u1LzoyRz
This commit is contained in:
notplants
2026-09-07 20:49:53 +00:00
co-authored by Claude Fable 5.1
parent 8f7f125fc0
commit b23ad96616
3 changed files with 85 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+10 -5
View File
@@ -157,8 +157,12 @@ Everything in this section is **outside git**. Do it as root over SSH, in this o
### 4a. No tailscale
The combined host is NOT on the tailnet (operator decision 2026-09-07): ssh is key-only on the
public IP, the CI front doors are public via traefik, and the opencode UI is public on 8443
behind TLS + basic auth, all guarded by fail2ban (`nix/hosts/cc-ci/configuration.nix`).
public IP, the CI front doors are public via traefik, and the opencode UI is public on the same
443 — traefik routes `oc.ci.commoninternet.net` (a dynamic-config file that
`opencode-ui-route.service` drops into the traefik stack's file-providers volume) to an nginx
vhost on the docker bridge that enforces basic auth. fail2ban guards sshd and that login
(`nix/hosts/cc-ci/configuration.nix`; the nginx jail bans in the DOCKER-USER chain because the
traffic is docker-forwarded, not host INPUT).
### 4b. The CI server's checkout and its one out-of-band secret
@@ -210,7 +214,7 @@ Then the files below (`install -m600 -o loops -g users`):
| `~loops/.ssh/tangled-ed25519` | optional, tangled.org mirrors | old host |
| `~loops/.claude/` | Claude Code auth + settings + the orchestrator session history | old host (`rsync -a`); fresh: `claude auth login` as loops (device code, interactive) |
| `~loops/.local/share/opencode/auth.json`, `~loops/.config/opencode/` | opencode provider auth (the weekly upgrader runs on opencode) | old host; fresh: `opencode auth login` |
| `/etc/nginx/oc-htpasswd` | basic auth for the opencode UI (`https://oc.ci.commoninternet.net:8443`, TLS from the CI wildcard cert); **nginx refuses to start without it**, and its config check runs as the `nginx` user, so `root:nginx 0640` (the `nginx` group exists after the first rebuild — fix ownership then and `systemctl restart nginx`) | old host (`/secrets/files/oc-basic-auth.txt` has the plaintext), or a new `oc:<bcrypt>` line via `mkpasswd -m bcrypt` |
| `/etc/nginx/oc-htpasswd` | basic auth for the opencode UI (`https://oc.ci.commoninternet.net`, via traefik); **nginx refuses to start without it**, and its config check runs as the `nginx` user, so `root:nginx 0640` (the `nginx` group exists after the first rebuild — fix ownership then and `systemctl restart nginx`) | old host (`/secrets/files/oc-basic-auth.txt` has the plaintext), or a new `oc:<bcrypt>` line via `mkpasswd -m bcrypt` |
`~loops/.ssh/config` is written by the activation script on first rebuild (`Host cc-ci` →
`127.0.0.1`, `git.autonomic.zone`, `tangled.org`); it is not overwritten if present.
@@ -259,8 +263,9 @@ curl -s --resolve drone.ci.commoninternet.net:443:127.0.0.1 -o /dev/null -w '%{h
dig +short @<ip> ns-acme.commoninternet.net # acme-dns answering on the public 53
```
The opencode UI: `curl -sk --resolve oc.ci.commoninternet.net:8443:<ip> -o /dev/null -w '%{http_code}' https://oc.ci.commoninternet.net:8443/`
→ 401 without credentials, 200 with `-u oc:<password>`.
The opencode UI: `curl -s --resolve oc.ci.commoninternet.net:443:<ip> -o /dev/null -w '%{http_code}' https://oc.ci.commoninternet.net/`
→ 401 without credentials, 200 with `-u oc:<password>`; `fail2ban-client status nginx-http-auth`
counts the failures.
When it is healthy: `sudo nixos-rebuild switch --flake .#cc-ci` (same config, now also the boot
default). **If you are migrating from another host, do §6 before letting it serve anything**: right
nix/hosts/cc-ci/configuration.nix
+13 -9
View File
@@ -27,13 +27,14 @@
# ---- orchestrator identity --------------------------------------------------------------
# The CI server is this very host, so `ssh cc-ci` goes to loopback (the module default).
cc-ci-orchestrator.ciSshHost = "127.0.0.1";
# The opencode UI rides the CI server's Let's Encrypt wildcard (*.ci.commoninternet.net).
# The opencode UI: traefik (public 443, the *.ci.commoninternet.net cert) → nginx basic auth.
cc-ci-orchestrator.opencodeUiHost = "oc.ci.commoninternet.net";
cc-ci-orchestrator.opencodeUiAcmeHost = "ci.commoninternet.net";
cc-ci-orchestrator.opencodeUiTraefikProvidersDir =
"/var/lib/docker/volumes/traefik_ci_commoninternet_net_file-providers/_data";
# ---- no tailscale on this host (operator 2026-09-07) --------------------------------------
# Everything is reached over the public IP: ssh (keys only), the CI front doors via traefik,
# and the opencode UI on 8443 behind TLS + basic auth. fail2ban below guards the two logins.
# and the opencode UI on 443 (traefik → nginx basic auth). fail2ban below guards the two logins.
# ---- ssh ----------------------------------------------------------------------------------
services.openssh = {
@@ -50,16 +51,18 @@
services.openssh.settings.KbdInteractiveAuthentication = false;
# ---- firewall -------------------------------------------------------------------------------
# 80/443 (traefik) and 53 (acme-dns) are opened by the cc-ci-server module. 8443 is the
# opencode UI (nginx, LE cert, basic auth — see orchestrator-host.nix).
# 80/443 (traefik) and 53 (acme-dns) are opened by the cc-ci-server module; the opencode UI
# rides 443 through traefik (orchestrator-host.nix), so only ssh is opened here.
networking.firewall = {
enable = true;
allowedTCPPorts = [ 22 8443 ];
allowedTCPPorts = [ 22 ];
};
# ---- fail2ban: sshd (password auth is off, this stops the log noise and slow brute force) and
# the opencode UI's basic auth (nginx logs 401s to its error log; the built-in
# nginx-http-auth filter matches them).
# the opencode UI's basic auth (nginx logs 401s with the real client IP to its error log; the
# built-in nginx-http-auth filter matches them). Those clients arrive through traefik's
# docker-published 443, which iptables FORWARDs rather than INPUTs, so the ban for that jail
# goes into the DOCKER-USER chain — an INPUT rule would never see the traffic.
services.fail2ban = {
enable = true;
maxretry = 5;
@@ -71,7 +74,8 @@
filter = "nginx-http-auth";
logpath = "/var/log/nginx/error.log";
backend = "auto";
port = "8443";
banaction = "iptables-allports";
chain = "DOCKER-USER";
};
};
nix/modules/orchestrator-host.nix
+62 -35
View File
@@ -29,12 +29,25 @@ in
'';
};
opencodeUiPort = lib.mkOption {
opencodeUiBackendPort = lib.mkOption {
type = lib.types.port;
default = 8443;
default = 8080;
description = ''
TLS port of the nginx front door for the opencode web UI. Not 443: on the combined host
Traefik (docker swarm) owns 80/443. The host decides whether to open it in the firewall.
Plain-HTTP port nginx listens on for the opencode UI, reachable ONLY from the docker
bridge (firewall rule on docker_gwbridge). Traefik — which owns the public 443 on the
combined host — terminates TLS for opencodeUiHost and forwards here; nginx adds the basic
auth and logs failures for fail2ban with the real client IP.
'';
};
opencodeUiTraefikProvidersDir = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/docker/volumes/traefik_ci_commoninternet_net_file-providers/_data";
description = ''
Host path of the traefik stack's file-providers directory (watched by traefik). When set,
a dynamic-config file routing opencodeUiHost on the `web-secure` entrypoint to nginx is
written there after deploy-proxy. null = no traefik route (serve some other way).
'';
};
@@ -44,17 +57,6 @@ in
description = "nginx server_name for the opencode web UI (TLS + basic auth).";
};
opencodeUiAcmeHost = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "ci.commoninternet.net";
description = ''
Name of a `security.acme.certs` entry whose certificate covers opencodeUiHost (the
combined host has the *.ci.commoninternet.net wildcard). When set, nginx serves that
certificate; when null, the self-signed pair /etc/nginx/oc-selfsigned.{crt,key} staged
out of band is used.
'';
};
};
config = {
@@ -188,36 +190,61 @@ SSHCFG
path = [ pkgs.bash pkgs.coreutils pkgs.git pkgs.python3 pkgs.openssh pkgs.tmux pkgs.nettools ];
};
# ---- nginx front door for the opencode UI --------------------------------------------
# TLS (LE via opencodeUiAcmeHost, else the self-signed pair below) + HTTP basic auth. The
# opencode web UI has no auth of its own and can drive agent sessions, so the htpasswd is
# mandatory. Files created out of band (a store path would be world readable) — nginx
# FAILS TO START without them, and its config check runs as the nginx user:
# /etc/nginx/oc-htpasswd root:nginx 0640 (`oc:<bcrypt>`; plaintext kept in /secrets)
# /etc/nginx/oc-selfsigned.crt root:nginx 0644 (only when opencodeUiAcmeHost is null)
# /etc/nginx/oc-selfsigned.key root:nginx 0640
# Rotate the password with:
# printf 'oc:%s\n' "$(mkpasswd -m bcrypt "$P")" > /etc/nginx/oc-htpasswd && systemctl reload nginx
# nginx must be able to read the ACME-issued key (the acme group owns it).
users.users.nginx.extraGroups = lib.mkIf (cfg.opencodeUiAcmeHost != null)
[ config.security.acme.certs.${cfg.opencodeUiAcmeHost}.group ];
# ---- nginx: basic auth for the opencode UI, behind traefik -----------------------------
# Traefik (public 443, the CI wildcard cert) routes opencodeUiHost to this plain-HTTP vhost
# on the docker bridge address. nginx enforces HTTP basic auth (the opencode web UI has no
# auth of its own and can drive agent sessions), and — via real_ip from traefik's
# X-Forwarded-For — logs the CLIENT address on a 401, which is what the fail2ban jail bans.
# The htpasswd is created out of band (a store path would be world readable); nginx FAILS TO
# START without it, and its config check runs as the nginx user:
# /etc/nginx/oc-htpasswd root:nginx 0640 (`oc:<bcrypt>`; plaintext kept in /secrets)
# Rotate with: printf 'oc:%s\n' "$(mkpasswd -m bcrypt "$P")" > /etc/nginx/oc-htpasswd && systemctl reload nginx
services.nginx = {
enable = true;
recommendedProxySettings = true;
virtualHosts.${cfg.opencodeUiHost} = {
listen = [ { addr = "0.0.0.0"; port = cfg.opencodeUiPort; ssl = true; } ];
# onlySSL flags the vhost as SSL so the module renders ssl_certificate for the listener.
onlySSL = true;
useACMEHost = cfg.opencodeUiAcmeHost;
sslCertificate = lib.mkIf (cfg.opencodeUiAcmeHost == null) "/etc/nginx/oc-selfsigned.crt";
sslCertificateKey = lib.mkIf (cfg.opencodeUiAcmeHost == null) "/etc/nginx/oc-selfsigned.key";
listen = [ { addr = "0.0.0.0"; port = cfg.opencodeUiBackendPort; } ];
basicAuthFile = "/etc/nginx/oc-htpasswd";
extraConfig = ''
# traefik sits on the docker networks (ingress 10.0.0.0/24, gwbridge 172.18.0.0/16)
set_real_ip_from 172.16.0.0/12;
set_real_ip_from 10.0.0.0/8;
real_ip_header X-Forwarded-For;
'';
locations."/" = {
proxyPass = "http://127.0.0.1:4096";
proxyWebsockets = true;
};
};
};
# Only docker's bridge may reach the plain-HTTP backend; the public interface stays closed.
networking.firewall.interfaces.docker_gwbridge.allowedTCPPorts = [ cfg.opencodeUiBackendPort ];
# The traefik side of the route: a dynamic-config file in the stack's watched providers dir.
systemd.services.opencode-ui-route = lib.mkIf (cfg.opencodeUiTraefikProvidersDir != null) {
description = "traefik route: ${cfg.opencodeUiHost} -> nginx (opencode UI basic auth)";
wantedBy = [ "multi-user.target" ];
after = [ "deploy-proxy.service" "docker.service" ];
wants = [ "deploy-proxy.service" ];
serviceConfig = { Type = "oneshot"; RemainAfterExit = true; };
script = ''
install -d -m 0755 "${cfg.opencodeUiTraefikProvidersDir}"
install -m 0644 ${pkgs.writeText "opencode-ui.yml" ''
# Written by opencode-ui-route.service (nix/modules/orchestrator-host.nix). Do not edit.
http:
routers:
opencode-ui:
rule: Host(`${cfg.opencodeUiHost}`)
entryPoints: [web-secure]
service: opencode-ui
tls: {}
services:
opencode-ui:
loadBalancer:
servers:
- url: http://172.18.0.1:${toString cfg.opencodeUiBackendPort}
''} "${cfg.opencodeUiTraefikProvidersDir}/opencode-ui.yml"
'';
};
};
}