recipe-maintainers/agent-orchestrator
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

gateway-domain: give a tailnet box a real public domain

Browse Source 
An agent on a box with no public IP frequently needs a reachable HTTPS URL —
an OAuth callback, a webhook receiver, a demo link. The testing gateway
already holds a wildcard record for *.gtest.commoninternet.net and forwards
by SNI, but nothing here knew that, so every agent had to be told by hand.

    tools/gateway-domain.py add myapp
    #   myapp.gtest.commoninternet.net  ->  100.84.190.30

The backend defaults to the running box's own tailscale IP, which is the case
that comes up almost every time.

The admin password comes from gateway.admin_password in the secret store; the
tool reads it itself, so no caller handles the value and there is no second
copy to drift or get committed.

Two things the tool refuses to do, both learned by doing them:

Backends must be a literal IPv4 address. The gateway's validate_ip accepts a
hostname, but put_domain/remove_domain only match lines whose backend is
numeric ([\d.:]+). A hostname mapping can therefore be written once and never
updated or removed through the admin UI — it becomes an orphan that only a
hand-edit of tunnel_map.conf clears. One got created while testing this.

Verification re-reads the mapping table instead of trusting the POST body.
The admin app mutates its in-memory dict and renders that, so a delete that
silently failed still renders as success. Checking the response alone
reported "removed" for an entry that was still on disk.

skills/gateway-domain/ carries the rest: that the gateway does NOT terminate
TLS (your box serves the cert for that name), how ACME still works through
it, that only 22/80/443 are open at the edge, and how to recover if an
interrupted e2e run leaves the admin password reseeded.
This commit is contained in:
notplants
2026-08-20 17:10:28 +00:00
parent bb03bed218
commit 22bd897a86
3 changed files with 361 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+16
View File
@@ -379,6 +379,22 @@ so reviewers read the pre-fixup code and no interdiff exists — with no warning
push itself succeeded. Always: push, resubmit, then reply with the printed interdiff URL. See
`machine-docs/PR-WORKFLOW.md`, "A push does NOT advance the round".
## The testing gateway — public domains for tailnet boxes
A box on the tailnet with no public IP can still have a real HTTPS domain: the shared testing
gateway holds a wildcard record for `*.gtest.commoninternet.net` and forwards by SNI.
```bash
python3 engine/tools/gateway-domain.py add myapp # -> myapp.gtest.commoninternet.net -> this box
python3 engine/tools/gateway-domain.py list
python3 engine/tools/gateway-domain.py remove myapp
```
The admin password is `gateway.admin_password` in the secret store below; the tool reads it
itself. **The gateway does not terminate TLS** — it proxies the encrypted stream, so your box
serves the certificate for that name. Full detail, including how to get a cert and why hostname
backends are refused, is in the `gateway-domain` skill (`skills/gateway-domain/SKILL.md`).
## Secrets — one encrypted store, never in git
**Every credential on an orchestrator host lives in one sops+age encrypted file. Do not put a
skills/gateway-domain/SKILL.md
+129
View File
@@ -0,0 +1,129 @@
---
name: gateway-domain
description: Give a tailnet box a real public HTTPS domain (<name>.gtest.commoninternet.net) by mapping it on the shared testing gateway. Use when an agent needs a publicly reachable URL for a box with no public IP — an OAuth callback, a webhook receiver, a demo link, an ACME challenge. Covers the add/remove tool, where the admin password lives, and the traps (your box serves the TLS cert, not the gateway; hostname backends are unremovable).
---
# Giving your box a public domain
Your machine is on the tailnet with no public IP. You need a real HTTPS URL for it. The
**testing gateway** already owns a wildcard DNS record, so every name under
`*.gtest.commoninternet.net` resolves to it. Map your name to your tailnet IP and it forwards
matching traffic to you.
```bash
python3 engine/tools/gateway-domain.py add myapp
# myapp.gtest.commoninternet.net -> 100.84.190.30
```
That is the whole happy path. The backend defaults to **this box's own tailscale IP**, so run
it on the machine that will serve the domain.
```bash
python3 engine/tools/gateway-domain.py list
python3 engine/tools/gateway-domain.py add myapp # this box, port 443
python3 engine/tools/gateway-domain.py add myapp 100.64.1.5 # another box
python3 engine/tools/gateway-domain.py add myapp 100.64.1.5:8443 # backend not on 443
python3 engine/tools/gateway-domain.py remove myapp
```
A bare label is expanded to `<label>.gtest.commoninternet.net`. Anything containing a dot is
used verbatim, so you can map a domain you control elsewhere — but then **you** must point its
DNS at the gateway (`49.13.156.72`); only `*.gtest.commoninternet.net` is pre-pointed.
## The credential
The admin password is in the orchestrator secret store, **not** in any repo:
```bash
python3 engine/secrets.py get gateway.admin_password
```
| | |
|---|---|
| store | `/secrets/store.yaml` (sops+age, `0600`, outside every git tree) |
| keys | `gateway.admin_password`, `gateway.fqdn`, `gateway.admin_user` |
| age key | `~/.config/sops/age/keys.txt` |
| add/edit | `sops /secrets/store.yaml` |
`gateway-domain.py` reads it itself — you should never need to handle the value. Do not copy it
into a config file, an env file, or a repo. If you need it in a subprocess, use
`engine/secrets.py exec-env` or `with-file` rather than writing a second copy.
## The one thing that surprises people: your box serves the certificate
The gateway does **not** terminate TLS. It reads the SNI name from the TLS handshake and proxies
the still-encrypted bytes onward:
```
browser --TLS--> gateway :443 --reads SNI, proxies encrypted--> your box (tailnet)
```
So after mapping `myapp.gtest.commoninternet.net`, **your box** must serve a certificate valid
for that exact name, on the backend port (443 unless you set one). The gateway has a cert for
its own name only; it never sees your plaintext.
Getting a cert on your box works: the gateway passes HTTP-01 challenges through on `:80` for
mapped names, so ACME can complete normally. Add the domain here **first**, then request the
cert — issuance needs the mapping to already exist.
If you only need plain HTTP for a quick test, that also passes through on `:80`.
## Traps
**Backends must be a literal IPv4 address** (optionally `IP:port`). The tool refuses hostnames,
and it is protecting you: the gateway's `validate_ip` accepts a hostname, but its
`put_domain`/`remove_domain` only match lines whose backend is numeric. A hostname mapping can
be written once and then **never updated or removed** through the admin UI — it becomes an
orphan that only a hand-edit of `/var/lib/tunnel-gateway/tunnel_map.conf` on the box can clear.
**Only ports 22/80/443 are open at the edge.** Mapping `IP:8443` changes which port on *your
box* the gateway connects to; it does not open 8443 to the internet. Exposing a different
*gateway* port is a config change in `nix/hosts/hetzner-test.nix`
(`services.tunnelGateway.openTCPPorts`) **and** the Hetzner firewall in `nix/terraform/` — not
something this tool can do.
**Names are shared.** Anyone with the password can list, overwrite, or delete any mapping.
Prefix yours with something recognisable and remove it when you are finished.
**This is the test gateway.** Never point this tool at the production one. The box is long-lived
(it is also the e2e and CI target), but its map is not sacred.
## When the password stops working
The e2e suite reseeds `.htpasswd` with a throwaway credential for the duration of a run and
restores the previous file afterwards. If a run was killed part-way, the real one is still on
the box at `/var/lib/tunnel-gateway/.htpasswd.e2e-backup`:
```bash
ssh -i /srv/gateway-coop/.secrets/id_admin root@49.13.156.72 \
'mv -f /var/lib/tunnel-gateway/.htpasswd.e2e-backup /var/lib/tunnel-gateway/.htpasswd'
```
To reseed it from the store instead — piping so the value never lands on disk:
```bash
python3 engine/secrets.py get gateway.admin_password | \
ssh -i /srv/gateway-coop/.secrets/id_admin root@49.13.156.72 \
'htpasswd -ci /var/lib/tunnel-gateway/.htpasswd admin \
&& chgrp nginx /var/lib/tunnel-gateway/.htpasswd \
&& chmod 640 /var/lib/tunnel-gateway/.htpasswd'
```
## If it still does not work
Check in this order — most failures are the last two.
1. `gateway-domain.py list` — is the mapping actually there?
2. `getent hosts myapp.gtest.commoninternet.net` — should be `49.13.156.72`.
3. Is your box reachable from the gateway on the tailnet? The gateway is `gateway-test-1`
(`100.91.44.90`), tagged `tag:testing-gateway`. If tailnet ACLs block it from reaching your
node, the mapping is correct and traffic still will not flow.
4. **Is your service actually serving TLS for that name on the backend port?** A backend that
speaks plain HTTP on 443, or serves a cert for a different name, fails here and nowhere else.
## The gateway itself
`gtest.commoninternet.net` / `49.13.156.72` — a Hetzner `cx23` running NixOS, configured in the
`tunnel-gateway-server` repo (`nix/hosts/hetzner-test.nix`), which `notplants-nix` pins as a
submodule under `external/`. It also accepts reverse-SSH tunnels, for boxes not on the tailnet;
that path is separate from this one and is not covered here.
tools/gateway-domain.py
Executable
+216
View File
@@ -0,0 +1,216 @@
#!/usr/bin/env python3
"""gateway-domain.py — point a public domain at your box, via the testing gateway.
WHAT THIS IS FOR. Your machine is on the tailnet but has no public IP, and you need a real
HTTPS domain for it — an OAuth callback, a webhook receiver, a demo someone else can open.
The testing gateway at gtest.commoninternet.net already holds a wildcard DNS record, so
*.gtest.commoninternet.net resolves to it. Map your name to your tailnet IP and the gateway
streams matching connections to you:
browser --TLS--> gateway :443 --reads SNI--> your box over the tailnet
The gateway never terminates that TLS. It reads the SNI name and proxies the still-encrypted
bytes, so **your box serves the certificate**, not the gateway. See SKILL.md.
USAGE
tools/gateway-domain.py list
tools/gateway-domain.py add myapp # -> myapp.gtest.commoninternet.net -> this box's tailscale IP
tools/gateway-domain.py add myapp 100.64.1.5 # explicit backend
tools/gateway-domain.py add myapp 100.64.1.5:8443 # backend listening somewhere other than 443
tools/gateway-domain.py remove myapp
CREDENTIALS. The admin password is in the orchestrator secret store, never here:
python3 engine/secrets.py get gateway.admin_password
This script reads it from there itself. You should not need to handle the value.
"""
import argparse
import base64
import os
import re
import shutil
import subprocess
import sys
import urllib.error
import urllib.parse
import urllib.request
# engine/secrets.py lives one directory up.
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
import secrets as secret_store # noqa: E402 (engine/secrets.py, not the stdlib module)
TIMEOUT = 30
def _cfg():
"""Gateway coordinates. Only the password is genuinely secret; the rest lives
alongside it so there is one thing to change if the gateway ever moves."""
return (
secret_store.get("gateway.fqdn", "gtest.commoninternet.net"),
secret_store.get("gateway.admin_user", "admin"),
secret_store.get("gateway.admin_password"),
)
def _request(method="GET", form=None):
fqdn, user, password = _cfg()
if not password:
sys.exit(
"no gateway.admin_password in the secret store.\n"
" check: python3 engine/secrets.py list\n"
" see: skills/gateway-domain/SKILL.md"
)
url = f"https://{fqdn}/admin/"
data = urllib.parse.urlencode(form).encode() if form else None
req = urllib.request.Request(url, data=data, method=method)
token = base64.b64encode(f"{user}:{password}".encode()).decode()
req.add_header("Authorization", f"Basic {token}")
try:
with urllib.request.urlopen(req, timeout=TIMEOUT) as r:
return r.status, r.read().decode("utf-8", "replace")
except urllib.error.HTTPError as e:
body = e.read().decode("utf-8", "replace")
if e.code == 401:
sys.exit(
f"401 from {url} — the stored password is not what the gateway expects.\n"
"The e2e suite reseeds .htpasswd while it runs and restores it afterwards;\n"
"if a run was interrupted, /var/lib/tunnel-gateway/.htpasswd.e2e-backup is\n"
"still on the box. See SKILL.md, 'When the password stops working'."
)
sys.exit(f"HTTP {e.code} from {url}\n{body[:600]}")
except urllib.error.URLError as e:
sys.exit(f"cannot reach {url}: {e.reason}")
# The admin app answers 200 and renders a red <p> for a rejected value, rather than
# using a status code. Checking only the status would report success on a no-op.
_ERR = re.compile(r'<p style="color:red">(.*?)</p>', re.S)
# Backends are restricted to a literal IPv4 address, optionally with a port.
#
# The gateway would accept a hostname -- its validate_ip is ^[a-zA-Z0-9_.-]+$ -- but its
# put_domain/remove_domain match existing lines with ^<domain>\s+[\d.:]+; which only ever
# matches a numeric backend. A hostname mapping can therefore be written once and then never
# updated or removed through the admin UI: it becomes an orphan only a hand-edit of
# /var/lib/tunnel-gateway/tunnel_map.conf can clear. Refuse to create one.
_BACKEND = re.compile(r"^(\d{1,3}(?:\.\d{1,3}){3})(?::(\d{1,5}))?$")
def _validate_backend(backend):
m = _BACKEND.match(backend)
if not m:
sys.exit(
f"backend must be an IPv4 address, optionally IP:port -- got {backend!r}.\n"
"Hostnames are rejected on purpose: the gateway can store one but cannot\n"
"remove it again. Use the tailnet IP (tailscale ip -4 on the target box)."
)
ip, port = m.group(1), m.group(2)
if any(int(o) > 255 for o in ip.split(".")):
sys.exit(f"not a valid IPv4 address: {ip}")
if port is not None and not (1 <= int(port) <= 65535):
sys.exit(f"port out of range: {port}")
return backend
def _check(body):
m = _ERR.search(body)
if m:
sys.exit(f"gateway rejected the request: {m.group(1).strip()}")
def _parse_domains(body):
"""Pull the Domain Mappings table out of the admin page."""
section = body.split("Domain Mappings", 1)[-1].split("Port Mappings", 1)[0]
return re.findall(r"<td>\s*(.*?)\s*</td>\s*<td>\s*(.*?)\s*</td>", section, re.S)
def _tailscale_ip():
exe = shutil.which("tailscale") or "/run/current-system/sw/bin/tailscale"
try:
out = subprocess.run([exe, "ip", "-4"], capture_output=True, text=True, timeout=15)
except (OSError, subprocess.SubprocessError) as e:
sys.exit(f"could not run tailscale to detect this box's IP ({e}); pass the backend explicitly")
ip = out.stdout.strip().splitlines()[0].strip() if out.stdout.strip() else ""
if not ip:
sys.exit(
"tailscale reported no IPv4 address — is this box on the tailnet?\n"
f" {exe} status\n"
"Or pass the backend explicitly: gateway-domain.py add <name> <ip>"
)
return ip
def _qualify(name, fqdn):
"""A bare label becomes a subdomain of the gateway; anything with a dot is used as given."""
return name if "." in name else f"{name}.{fqdn}"
def cmd_list(args):
_, body = _request()
rows = _parse_domains(body)
if not rows:
print("no domain mappings")
return
width = max(len(d) for d, _ in rows)
for domain, backend in rows:
print(f" {domain:<{width}} -> {backend}")
def cmd_add(args):
fqdn, _, _ = _cfg()
domain = _qualify(args.name, fqdn)
backend = args.backend or _tailscale_ip()
_validate_backend(backend)
_, body = _request("POST", {"domain": domain, "ip": backend})
_check(body)
# Re-read rather than trusting the POST body: app.py mutates its in-memory dict and
# renders that, so the response shows what it meant to do, not what landed on disk.
_, fresh = _request()
if not any(d == domain for d, _ in _parse_domains(fresh)):
sys.exit(f"POST returned 200 but {domain} is not in the mapping table — check `list`")
print(f" {domain} -> {backend}")
print(f"\nYour box must now serve TLS for {domain} on the backend port (443 unless you")
print("set one). The gateway does not terminate TLS; it proxies the encrypted stream.")
def cmd_remove(args):
fqdn, _, _ = _cfg()
domain = _qualify(args.name, fqdn)
# The admin app treats a backend of exactly "0" as delete.
_, body = _request("POST", {"domain": domain, "ip": "0"})
_check(body)
_, fresh = _request()
if any(d == domain for d, _ in _parse_domains(fresh)):
sys.exit(
f"{domain} is still in the mapping table after the delete.\n"
"If its backend is a hostname rather than an IP, the gateway cannot remove it:\n"
"edit /var/lib/tunnel-gateway/tunnel_map.conf on the box and reload nginx."
)
print(f" removed {domain}")
def main():
ap = argparse.ArgumentParser(
description="Map a public domain to your tailnet box via the testing gateway.",
epilog="Password comes from the secret store (gateway.admin_password); see skills/gateway-domain/SKILL.md.",
)
sub = ap.add_subparsers(dest="cmd", required=True)
sub.add_parser("list", help="show current domain mappings").set_defaults(fn=cmd_list)
a = sub.add_parser("add", help="point a domain at a backend")
a.add_argument("name", help="bare label (myapp) or a full domain")
a.add_argument("backend", nargs="?", help="IP or IP:port (default: this box's tailscale IP, port 443)")
a.set_defaults(fn=cmd_add)
r = sub.add_parser("remove", help="delete a domain mapping")
r.add_argument("name", help="bare label (myapp) or a full domain")
r.set_defaults(fn=cmd_remove)
args = ap.parse_args()
args.fn(args)
if __name__ == "__main__":
main()