# Phase `pxgate` — break the deploy-proxy ↔ dashboard health-gate circular dependency (D8 fix)

**Mission:** fix the boot-ordering deadlock the Adversary filed as A1 (BACKLOG-pvfix /
DEFERRED 2026-06-13) so a **from-scratch (D8) boot brings traefik up cleanly** instead of
deploy-proxy hanging 15 min and failing. The traefik proxy must still be health-gated with
rollback — just gated on a signal that does NOT depend on a service ordered after it.

State files under `machine-docs/`: `STATUS-pxgate.md`, `BACKLOG-pxgate.md`,
`REVIEW-pxgate.md`, `JOURNAL-pxgate.md`. DECISIONS.md shared.

## Root cause (verified)
1. `nix/modules/proxy.nix` — `deploy-proxy.service` runs `warm_reconcile.py traefik`; its
   `wait_healthy` polls **`https://ci.commoninternet.net` → 200**. `TimeoutStartSec = 900`.
2. `https://ci.commoninternet.net` is served by the **dashboard**.
3. `nix/modules/dashboard.nix` — `deploy-dashboard.service` has **`After=deploy-proxy.service`**,
   so systemd won't start the dashboard until deploy-proxy exits.
→ On a fresh boot: proxy waits for the dashboard, dashboard waits for proxy → proxy times out
   at 900s and **fails**, then the dashboard starts onto a failed proxy. (On the *running*
   server it's invisible — the dashboard is already up — which is why it's a latent D8 risk.)

## Required fix
Break the cycle while keeping a MEANINGFUL traefik health signal + the rollback semantics
(deploy latest → health-gate → commit last-good if healthy / roll back to last-good if not).

**Recommended approach (Builder confirms in DECISIONS.md):** change the deploy-proxy health
gate to probe a **traefik-self-served endpoint that is up the moment traefik is, with no
backend/dashboard dependency** — e.g. traefik's own `ping` entrypoint (`/ping` → 200) or
`https://traefik.ci.commoninternet.net/api/version`. That confirms "traefik (re)deployed and
is serving" — enough to drive the rollback decision — without waiting on `ci.commoninternet.net`.
- If you still want an end-to-end "traefik routes to a real backend" assertion, move THAT to a
  **separate converge/health step ordered AFTER the dashboard** (or rely on the dashboard's own
  health gate); it must NOT be inside the deploy-proxy oneshot.
- Do NOT simply delete the health gate (loses the rollback safety net). Do NOT just bump the
  timeout (the deadlock still fails on a cold boot, just slower). Do NOT reintroduce any
  dependency on a service that is `After=deploy-proxy`.
- Make sure traefik actually EXPOSES whatever endpoint you choose (the `ping` entrypoint /
  api may need enabling in the traefik recipe/compose) — verify it returns 200 with only
  traefik up.

Touch points: `nix/modules/proxy.nix`, the traefik health check in `runner/warm_reconcile.py`
(and/or `runner/harness/*` health helpers), possibly the traefik recipe overlay to expose the
health endpoint. Check the other `After=deploy-proxy` consumers (drone, warm-keycloak, reports,
bridge, backupbot, nightly-sweep) still order correctly after the change.

## Gates

**M1 — Fix + controlled reproduction (Builder/Adversary, on a test box / the cc-ci host
without a real wipe).** Implement the fix. Reproduce the cycle in a CONTROLLED way (per the
Adversary's A1 repro): with the dashboard held back / absent, show the OLD gate hangs→fails and
the NEW gate goes healthy on traefik alone. Unit/integration-test the new health check.
Adversary cold-verifies: cycle is genuinely broken, health signal is still meaningful (a
broken traefik must still fail the gate and roll back), no `After=deploy-proxy` consumer
regressed. PR to cc-ci repo; do NOT merge to the live host yet.

**M2 — Proven on a real from-scratch boot.** The durable proof is a **from-scratch / cold
boot** where `deploy-proxy` reaches `active` without the dashboard and the whole control plane
converges. Coordinate with the orchestrator: the orchestrator owns the live `nixos-rebuild` and
any throwaway-VM / from-scratch rebuild (host infra is orchestrator/operator territory, not the
loops). Acceptance: on a clean boot deploy-proxy is `active` (not failed), the rollback path
still works on a deliberately-broken traefik, and the running server is unaffected. Fresh
Adversary PASS → `## DONE`.

## Guardrails
- **Health verification must stay honest** — a traefik that isn't actually serving must still
  fail the gate and roll back to last-good. Breaking the cycle must not become "always pass".
- Live host changes (`nixos-rebuild`, from-scratch boot) are **orchestrator/operator-owned**;
  the loops produce the code + controlled proof and a PR, the orchestrator deploys + runs the
  cold-boot test. The loops must NOT nixos-rebuild the live host themselves.
- No gate weakening elsewhere; minimal change; no secrets in commits. Commit author
  `autonomic-bot <autonomic-bot@noreply.git.autonomic.zone>`; push every commit. CI host has no
  python3 on default PATH — use the harness venv.

## Definition of Done
The deploy-proxy↔dashboard cycle is broken: a from-scratch boot brings traefik up health-gated
(on a dashboard-independent signal) with rollback intact, no consumer mis-ordered, the running
server unaffected, the A1 / DEFERRED entry closed with pointers, and M1+M2 fresh Adversary PASSes.