weekly-run: pre-reclaim stale cc-ci images + hourly glm-5.2 supervisor

Root-cause fix for the 2026-07-03 run stalling: the cc-ci host disk filled to
100% (ENOSPC) mid-run (Wave 6, lasuite-drive), the agent stopped to reclaim
space, and nothing resumed it — the log-idle/429 watchdog only covers opencode-go
usage-limit stalls, not an environmental wedge.

- launch-upgrader.py: step-0 prereclaim_cc_ci() prunes STALE cc-ci docker images
  (unused AND older than a week, so this week's likely-reused images stay) before
  each weekly run. Best-effort; env-tunable (UPGRADER_PRERECLAIM*).
- launch-supervisor.py (new): hourly glm-5.2 orchestrator wake-up. Cheap
  deterministic gate — no-ops (zero tokens) when the run is complete or
  progressing; only when a run stalled/died before completing does it launch a
  short-lived glm-5.2 agent to diagnose + drive it to a clean DONE. Progress is
  judged by live run-proc + log mtime (session_busy() is claude-tuned and misreads
  a headless opencode run as idle).
- configuration.nix: cc-ci-upgrade-supervisor service + hourly timer (:07).
- upgrade-all SKILL §0: note the stale-image reclaim for manual runs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WxbpH3DquKzoSTSwGvGuET

cc-ci-plan/launch-upgrader.py

@@ -58,6 +58,15 @@ OPENCODE_SHARE = os.environ.get("OPENCODE_SHARE", "1") == "1"
 
 UPGRADER_ARGS = os.environ.get("UPGRADER_ARGS", "")
 
+# First step of the weekly run: reclaim STALE docker images on the cc-ci server BEFORE the run so a
+# heavy run can't fill the disk mid-flight (root cause of the 2026-07-03 stall — 100% ENOSPC killed
+# lasuite-drive + wedged the run). "Stale" = unused by any container AND older than PRERECLAIM_UNTIL,
+# so recently-built/pulled images (the ones this week's tests will reuse) are KEPT — we only evict
+# leftovers from prior weeks. Best-effort; never fails the run.
+PRERECLAIM       = os.environ.get("UPGRADER_PRERECLAIM", "1") == "1"
+PRERECLAIM_UNTIL = os.environ.get("UPGRADER_PRERECLAIM_UNTIL", "168h")  # 7d: older than one run ago
+PRERECLAIM_HOST  = os.environ.get("UPGRADER_PRERECLAIM_HOST", "cc-ci")
+
 # ── helpers ───────────────────────────────────────────────────────────────────
 
 def log(msg):
@@ -83,6 +92,27 @@ def session_busy():
 def kill_session():
     subprocess.run(["tmux", "kill-session", "-t", SESSION], capture_output=True)
 
+def prereclaim_cc_ci():
+    """Weekly-run step 0: prune STALE (unused AND older than PRERECLAIM_UNTIL) docker images on the
+    cc-ci server so the run has disk headroom. Keeps recent images (reused this week); only clears
+    prior-weeks' leftovers. Best-effort — a reclaim failure must never abort the run."""
+    if not PRERECLAIM:
+        return
+    filt = f"--filter until={PRERECLAIM_UNTIL}"
+    remote = (f"docker image prune -af {filt} 2>&1 | tail -1; "
+              f"docker builder prune -af {filt} >/dev/null 2>&1 || true; "
+              f"df -h / | tail -1")
+    log(f"  step 0: pre-reclaim stale docker images on {PRERECLAIM_HOST} (unused & >{PRERECLAIM_UNTIL})")
+    try:
+        r = subprocess.run(["ssh", "-o", "ConnectTimeout=15", PRERECLAIM_HOST, remote],
+                           capture_output=True, text=True, timeout=900)
+        out = (r.stdout or r.stderr or "").strip()
+        for ln in out.splitlines():
+            if ln.strip():
+                log(f"    {ln.strip()}")
+    except Exception as e:
+        log(f"    pre-reclaim skipped (non-fatal): {e}")
+
 # ── kickoff prompt ────────────────────────────────────────────────────────────
 
 def build_kickoff():
@@ -130,6 +160,11 @@ def start(mode="use-or-create"):
         kill_session()
         import time; time.sleep(1)
 
+    # Step 0 of the weekly run: clear STALE cc-ci docker images so a heavy run can't run the disk
+    # out mid-flight (root cause of the 2026-07-03 stall). Only for the actual upgrade run.
+    if SESSION == "cc-ci-upgrader":
+        prereclaim_cc_ci()
+
     kf = Path(LOG_DIR) / f".kickoff-{SESSION}.txt"
     kf.write_text(build_kickoff())