The CI server filled up and every !testme from build 1236 to 1242 died at harness startup with ENOSPC on /var/lib/cc-ci-runs/<build>. Because the harness never got far enough to write results.json, the PR badges just said 'failure' — so it read as recipe regressions, and plausible's genuinely-fixed suite looked still-broken. Cause: every run pulls each recipe's images and nothing ever removed the old ones. 72GB of images, 63GB of it unused. Reclaimed 69.8GB; the host went 73% -> 22%. Two changes so it does not recur: - sweep-orphans.sh (runs at the start AND end of every /upgrade-all) now prunes unused images when the disk is >=60% (DISK_PRUNE_PCT). Below that it keeps the layer cache so runs stay fast. 'docker image prune -a' spares anything a container references, so infra and warm-* canonicals are safe. Volumes are still NOT pruned — warm-* canonical volumes are data-warm and legitimately dangling. - /cc-ci-status flags server disk at >65% rather than >80%, because this is not a steady-state measure: the host was at 73% when runs started failing. It also now checks that recent builds actually produced results.json — an empty run dir is the fingerprint of a host problem masquerading as a recipe failure — and records how to read a drone step log out of its sqlite when the API token is unreachable.
cc-ci-orchestrator
Orchestrator workspace for building the cc-ci Co-op Cloud recipe CI server. The plan, launch
tooling, and loop prompts live in cc-ci-plan/; see AGENTS.md for the
roles and operating model. Secrets (.testenv) are gitignored — never commit them.
Run the orchestrator in tmux (survives disconnects + closing your laptop)
Keep this supervising session alive on the host with tmux, and use --remote-control so you can
watch/steer it from claude.ai/code (or the mobile app).
# 0. Exit any running orchestrator session first — a conversation can't be resumed while it's live:
# /exit (inside Claude) or Ctrl-D
# 1. Start a detachable tmux session on this host
tmux new -s orchestrator
# 2. Inside tmux, resume the orchestrator conversation WITH remote control:
claude --resume autonomous-orchestrator \
--remote-control "autonomous-orchestrator" \
--dangerously-skip-permissions
# - If name-resume opens a picker instead of resuming directly, choose "autonomous-orchestrator".
# - Or resume by the stable session id (more deterministic in a fresh pane):
# claude --resume 34a80a99-b37e-4809-b8da-ccc9fafe785e \
# --remote-control "autonomous-orchestrator" --dangerously-skip-permissions
# 3. Detach — the process keeps running: press Ctrl-b, then d
Reconnect later
- On this host:
tmux attach -t orchestrator - From anywhere: claude.ai/code → the
autonomous-orchestratorsession
Why it survives: tmux keeps the claude process alive across SSH disconnects and your laptop
closing; remote-control runs outbound from this host to Anthropic, so it stays connected
regardless of the viewer. After a host reboot, re-run steps 1–2.
Two different "names":
--resume <name|id>selects the conversation to restore (shown in the/resumepicker); the--remote-control "<name>"value is only the web display label and resumes nothing. Resuming reuses the same session id each time (stays34a8…) — don't pass--fork-sessionunless you intend to branch a new conversation.Already inside a live session and just want the web surface? Run
/remote-control— no exit/resume.
Kick off / supervise the loops
cd /srv/cc-ci/cc-ci-plan
./launch.sh start # Builder + Adversary loops (interactive --remote-control in tmux) + watchdog
./launch.sh status # session + DONE state
./launch.sh logs builder|adversary|watchdog
./launch.sh stop
Full supervision guide, credential map, and the Incus VM fallback are in
cc-ci-plan/kickoff.md and cc-ci-plan/plan.md §1.5.