cc-ci-orchestrator/cc-ci-plan/orchestration.md

orchestration.md — cc-ci agent map (root)

The root structure of the cc-ci build system: who the agents are, where each one's prompt and plan live, and how they're kept on track. Start here. The watchdog and the coordination protocol are the subtlety beneath this map (see the last two sections).

Agents

Agent	Role (one line)	Prompt	Plan / SSOT	Session · workdir · launcher
Builder	Builds the CI server; one of two independent loops	cc-ci-plan/prompts/builder.md	the current phase plan (launch.py status names it) + master cc-ci-plan/plan.md	cc-ci-builder · /srv/cc-ci/cc-ci · launch.py
Adversary	Independently disbelieves & verifies the Builder; owns REVIEW + veto	cc-ci-plan/prompts/adversary.md	same current phase plan (verifies against it) + plan.md	cc-ci-adv · /srv/cc-ci/cc-ci-adv · launch.py
Orchestrator	Keeps everyone on track — supervises, nudges, fixes plans/prompts, owns the host-level fallback	wake: cc-ci-plan/ai-progress-monitor-prompt.txt → this doc (§The orchestrator's job)	this doc + cc-ci-plan/JOURNAL.md (handoff record)	cc-ci-orchestrator-vm · /srv/cc-ci-orch · launch-orchestrator.py
Assistant	One-shot agent dispatched for cross-cutting passes (e.g. mirror reconcile); idle unless dispatched	assignment set at launch (launch-assistant.py)	the task it's dispatched with	cc-ci-assistant · /srv/cc-ci-orch · launch-assistant.py
Upgrader	Weekly one-shot: runs /upgrade-all (recipe-upgrade survey + PRs, never merges)	the /upgrade-all skill	triggered by the cc-ci-upgrade-all systemd timer (Sun 02:00 UTC)	cc-ci-upgrader · /srv/cc-ci · launch-upgrader.py

Phases are defined in .cc-ci-logs/.phases-spec (id|planfile|statusfile, persisted by launch.py start); launch.py status shows the current one. Backend is claude/sonnet (.loop-backend/ .loop-model).

The orchestrator's job — keep everyone on track

On each scheduled wake (and on startup — see AGENTS.md):

1. Current phase, read live: python3 cc-ci-plan/launch.py status → current phase id, its plan file, its STATUS-<id>.md. Never assume a phase; whatever it reports IS the phase.
2. Live-state checks: builder/adv/watchdog panes (tmux capture-pane -pt …); .loop-backend=claude & .loop-model=sonnet; ssh cc-ci hostname reachable.
3. Keep them moving — only intervene where the watchdog can't:
   • Builder stalled / idle past its WAITING-UNTIL with no work → nudge to continue the current phase.
   • Adversary stale / on old evidence → nudge to re-orient + verify outstanding claims.
   • Loop at high context (≳85%) → nudge it to /compact (lossless; state is in git + STATUS/REVIEW).
   • Loop session missing → RESUME_PHASE=1 LOOP_BACKEND=claude LOOP_MODEL=sonnet python3 cc-ci-plan/launch.py start.
   • Revised a plan a loop is already working in? Ping the session to re-read it — loops read the plan at kickoff and won't see later edits unless told.
4. Completion: a phase is done when its STATUS-<id>.md has a line starting with ## DONE; the watchdog auto-advances. When the LAST phase finishes the watchdog writes SEQUENCE-COMPLETE, stops the loops, and exits (so the hourly wake stops too) → append to JOURNAL.md + proactive PushNotification.
5. Be decisive but minimal. Healthy + active → just note the state. Don't make unrelated changes.

Subtlety: the watchdog (launch.py watchdog)

A non-agent supervisor loop (cc-ci-watchdog tmux session, started by launch.py start / the cc-ci-loops.service boot unit). It: heals dead/wedged loop sessions, pings the other loop on every claim(...)/review(...) commit (the handoff signal), enforces liveness (kills+reboots a loop idle past its WAITING-UNTIL), auto-advances phases when a STATUS-<id>.md hits ## DONE, and writes SEQUENCE-COMPLETE at the end. It also fires the hourly orchestrator wake (this doc, via the wake prompt). It does NOT compact loops or make decisions — that's the orchestrator.

Subtlety: coordination protocol (plan.md §6.1)

The two loops coordinate only through the cc-ci git repo — never directly:

• git pull --rebase before every edit; smallest change; commit; push (never --force).
• Commit-prefix convention (the watchdog depends on it): claim(...) = Builder claims a gate; review(...) = Adversary verdict. Those prefixes ARE the handoff signal.
• Phase-namespaced state files in the repo: STATUS-<id>.md, BACKLOG-<id>.md, REVIEW-<id>.md, JOURNAL-<id>.md; DECISIONS.md is shared.
• Inbox side-channels for non-gate messages: BUILDER-INBOX.md / ADVERSARY-INBOX.md (watchdog edge-pings on appearance; consumer deletes on read).
• Full rules: plan.md §6.1 (coordination), §7 (pacing/liveness), §9 (guardrails).

See also

• AGENTS.md — orchestrator on-startup routine + host/reboot facts (Hetzner cpx22).
• plan.md — master build SSOT.