399e999978
Live-testing the resume path surfaced two gaps: (1) an `opencode run` proc
EXITS when the model ends its turn, so a long /upgrade-all run's process dies
repeatedly before the whole run completes — and the log mtime freezes on death,
so the watchdog's log-idle>15min signal is both too slow and unreliable. (2) A
resumed run had no watchdog, so nothing re-continued it.
- watchdog(): detect PROC-DEATH (no live `opencode run` proc for the session +
not completed) and resume promptly, in addition to log-idle. Guarded by
MAX_RESUMES (default 20) so a no-progress loop (e.g. disk-full) eventually hands
off to the supervisor/operator instead of spinning forever.
- resume(): auto-spawn a watchdog if none is alive (skips when the watchdog itself
called resume — it lives in {SESSION}-watchdog — so no duplicate).
- launch-supervisor.py gate: defer while the per-run watchdog is alive (it is the
single writer for prompt-recovery). The supervisor only takes over once the
watchdog gives up (MAX_RESUMES) — i.e. a wedge a bare resume can't fix. Removes
the supervisor/watchdog double-resume race.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WxbpH3DquKzoSTSwGvGuET
cc-ci-plan
Self-contained handoff package for building the cc-ci Co-op Cloud recipe CI server with two autonomous Claude loops (a Builder and an adversarial Reviewer) running over days.
Start here
- Read
plan.md— the full plan and single source of truth (mission, Definition of Done, architecture, milestones, the two-agent coordination protocol, loop discipline). - Read
kickoff.md— how to launch and supervise the loops. - Run
./launch.sh startto bring up both loops + the watchdog.
Files
| File | Purpose |
|---|---|
plan.md |
The Phase-1 plan (build the CI server). Agents treat it as their single source of truth. |
plan-phase1c-full-reproducibility.md |
Phase 1c (runs first): make the VM fully reproducible from git (all secrets incl. the wildcard cert in sops, in a separate private cc-ci-secrets repo as a flake input; base stays well-parameterized) and do the genuine throwaway-VM live rebuild to close D8 honestly (the "infeasible by design" was overstated). |
plan-phase1b-review-lint.md |
Phase 1b (after 1c): deterministic linting/formatting in CI + a white-box review checklist (real tests, DRY harness, idempotent Nix, no footguns/secrets), ending in a full cold re-verification of all D1–D10 — now covering 1c's refactor. |
plan-phase1d-generic-test-suite.md |
Phase 1d (after 1b, before 2): a generic install/upgrade/backup/restore suite that runs on any recipe with zero config, with a recipe's own test_<op>.py overriding or extending the generic (Builder's call) and reusing the generic's deployment — no redeploy, plus optional custom install-steps; recipes needing special setup fail the generic form gracefully. The test-architecture foundation Phase 2 builds on. |
plan-phase1e-harness-corrections.md |
Phase 1e (after 1d, before 2): three operator-review corrections to the shared generic harness — (HC1) upgrade goes previous-release → PR head via deploy --chaos; (HC2) repo-local PR code runs only for approved recipes (default = cc-ci overlays + generic only); (HC3) the generic runs by default alongside an overlay, skipped only via explicit opt-out. |
plan-phase2-recipe-tests.md |
Phase 2 (after Phase 1e): build on the corrected generic suite — author the recipe overlays (port recipe-maintainer tests as test_*.py) + define custom install steps where a recipe fails generically. |
plan-phase2b-test-performance.md |
Phase 2b (after Phase 2, before Phase 3): empirically measure where test time goes and reduce it (image cache, readiness tuning, dedup deploys, warm infra, concurrency) — no weakened tests. |
plan-phase3-results-ux.md |
Phase 3 (after Phase 2b): beautiful YunoHost-style results — per-run level, image-forward PR comment (badge + summary card + app screenshot), polished dashboard. |
IDEAS.md |
Deferred/future ideas, parked out of current scope. |
brief.md |
The original one-page brief (context only; plan.md supersedes it). |
kickoff.md |
Launch & supervision guide. |
launch.sh |
Starts both loops + a watchdog; restarts dead loops; stops on ## DONE. |
prompts/builder.md |
Builder loop prompt (fed to claude by the script). |
prompts/adversary.md |
Adversary loop prompt. |
Before launching
- Set the org in
plan.md(git.autonomic.zone/recipe-maintainers/cc-ci) and lock the six proof recipes (§8). - Ensure the launching shell has: SSH+sudo to
cc-ci, the Gitea token,git.autonomic.zoneaccess. - Preconfigure test-app DNS + TLS (plan §4.0): point a wildcard
*.ci.commoninternet.netrecord at a gateway that TLS-passthroughs to cc-ci, and pre-issue the wildcard cert (*.ci.commoninternet.net+ci.commoninternet.net, via Gandi DNS-01) into/var/lib/ci-certs/live/on cc-ci. The agent handles everything else on cc-ci (Traefik file provider → that cert, swarm, routing) and does no ACME; renewal (~90 days) is an out-of-band operator task, so the DNS token never goes to the agent. export CC_CI_REPO=https://git.autonomic.zone/recipe-maintainers/cc-ci.gitso the watchdog can detect## DONE.
What "done" means
The loops stop only when all of plan.md §2 (D1–D10) hold and the Adversary has independently
re-verified each within 24h. The watchdog then tears the loops down automatically.