cc-ci-orchestrator/.claude/skills/upgrade-all/SKILL.md

name, description

name

description

upgrade-all

Weekly autonomous upgrade run for the cc-ci CI server. Surveys every enrolled recipe for available upstream upgrades, then runs /recipe-upgrade on each upgradeable one via a subagent — plan, implement, verify green on cc-ci, open a recipe PR (and, only if a cc-ci test went stale, a verified cc-ci test PR). Collects results into one summary listing every PR to review. Sequential by default (shared Swarm); --parallel to fan out; --dry-run to preview. NEVER merges. Built to run once weekly on a cron. Invoke as /upgrade-all.

upgrade-all

The cc-ci analogue of recipe-maintainer's /recipe-upgrade-cron-all: an unattended weekly pass that keeps every enrolled recipe current, with cc-ci as the test gate and a human in the loop only where it matters — PR review. It surveys upgrades, runs /recipe-upgrade <recipe> per upgradeable recipe, and writes one summary of every PR to review. It never pushes upstream and never merges.

Drives cc-ci over ssh cc-ci. Logs/summary go to /srv/cc-ci/.cc-ci-logs/upgrades/.

Runs as the cc-ci-upgrader agent. This skill is normally executed by a dedicated, observable one-shot job agent — cc-ci-upgrader — spun up under remote-control (viewable/steerable at claude.ai/code, like the Builder) by cc-ci-plan/launch-upgrader.sh. That agent runs this skill to completion, then stops and stays idle so the run + summary remain reviewable in the web UI (it does NOT self-terminate). The weekly cron just invokes launch-upgrader.sh start; the next week's run clears the idle session and starts fresh. You can also run /upgrade-all inline in any /srv/cc-ci session, but the agent is the intended path so the weekly run isn't buried in headless output.

Arguments (optional $ARGUMENTS)

• A space-separated list of recipe names → only those (else all enrolled recipes).
• --dry-run → survey + print what WOULD upgrade; spawn nothing.
• --parallel → fan out all per-recipe subagents at once (faster, more host load — see safety below).

1. Build the candidate list

Enrolled recipes = the cc-ci tests/<recipe>/ dirs (same set ci-test-review sweeps):

ssh cc-ci 'cd /root/cc-ci/tests && ls -d */' | sed 's#/##' | grep -vE '^(_generic|unit|__pycache__)$'

(or the names passed in $ARGUMENTS). For each, on cc-ci, check availability — skip dirty/up-to-date.

⚠️ Three things that silently skip recipes — do ALL THREE per recipe before the version check:

1. pseudo-TTY: abra FATAs inappropriate ioctl for device under plain ssh — wrap every abra call in script: ssh cc-ci 'script -qec "abra <args> -n" /dev/null' (git/other commands need none).
2. go-git auth to git.autonomic.zone: recipes on the private mirror FATA unable to fetch tags … authentication required: Unauthorized — abra/go-git reads remote.origin.url literally and ignores git url.insteadOf. Bake creds into origin first (idempotent, only if origin is git.autonomic.zone): git -C ~/.abra/recipes/<r> remote set-url origin "https://$GITEA_USERNAME:$GITEA_PASSWORD@git.autonomic.zone/recipe-maintainers/<r>.git".
3. dirty worktree is usually just the untracked cc-ci overlay (compose.ccci.yml) — git stash -u before / stash pop after; only skip dirty-worktree if TRACKED changes remain.

ssh cc-ci 'export PATH=/run/current-system/sw/bin:$PATH; R=<r>; \
  url=$(git -C ~/.abra/recipes/$R config --get remote.origin.url); \
  case "$url" in https://git.autonomic.zone/*) \
    git -C ~/.abra/recipes/$R remote set-url origin "https://'"$GITEA_USERNAME"':'"$GITEA_PASSWORD"'@${url#https://}";; esac; \
  git -C ~/.abra/recipes/$R stash -u >/dev/null 2>&1 || true; \
  script -qec "abra recipe fetch $R --force -n" /dev/null; \
  script -qec "abra recipe upgrade $R -m -n" /dev/null; \
  git -C ~/.abra/recipes/$R stash pop >/dev/null 2>&1 || true'

Build RECIPES_TO_UPGRADE = recipes with ≥1 available upgrade (after the stash, so the untracked overlay doesn't count as dirty). Others go to SKIPPED_UPFRONT with a reason (dirty-worktree only for real TRACKED edits, up-to-date, not-fetchable).

Reconcile every candidate's mirror during the survey (even the up-to-date ones), so merged-upstream PRs are closed and every mirror main tracks true upstream — fleet-wide, not just where there's a new upgrade:

set -a; . /srv/cc-ci/.testenv; set +a
ssh cc-ci "GITEA_USERNAME='$GITEA_USERNAME' GITEA_PASSWORD='$GITEA_PASSWORD' GITEA_URL='$GITEA_URL' bash -s <recipe> --reconcile-only" \
   < /srv/cc-ci/.claude/skills/recipe-upgrade/open-recipe-pr.sh

(The per-recipe /recipe-upgrade also reconciles, so this is belt-and-suspenders for skipped recipes — count closed-merged PRs in the summary.)

2. Print the plan

A table — Recipe | Status (will upgrade / skipped:reason) | Available upgrade(s) — plus the mode (Sequential default / Parallel). If --dry-run, stop here.

3. Upgrade each recipe via a subagent

For each recipe in RECIPES_TO_UPGRADE, spawn an Agent (subagent_type: "general-purpose", description "Upgrade <recipe> on cc-ci") with a prompt like:

Run the /recipe-upgrade <recipe> skill end-to-end — DEFAULT mode, do NOT pass --with-tests (/srv/cc-ci/.claude/skills/recipe-upgrade/SKILL.md): plan, implement, open a recipe PR, and verify it by posting !testme on the PR (results visible in the PR; iterate ≤3×). Use the existing tests — if a test fails because it is genuinely stale, leave an explanatory comment on the PR for the operator and do NOT modify any test. Drive cc-ci over ssh cc-ci. Do NOT prompt. Do NOT push upstream. Do NOT merge. Print exactly one RESULT: line as your final line (SUCCESS / SUCCESS-PENDING-TESTS / FAILED / SKIPPED — SUCCESS+TESTPR will not occur in default mode).

Why default (no --with-tests): the weekly cron must never auto-edit cc-ci tests unattended — a test change deserves a human decision. So the cron opens recipe PRs only; where a recipe's existing test looks stale, the operator sees the explanation in the PR comment and can re-run that one recipe with /recipe-upgrade <recipe> --with-tests to also get a verified test-update PR.

• Sequential (default): one Agent at a time; wait, collect its RESULT:, then the next. If the Agent tool call itself errors, record FAILED — agent tool error and continue. One failure never aborts the run.
• Parallel (--parallel): emit all Agent calls in one message (no run_in_background — you need the results). Failures are isolated per recipe.

4. Collect results

Parse each final RESULT: line into SUCCESS / SUCCESS-PENDING-TESTS / FAILED / SKIPPED (default mode won't emit SUCCESS+TESTPR). A subagent that emitted no RESULT: line → FAILED — no result emitted.

5. Write + print the summary

Write /srv/cc-ci/.cc-ci-logs/upgrades/upgrade-all-<YYYY-MM-DD>.md and print it, leading with the PR list (the actionable output):

# cc-ci Weekly Upgrade Run — <YYYY-MM-DD>
## Summary
- Considered: N · PR green (!testme): N · PR open but tests look stale (commented): N · Failed: N · Skipped: N
## PRs to review (NOT merged)
- <recipe> <old> → <new> — recipe PR: <url> — !testme GREEN
## PRs where a test looks stale (operator: re-run `--with-tests` to update tests)
- <recipe> <old> → <new> — recipe PR: <url> — !testme RED on <test>; see PR comment
## Failed (investigate)
- <recipe> — at <step>: <reason>   (log: .cc-ci-logs/upgrades/<recipe>-upgrade-<date>.md)
## Skipped
| recipe | reason |

End with the report path and a reminder that nothing was merged.

6. Launch the public Recipe Report

Once the summary is written, kick off the weekly public report (its own agent, separate model — default opus): python3 /srv/cc-ci/cc-ci-plan/launch-report.py start. It runs /recipe-report, reviews this run + the live recipe/PR state, and publishes to https://report.ci.commoninternet.net. Fire-and-forget — it runs independently; you can then go idle.

Safety / coordination (this matters — shared host with the build loops)

• Sequential is the default for a reason. Recipe deploys are stateful on the shared Swarm and parallel deploys can OOM/collide. Between sequential recipes, the per-recipe recipe-upgrade tears down what it deployed; verify a recipe is undeployed before the next starts (ssh cc-ci 'script -qec "abra app ls -n" /dev/null' — pseudo-TTY wrapped, per the box above).
• Single-writer: every PR (recipe or cc-ci test) is on a dedicated branch; never push main, never touch the build loops' /cc-ci /cc-ci-adv working clones or their in-flight state.
• Contention with active loop development: while the loops are still building cc-ci, this run competes for the host. Prefer a quiescent window; if a recipe fails due to contention it's simply retried next week. (Once cc-ci is built and the loops are idle, this is the steady-state weekly job.)
• Never merges; failures/ skips are surfaced and retried next week — safe to re-run anytime.

Cron

Designed for a weekly Claude Code scheduled task that runs cc-ci-plan/launch-upgrader.sh start — which spins up the cc-ci-upgrader remote-control agent to run this skill to completion (the agent then stays idle/viewable). The cron does NOT invoke /upgrade-all inline.

Schedule (operator, 2026-05-29): the weekly cron is installed as the closing action of the final build phase (Phase 5), and its first run is ~1 hour after the build completes, weekly from then on — i.e. anchored to actual completion, not a fixed clock slot. Compute T0 = completion + 1h and install a weekly job at T0's day-of-week + HH:MM (MM HH * * DOW) running launch-upgrader.sh start. Then verify the first kickoff actually launches the cc-ci-upgrader agent. Do NOT install it earlier — while the loops are still building cc-ci it would contend for the shared host; until then run it manually via launch-upgrader.sh start/fresh. Full procedure: Phase 5 plan §4 (cc-ci-plan/plan-phase5-verify-upgrade-flow.md).

Re-running is idempotent: already-current recipes report SKIPPED — up-to-date; recipes with an open PR for the same branch report the existing PR rather than duplicating it.