autonomic-botandClaude Opus 4.8
c7301a9e39
- recipe-report.py: survey (run + per-recipe PRs + CI verdicts) / render (spec->HTML) / publish (copy to cc-ci:/var/lib/cc-ci-reports + regen index). - skill .claude/skills/recipe-report: review the weekly run, classify needs-attention vs routine, publish one public HTML page per week + index at report.ci.commoninternet.net. Read-only. - launch-report.py: one-shot cc-ci-report agent, REPORT_MODEL default opus (separate from the sonnet upgrader), REPORT_BACKEND default claude. - upgrade-all SKILL: closing step launches the report agent. Serving (nix/modules/reports.nix) already deployed + live. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
152 lines
10 KiB
Markdown
152 lines
10 KiB
Markdown
---
|
||
name: upgrade-all
|
||
description: Weekly autonomous upgrade run for the cc-ci CI server. Surveys every enrolled recipe for available upstream upgrades, then runs /recipe-upgrade on each upgradeable one via a subagent — plan, implement, verify green on cc-ci, open a recipe PR (and, only if a cc-ci test went stale, a verified cc-ci test PR). Collects results into one summary listing every PR to review. Sequential by default (shared Swarm); --parallel to fan out; --dry-run to preview. NEVER merges. Built to run once weekly on a cron. Invoke as /upgrade-all.
|
||
---
|
||
|
||
# upgrade-all
|
||
|
||
The cc-ci analogue of recipe-maintainer's `/recipe-upgrade-cron-all`: an unattended **weekly** pass
|
||
that keeps every enrolled recipe current, with **cc-ci as the test gate** and a human in the loop only
|
||
where it matters — **PR review**. It surveys upgrades, runs `/recipe-upgrade <recipe>` per upgradeable
|
||
recipe, and writes one summary of every PR to review. It **never pushes upstream and never merges**.
|
||
|
||
Drives cc-ci over `ssh cc-ci`. Logs/summary go to `/srv/cc-ci/.cc-ci-logs/upgrades/`.
|
||
|
||
**Runs as the `cc-ci-upgrader` agent.** This skill is normally executed by a dedicated, observable
|
||
**one-shot job agent** — `cc-ci-upgrader` — spun up under remote-control (viewable/steerable at
|
||
claude.ai/code, like the Builder) by `cc-ci-plan/launch-upgrader.sh`. That agent runs this skill to
|
||
completion, then **stops and stays idle** so the run + summary remain reviewable in the web UI (it does
|
||
NOT self-terminate). The weekly cron just invokes `launch-upgrader.sh start`; the next week's run
|
||
clears the idle session and starts fresh. You can also run `/upgrade-all` inline in any `/srv/cc-ci`
|
||
session, but the agent is the intended path so the weekly run isn't buried in headless output.
|
||
|
||
## Arguments (optional `$ARGUMENTS`)
|
||
- A space-separated list of recipe names → only those (else all enrolled recipes).
|
||
- `--dry-run` → survey + print what WOULD upgrade; spawn nothing.
|
||
- `--parallel` → fan out all per-recipe subagents at once (faster, more host load — see safety below).
|
||
|
||
## 1. Build the candidate list
|
||
Enrolled recipes = the cc-ci `tests/<recipe>/` dirs (same set `ci-test-review` sweeps):
|
||
```
|
||
ssh cc-ci 'cd /root/cc-ci/tests && ls -d */' | sed 's#/##' | grep -vE '^(_generic|unit|__pycache__)$'
|
||
```
|
||
(or the names passed in `$ARGUMENTS`). For each, on cc-ci, check availability — skip dirty/up-to-date.
|
||
|
||
> ⚠️ **Three things that silently skip recipes — do ALL THREE per recipe before the version check:**
|
||
> 1. **pseudo-TTY:** abra FATAs `inappropriate ioctl for device` under plain ssh — wrap every abra call
|
||
> in `script`: `ssh cc-ci 'script -qec "abra <args> -n" /dev/null'` (git/other commands need none).
|
||
> 2. **go-git auth to git.autonomic.zone:** recipes on the private mirror FATA `unable to fetch tags …
|
||
> authentication required: Unauthorized` — abra/go-git reads `remote.origin.url` literally and ignores
|
||
> `git url.insteadOf`. Bake creds into origin first (idempotent, only if origin is git.autonomic.zone):
|
||
> `git -C ~/.abra/recipes/<r> remote set-url origin "https://$GITEA_USERNAME:$GITEA_PASSWORD@git.autonomic.zone/recipe-maintainers/<r>.git"`.
|
||
> 3. **dirty worktree** is usually just the untracked cc-ci overlay (`compose.ccci.yml`) — `git stash -u`
|
||
> before / `stash pop` after; only skip `dirty-worktree` if TRACKED changes remain.
|
||
```
|
||
ssh cc-ci 'export PATH=/run/current-system/sw/bin:$PATH; R=<r>; \
|
||
url=$(git -C ~/.abra/recipes/$R config --get remote.origin.url); \
|
||
case "$url" in https://git.autonomic.zone/*) \
|
||
git -C ~/.abra/recipes/$R remote set-url origin "https://'"$GITEA_USERNAME"':'"$GITEA_PASSWORD"'@${url#https://}";; esac; \
|
||
git -C ~/.abra/recipes/$R stash -u >/dev/null 2>&1 || true; \
|
||
script -qec "abra recipe fetch $R --force -n" /dev/null; \
|
||
script -qec "abra recipe upgrade $R -m -n" /dev/null; \
|
||
git -C ~/.abra/recipes/$R stash pop >/dev/null 2>&1 || true'
|
||
```
|
||
Build `RECIPES_TO_UPGRADE` = recipes with **≥1 available upgrade** (after the stash, so the untracked
|
||
overlay doesn't count as dirty). Others go to `SKIPPED_UPFRONT` with a reason (`dirty-worktree` only for
|
||
real TRACKED edits, `up-to-date`, `not-fetchable`).
|
||
|
||
**Reconcile every candidate's mirror during the survey** (even the up-to-date ones), so merged-upstream
|
||
PRs are closed and every mirror `main` tracks true upstream — fleet-wide, not just where there's a new
|
||
upgrade:
|
||
```
|
||
set -a; . /srv/cc-ci/.testenv; set +a
|
||
ssh cc-ci "GITEA_USERNAME='$GITEA_USERNAME' GITEA_PASSWORD='$GITEA_PASSWORD' GITEA_URL='$GITEA_URL' bash -s <recipe> --reconcile-only" \
|
||
< /srv/cc-ci/.claude/skills/recipe-upgrade/open-recipe-pr.sh
|
||
```
|
||
(The per-recipe `/recipe-upgrade` also reconciles, so this is belt-and-suspenders for skipped recipes —
|
||
count closed-merged PRs in the summary.)
|
||
|
||
## 2. Print the plan
|
||
A table — Recipe | Status (will upgrade / skipped:reason) | Available upgrade(s) — plus the mode
|
||
(`Sequential` default / `Parallel`). If `--dry-run`, **stop here**.
|
||
|
||
## 3. Upgrade each recipe via a subagent
|
||
For each recipe in `RECIPES_TO_UPGRADE`, spawn an Agent (`subagent_type: "general-purpose"`,
|
||
description `"Upgrade <recipe> on cc-ci"`) with a prompt like:
|
||
|
||
> Run the `/recipe-upgrade <recipe>` skill end-to-end — **DEFAULT mode, do NOT pass `--with-tests`**
|
||
> (`/srv/cc-ci/.claude/skills/recipe-upgrade/SKILL.md`): plan, implement, open a recipe PR, and verify
|
||
> it by posting `!testme` on the PR (results visible in the PR; iterate ≤3×). Use the **existing
|
||
> tests** — if a test fails because it is genuinely stale, **leave an explanatory comment on the PR**
|
||
> for the operator and do NOT modify any test. Drive cc-ci over `ssh cc-ci`. Do NOT prompt. Do NOT push
|
||
> upstream. Do NOT merge. Print exactly one `RESULT:` line as your final line (SUCCESS /
|
||
> SUCCESS-PENDING-TESTS / FAILED / SKIPPED — `SUCCESS+TESTPR` will not occur in default mode).
|
||
|
||
**Why default (no `--with-tests`):** the weekly cron must never auto-edit cc-ci tests unattended — a
|
||
test change deserves a human decision. So the cron opens recipe PRs only; where a recipe's existing
|
||
test looks stale, the operator sees the explanation in the PR comment and can re-run that one recipe
|
||
with `/recipe-upgrade <recipe> --with-tests` to also get a verified test-update PR.
|
||
|
||
- **Sequential (default):** one Agent at a time; wait, collect its `RESULT:`, then the next. If the
|
||
Agent tool call itself errors, record `FAILED — agent tool error` and continue. One failure never
|
||
aborts the run.
|
||
- **Parallel (`--parallel`):** emit all Agent calls in one message (no `run_in_background` — you need
|
||
the results). Failures are isolated per recipe.
|
||
|
||
## 4. Collect results
|
||
Parse each final `RESULT:` line into SUCCESS / SUCCESS-PENDING-TESTS / FAILED / SKIPPED (default mode
|
||
won't emit `SUCCESS+TESTPR`). A subagent that emitted no `RESULT:` line → `FAILED — no result emitted`.
|
||
|
||
## 5. Write + print the summary
|
||
Write `/srv/cc-ci/.cc-ci-logs/upgrades/upgrade-all-<YYYY-MM-DD>.md` and print it, **leading with the
|
||
PR list** (the actionable output):
|
||
```markdown
|
||
# cc-ci Weekly Upgrade Run — <YYYY-MM-DD>
|
||
## Summary
|
||
- Considered: N · PR green (!testme): N · PR open but tests look stale (commented): N · Failed: N · Skipped: N
|
||
## PRs to review (NOT merged)
|
||
- <recipe> <old> → <new> — recipe PR: <url> — !testme GREEN
|
||
## PRs where a test looks stale (operator: re-run `--with-tests` to update tests)
|
||
- <recipe> <old> → <new> — recipe PR: <url> — !testme RED on <test>; see PR comment
|
||
## Failed (investigate)
|
||
- <recipe> — at <step>: <reason> (log: .cc-ci-logs/upgrades/<recipe>-upgrade-<date>.md)
|
||
## Skipped
|
||
| recipe | reason |
|
||
```
|
||
End with the report path and a reminder that **nothing was merged**.
|
||
|
||
## 6. Launch the public Recipe Report
|
||
Once the summary is written, kick off the weekly public report (its own agent, separate model — default
|
||
opus): `python3 /srv/cc-ci/cc-ci-plan/launch-report.py start`. It runs `/recipe-report`, reviews this
|
||
run + the live recipe/PR state, and publishes to https://report.ci.commoninternet.net. Fire-and-forget
|
||
— it runs independently; you can then go idle.
|
||
|
||
## Safety / coordination (this matters — shared host with the build loops)
|
||
- **Sequential is the default for a reason.** Recipe deploys are **stateful on the shared Swarm** and
|
||
parallel deploys can OOM/collide. Between sequential recipes, the per-recipe `recipe-upgrade` tears
|
||
down what it deployed; verify a recipe is undeployed before the next starts
|
||
(`ssh cc-ci 'script -qec "abra app ls -n" /dev/null'` — pseudo-TTY wrapped, per the box above).
|
||
- **Single-writer:** every PR (recipe or cc-ci test) is on a dedicated branch; **never push `main`**,
|
||
never touch the build loops' `/cc-ci` `/cc-ci-adv` working clones or their in-flight state.
|
||
- **Contention with active loop development:** while the loops are still building cc-ci, this run
|
||
competes for the host. Prefer a quiescent window; if a recipe fails due to contention it's simply
|
||
retried next week. (Once cc-ci is built and the loops are idle, this is the steady-state weekly job.)
|
||
- **Never merges**; failures/ skips are surfaced and retried next week — safe to re-run anytime.
|
||
|
||
## Cron
|
||
Designed for a weekly Claude Code scheduled task that runs **`cc-ci-plan/launch-upgrader.sh start`** —
|
||
which spins up the `cc-ci-upgrader` remote-control agent to run this skill to completion (the agent
|
||
then stays idle/viewable). The cron does NOT invoke `/upgrade-all` inline.
|
||
|
||
**Schedule (operator, 2026-05-29):** the weekly cron is installed as the **closing action of the final
|
||
build phase (Phase 5)**, and its **first run is ~1 hour after the build completes, weekly from then on**
|
||
— i.e. anchored to actual completion, not a fixed clock slot. Compute `T0 = completion + 1h` and
|
||
install a weekly job at T0's day-of-week + `HH:MM` (`MM HH * * DOW`) running `launch-upgrader.sh
|
||
start`. Then **verify the first kickoff** actually launches the `cc-ci-upgrader` agent. Do NOT install
|
||
it earlier — while the loops are still building cc-ci it would contend for the shared host; until then
|
||
run it manually via `launch-upgrader.sh start`/`fresh`. Full procedure: Phase 5 plan §4
|
||
(`cc-ci-plan/plan-phase5-verify-upgrade-flow.md`).
|
||
|
||
Re-running is idempotent: already-current recipes report `SKIPPED — up-to-date`; recipes with an open
|
||
PR for the same branch report the existing PR rather than duplicating it.
|