Per operator: an open mirror PR must mean "genuinely still open against true current upstream main". On every run the recipe-upgrade flow now: - force-syncs the recipe-maintainers/<recipe> mirror `main` to be IDENTICAL to upstream main (origin/main of the abra checkout = coopcloud); - closes any open mirror PR whose changes are already in upstream main (merged upstream, no-op merge detected via `git merge-tree` vs main's tree) — even when the recipe is up to date (new `--reconcile-only` mode, run in step 1); - when opening a new upgrade PR, closes any other still-open PR for that recipe (superseded) and opens the new one IN ITS PLACE; same-version re-runs just update the existing same-branch PR. open-recipe-pr.sh gains the --reconcile-only mode + the close logic (with an auto-close comment naming the reason). upgrade-all reconciles every candidate's mirror during the survey so merged PRs are closed fleet-wide. Still never merges. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
112 lines
6.4 KiB
Markdown
112 lines
6.4 KiB
Markdown
---
|
|
name: upgrade-all
|
|
description: Weekly autonomous upgrade run for the cc-ci CI server. Surveys every enrolled recipe for available upstream upgrades, then runs /recipe-upgrade on each upgradeable one via a subagent — plan, implement, verify green on cc-ci, open a recipe PR (and, only if a cc-ci test went stale, a verified cc-ci test PR). Collects results into one summary listing every PR to review. Sequential by default (shared Swarm); --parallel to fan out; --dry-run to preview. NEVER merges. Built to run once weekly on a cron. Invoke as /upgrade-all.
|
|
---
|
|
|
|
# upgrade-all
|
|
|
|
The cc-ci analogue of recipe-maintainer's `/recipe-upgrade-cron-all`: an unattended **weekly** pass
|
|
that keeps every enrolled recipe current, with **cc-ci as the test gate** and a human in the loop only
|
|
where it matters — **PR review**. It surveys upgrades, runs `/recipe-upgrade <recipe>` per upgradeable
|
|
recipe, and writes one summary of every PR to review. It **never pushes upstream and never merges**.
|
|
|
|
Drives cc-ci over `ssh cc-ci`. Logs/summary go to `/srv/cc-ci/.cc-ci-logs/upgrades/`.
|
|
|
|
## Arguments (optional `$ARGUMENTS`)
|
|
- A space-separated list of recipe names → only those (else all enrolled recipes).
|
|
- `--dry-run` → survey + print what WOULD upgrade; spawn nothing.
|
|
- `--parallel` → fan out all per-recipe subagents at once (faster, more host load — see safety below).
|
|
|
|
## 1. Build the candidate list
|
|
Enrolled recipes = the cc-ci `tests/<recipe>/` dirs (same set `ci-test-review` sweeps):
|
|
```
|
|
ssh cc-ci 'cd /root/cc-ci/tests && ls -d */' | sed 's#/##' | grep -vE '^(_generic|unit|__pycache__)$'
|
|
```
|
|
(or the names passed in `$ARGUMENTS`). For each, on cc-ci, check availability — skip dirty/up-to-date:
|
|
```
|
|
ssh cc-ci 'export PATH=/run/current-system/sw/bin:$PATH; \
|
|
git -C ~/.abra/recipes/<r> status --short; \
|
|
abra recipe fetch <r> --force; \
|
|
abra recipe upgrade <r> -m -n'
|
|
```
|
|
Build `RECIPES_TO_UPGRADE` = recipes with a **clean worktree** AND **≥1 available upgrade**. Others go
|
|
to `SKIPPED_UPFRONT` with a reason (`dirty-worktree`, `up-to-date`, `not-fetchable`).
|
|
|
|
**Reconcile every candidate's mirror during the survey** (even the up-to-date ones), so merged-upstream
|
|
PRs are closed and every mirror `main` tracks true upstream — fleet-wide, not just where there's a new
|
|
upgrade:
|
|
```
|
|
set -a; . /srv/cc-ci/.testenv; set +a
|
|
ssh cc-ci "GITEA_USERNAME='$GITEA_USERNAME' GITEA_PASSWORD='$GITEA_PASSWORD' GITEA_URL='$GITEA_URL' bash -s <recipe> --reconcile-only" \
|
|
< /srv/cc-ci/.claude/skills/recipe-upgrade/open-recipe-pr.sh
|
|
```
|
|
(The per-recipe `/recipe-upgrade` also reconciles, so this is belt-and-suspenders for skipped recipes —
|
|
count closed-merged PRs in the summary.)
|
|
|
|
## 2. Print the plan
|
|
A table — Recipe | Status (will upgrade / skipped:reason) | Available upgrade(s) — plus the mode
|
|
(`Sequential` default / `Parallel`). If `--dry-run`, **stop here**.
|
|
|
|
## 3. Upgrade each recipe via a subagent
|
|
For each recipe in `RECIPES_TO_UPGRADE`, spawn an Agent (`subagent_type: "general-purpose"`,
|
|
description `"Upgrade <recipe> on cc-ci"`) with a prompt like:
|
|
|
|
> Run the `/recipe-upgrade <recipe>` skill end-to-end
|
|
> (`/srv/cc-ci/.claude/skills/recipe-upgrade/SKILL.md`): plan, implement, **verify green on cc-ci**,
|
|
> open a recipe PR, and — only if the upgrade is correct but a cc-ci test went stale — open a verified
|
|
> cc-ci test PR too. Drive cc-ci over `ssh cc-ci`. Do NOT prompt. Do NOT push upstream. Do NOT merge.
|
|
> Print exactly one `RESULT:` line as your final line (the SUCCESS / SUCCESS+TESTPR / FAILED / SKIPPED
|
|
> forms from the recipe-upgrade skill).
|
|
|
|
- **Sequential (default):** one Agent at a time; wait, collect its `RESULT:`, then the next. If the
|
|
Agent tool call itself errors, record `FAILED — agent tool error` and continue. One failure never
|
|
aborts the run.
|
|
- **Parallel (`--parallel`):** emit all Agent calls in one message (no `run_in_background` — you need
|
|
the results). Failures are isolated per recipe.
|
|
|
|
## 4. Collect results
|
|
Parse each final `RESULT:` line into SUCCESS / SUCCESS+TESTPR / FAILED / SKIPPED. A subagent that
|
|
emitted no `RESULT:` line → `FAILED — no result emitted`.
|
|
|
|
## 5. Write + print the summary
|
|
Write `/srv/cc-ci/.cc-ci-logs/upgrades/upgrade-all-<YYYY-MM-DD>.md` and print it, **leading with the
|
|
PR list** (the actionable output):
|
|
```markdown
|
|
# cc-ci Weekly Upgrade Run — <YYYY-MM-DD>
|
|
## Summary
|
|
- Considered: N · Upgraded (PR opened): N · With cc-ci test PR: N · Failed: N · Skipped: N
|
|
## PRs to review (NOT merged)
|
|
- <recipe> <old> → <new> — recipe PR: <url> [+ cc-ci test PR: <url>]
|
|
## Failed (investigate)
|
|
- <recipe> — at <step>: <reason> (log: .cc-ci-logs/upgrades/<recipe>-upgrade-<date>.md)
|
|
## Skipped
|
|
| recipe | reason |
|
|
```
|
|
End with the report path and a reminder that **nothing was merged**.
|
|
|
|
## Safety / coordination (this matters — shared host with the build loops)
|
|
- **Sequential is the default for a reason.** Recipe deploys are **stateful on the shared Swarm** and
|
|
parallel deploys can OOM/collide. Between sequential recipes, the per-recipe `recipe-upgrade` tears
|
|
down what it deployed; verify a recipe is undeployed before the next starts (`abra app ls` on cc-ci).
|
|
- **Single-writer:** every PR (recipe or cc-ci test) is on a dedicated branch; **never push `main`**,
|
|
never touch the build loops' `/cc-ci` `/cc-ci-adv` working clones or their in-flight state.
|
|
- **Contention with active loop development:** while the loops are still building cc-ci, this run
|
|
competes for the host. Prefer a quiescent window; if a recipe fails due to contention it's simply
|
|
retried next week. (Once cc-ci is built and the loops are idle, this is the steady-state weekly job.)
|
|
- **Never merges**; failures/ skips are surfaced and retried next week — safe to re-run anytime.
|
|
|
|
## Cron
|
|
Designed for a weekly Claude Code scheduled task that invokes `/upgrade-all` in `/srv/cc-ci`.
|
|
|
|
**Agreed schedule:** **Saturday 03:00 UTC** (`0 3 * * 6`) — low-traffic weekend window, PRs waiting by
|
|
Monday.
|
|
|
|
**Activation trigger (operator, 2026-05-29):** do NOT activate while the build loops are still
|
|
constructing cc-ci — it would contend with them for the shared host. **Activate this weekly cron only
|
|
once the cc-ci build is complete (loops finished / cc-ci stable.)** Until then it's run manually /
|
|
on-demand. When activating, create a scheduled task that runs `/upgrade-all` in `/srv/cc-ci` at
|
|
`0 3 * * 6` UTC.
|
|
|
|
Re-running is idempotent: already-current recipes report `SKIPPED — up-to-date`; recipes with an open
|
|
PR for the same branch report the existing PR rather than duplicating it.
|