autonomic-botandClaude Opus 4.8
027fdbd161
abra over plain 'ssh cc-ci abra ...' has no TTY -> FATA 'inappropriate ioctl for device' (the abra error). The working harness (runner/harness/abra.py) wraps abra in util-linux 'script' for a pseudo-TTY + passes -n. Apply the same in the recipe-upgrade and upgrade-all skills: every abra call becomes ssh cc-ci 'script -qec "abra <args> -n" /dev/null'. Confirmed: abra server ls FATAs plain, works pty-wrapped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
135 lines
8.8 KiB
Markdown
135 lines
8.8 KiB
Markdown
---
|
||
name: upgrade-all
|
||
description: Weekly autonomous upgrade run for the cc-ci CI server. Surveys every enrolled recipe for available upstream upgrades, then runs /recipe-upgrade on each upgradeable one via a subagent — plan, implement, verify green on cc-ci, open a recipe PR (and, only if a cc-ci test went stale, a verified cc-ci test PR). Collects results into one summary listing every PR to review. Sequential by default (shared Swarm); --parallel to fan out; --dry-run to preview. NEVER merges. Built to run once weekly on a cron. Invoke as /upgrade-all.
|
||
---
|
||
|
||
# upgrade-all
|
||
|
||
The cc-ci analogue of recipe-maintainer's `/recipe-upgrade-cron-all`: an unattended **weekly** pass
|
||
that keeps every enrolled recipe current, with **cc-ci as the test gate** and a human in the loop only
|
||
where it matters — **PR review**. It surveys upgrades, runs `/recipe-upgrade <recipe>` per upgradeable
|
||
recipe, and writes one summary of every PR to review. It **never pushes upstream and never merges**.
|
||
|
||
Drives cc-ci over `ssh cc-ci`. Logs/summary go to `/srv/cc-ci/.cc-ci-logs/upgrades/`.
|
||
|
||
**Runs as the `cc-ci-upgrader` agent.** This skill is normally executed by a dedicated, observable
|
||
**one-shot job agent** — `cc-ci-upgrader` — spun up under remote-control (viewable/steerable at
|
||
claude.ai/code, like the Builder) by `cc-ci-plan/launch-upgrader.sh`. That agent runs this skill to
|
||
completion, then **stops and stays idle** so the run + summary remain reviewable in the web UI (it does
|
||
NOT self-terminate). The weekly cron just invokes `launch-upgrader.sh start`; the next week's run
|
||
clears the idle session and starts fresh. You can also run `/upgrade-all` inline in any `/srv/cc-ci`
|
||
session, but the agent is the intended path so the weekly run isn't buried in headless output.
|
||
|
||
## Arguments (optional `$ARGUMENTS`)
|
||
- A space-separated list of recipe names → only those (else all enrolled recipes).
|
||
- `--dry-run` → survey + print what WOULD upgrade; spawn nothing.
|
||
- `--parallel` → fan out all per-recipe subagents at once (faster, more host load — see safety below).
|
||
|
||
## 1. Build the candidate list
|
||
Enrolled recipes = the cc-ci `tests/<recipe>/` dirs (same set `ci-test-review` sweeps):
|
||
```
|
||
ssh cc-ci 'cd /root/cc-ci/tests && ls -d */' | sed 's#/##' | grep -vE '^(_generic|unit|__pycache__)$'
|
||
```
|
||
(or the names passed in `$ARGUMENTS`). For each, on cc-ci, check availability — skip dirty/up-to-date.
|
||
|
||
> ⚠️ **Run abra over a pseudo-TTY** or it FATAs `inappropriate ioctl for device` (no TTY under plain
|
||
> ssh). Wrap every abra call in `script` (the working-harness fix in `runner/harness/abra.py`) and pass
|
||
> `-n`: `ssh cc-ci 'script -qec "abra <args> -n" /dev/null'`. `git`/other commands need no wrapper.
|
||
```
|
||
ssh cc-ci 'export PATH=/run/current-system/sw/bin:$PATH; \
|
||
git -C ~/.abra/recipes/<r> status --short; \
|
||
script -qec "abra recipe fetch <r> --force -n" /dev/null; \
|
||
script -qec "abra recipe upgrade <r> -m -n" /dev/null'
|
||
```
|
||
Build `RECIPES_TO_UPGRADE` = recipes with a **clean worktree** AND **≥1 available upgrade**. Others go
|
||
to `SKIPPED_UPFRONT` with a reason (`dirty-worktree`, `up-to-date`, `not-fetchable`).
|
||
|
||
**Reconcile every candidate's mirror during the survey** (even the up-to-date ones), so merged-upstream
|
||
PRs are closed and every mirror `main` tracks true upstream — fleet-wide, not just where there's a new
|
||
upgrade:
|
||
```
|
||
set -a; . /srv/cc-ci/.testenv; set +a
|
||
ssh cc-ci "GITEA_USERNAME='$GITEA_USERNAME' GITEA_PASSWORD='$GITEA_PASSWORD' GITEA_URL='$GITEA_URL' bash -s <recipe> --reconcile-only" \
|
||
< /srv/cc-ci/.claude/skills/recipe-upgrade/open-recipe-pr.sh
|
||
```
|
||
(The per-recipe `/recipe-upgrade` also reconciles, so this is belt-and-suspenders for skipped recipes —
|
||
count closed-merged PRs in the summary.)
|
||
|
||
## 2. Print the plan
|
||
A table — Recipe | Status (will upgrade / skipped:reason) | Available upgrade(s) — plus the mode
|
||
(`Sequential` default / `Parallel`). If `--dry-run`, **stop here**.
|
||
|
||
## 3. Upgrade each recipe via a subagent
|
||
For each recipe in `RECIPES_TO_UPGRADE`, spawn an Agent (`subagent_type: "general-purpose"`,
|
||
description `"Upgrade <recipe> on cc-ci"`) with a prompt like:
|
||
|
||
> Run the `/recipe-upgrade <recipe>` skill end-to-end — **DEFAULT mode, do NOT pass `--with-tests`**
|
||
> (`/srv/cc-ci/.claude/skills/recipe-upgrade/SKILL.md`): plan, implement, open a recipe PR, and verify
|
||
> it by posting `!testme` on the PR (results visible in the PR; iterate ≤3×). Use the **existing
|
||
> tests** — if a test fails because it is genuinely stale, **leave an explanatory comment on the PR**
|
||
> for the operator and do NOT modify any test. Drive cc-ci over `ssh cc-ci`. Do NOT prompt. Do NOT push
|
||
> upstream. Do NOT merge. Print exactly one `RESULT:` line as your final line (SUCCESS /
|
||
> SUCCESS-PENDING-TESTS / FAILED / SKIPPED — `SUCCESS+TESTPR` will not occur in default mode).
|
||
|
||
**Why default (no `--with-tests`):** the weekly cron must never auto-edit cc-ci tests unattended — a
|
||
test change deserves a human decision. So the cron opens recipe PRs only; where a recipe's existing
|
||
test looks stale, the operator sees the explanation in the PR comment and can re-run that one recipe
|
||
with `/recipe-upgrade <recipe> --with-tests` to also get a verified test-update PR.
|
||
|
||
- **Sequential (default):** one Agent at a time; wait, collect its `RESULT:`, then the next. If the
|
||
Agent tool call itself errors, record `FAILED — agent tool error` and continue. One failure never
|
||
aborts the run.
|
||
- **Parallel (`--parallel`):** emit all Agent calls in one message (no `run_in_background` — you need
|
||
the results). Failures are isolated per recipe.
|
||
|
||
## 4. Collect results
|
||
Parse each final `RESULT:` line into SUCCESS / SUCCESS-PENDING-TESTS / FAILED / SKIPPED (default mode
|
||
won't emit `SUCCESS+TESTPR`). A subagent that emitted no `RESULT:` line → `FAILED — no result emitted`.
|
||
|
||
## 5. Write + print the summary
|
||
Write `/srv/cc-ci/.cc-ci-logs/upgrades/upgrade-all-<YYYY-MM-DD>.md` and print it, **leading with the
|
||
PR list** (the actionable output):
|
||
```markdown
|
||
# cc-ci Weekly Upgrade Run — <YYYY-MM-DD>
|
||
## Summary
|
||
- Considered: N · PR green (!testme): N · PR open but tests look stale (commented): N · Failed: N · Skipped: N
|
||
## PRs to review (NOT merged)
|
||
- <recipe> <old> → <new> — recipe PR: <url> — !testme GREEN
|
||
## PRs where a test looks stale (operator: re-run `--with-tests` to update tests)
|
||
- <recipe> <old> → <new> — recipe PR: <url> — !testme RED on <test>; see PR comment
|
||
## Failed (investigate)
|
||
- <recipe> — at <step>: <reason> (log: .cc-ci-logs/upgrades/<recipe>-upgrade-<date>.md)
|
||
## Skipped
|
||
| recipe | reason |
|
||
```
|
||
End with the report path and a reminder that **nothing was merged**.
|
||
|
||
## Safety / coordination (this matters — shared host with the build loops)
|
||
- **Sequential is the default for a reason.** Recipe deploys are **stateful on the shared Swarm** and
|
||
parallel deploys can OOM/collide. Between sequential recipes, the per-recipe `recipe-upgrade` tears
|
||
down what it deployed; verify a recipe is undeployed before the next starts
|
||
(`ssh cc-ci 'script -qec "abra app ls -n" /dev/null'` — pseudo-TTY wrapped, per the box above).
|
||
- **Single-writer:** every PR (recipe or cc-ci test) is on a dedicated branch; **never push `main`**,
|
||
never touch the build loops' `/cc-ci` `/cc-ci-adv` working clones or their in-flight state.
|
||
- **Contention with active loop development:** while the loops are still building cc-ci, this run
|
||
competes for the host. Prefer a quiescent window; if a recipe fails due to contention it's simply
|
||
retried next week. (Once cc-ci is built and the loops are idle, this is the steady-state weekly job.)
|
||
- **Never merges**; failures/ skips are surfaced and retried next week — safe to re-run anytime.
|
||
|
||
## Cron
|
||
Designed for a weekly Claude Code scheduled task that runs **`cc-ci-plan/launch-upgrader.sh start`** —
|
||
which spins up the `cc-ci-upgrader` remote-control agent to run this skill to completion (the agent
|
||
then stays idle/viewable). The cron does NOT invoke `/upgrade-all` inline.
|
||
|
||
**Schedule (operator, 2026-05-29):** the weekly cron is installed as the **closing action of the final
|
||
build phase (Phase 5)**, and its **first run is ~1 hour after the build completes, weekly from then on**
|
||
— i.e. anchored to actual completion, not a fixed clock slot. Compute `T0 = completion + 1h` and
|
||
install a weekly job at T0's day-of-week + `HH:MM` (`MM HH * * DOW`) running `launch-upgrader.sh
|
||
start`. Then **verify the first kickoff** actually launches the `cc-ci-upgrader` agent. Do NOT install
|
||
it earlier — while the loops are still building cc-ci it would contend for the shared host; until then
|
||
run it manually via `launch-upgrader.sh start`/`fresh`. Full procedure: Phase 5 plan §4
|
||
(`cc-ci-plan/plan-phase5-verify-upgrade-flow.md`).
|
||
|
||
Re-running is idempotent: already-current recipes report `SKIPPED — up-to-date`; recipes with an open
|
||
PR for the same branch report the existing PR rather than duplicating it.
|