From 5ade783a50c0326d6dcc2502649b9e99eeb6df4e Mon Sep 17 00:00:00 2001 From: autonomic-bot Date: Mon, 3 Aug 2026 23:12:49 +0000 Subject: [PATCH] =?UTF-8?q?skills:=20add=20/cc-ci-status=20=E2=80=94=20com?= =?UTF-8?q?prehensive=20read-only=20system=20status=20check?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Seven check areas: weekly-run recency+outcome, report publishing, stale recipes/tests, open recipe PRs (CVE-carrying PRs open >14d flagged high-priority, ready-to-merge PRs listed as normal), server+orchestrator flake-update recency vs channel tip, host/service health incl. the bridge !testme path (silent-401 stale-secret check from the 2026-08-03 finding), and maintained-set consistency. Ends with ALL HEALTHY or a prioritized findings list, each mapped to the skill to invoke. Strictly read-only. --- .claude/skills/cc-ci-status/SKILL.md | 14 +++ .opencode/skills/cc-ci-status/SKILL.md | 119 +++++++++++++++++++++++++ 2 files changed, 133 insertions(+) create mode 100644 .claude/skills/cc-ci-status/SKILL.md create mode 100644 .opencode/skills/cc-ci-status/SKILL.md diff --git a/.claude/skills/cc-ci-status/SKILL.md b/.claude/skills/cc-ci-status/SKILL.md new file mode 100644 index 0000000..dc2ab86 --- /dev/null +++ b/.claude/skills/cc-ci-status/SKILL.md @@ -0,0 +1,14 @@ +--- +name: cc-ci-status +description: Comprehensive read-only status check of the whole cc-ci system - how the recent weekly upgrade runs went and whether their reports published, which recipes/tests are stale, how long since the server + orchestrator host flake updates, open recipe PRs (flagging CVE-carrying PRs that have been open too long), host health (failed units, disk, timers, bridge/!testme path), ending with a verdict (ALL HEALTHY or a findings list) and recommended next steps mapped to the skills to invoke. Never changes anything - it only reads and reports. Invoke as /cc-ci-status. +--- + +# cc-ci-status (thin wrapper) + +The canonical definition of this skill lives in the **opencode** position: + +**`.opencode/skills/cc-ci-status/SKILL.md`** + +Read that file for the full procedure. This `.claude/skills/` copy is kept as a +thin pointer for Claude Code compatibility; opencode loads the canonical +definition from `.opencode/skills/` directly. diff --git a/.opencode/skills/cc-ci-status/SKILL.md b/.opencode/skills/cc-ci-status/SKILL.md new file mode 100644 index 0000000..21649d5 --- /dev/null +++ b/.opencode/skills/cc-ci-status/SKILL.md @@ -0,0 +1,119 @@ +--- +name: cc-ci-status +description: Comprehensive read-only status check of the whole cc-ci system - how the recent weekly upgrade runs went and whether their reports published, which recipes/tests are stale, how long since the server + orchestrator host flake updates, open recipe PRs (flagging CVE-carrying PRs that have been open too long), host health (failed units, disk, timers, bridge/!testme path), ending with a verdict (ALL HEALTHY or a findings list) and recommended next steps mapped to the skills to invoke. Never changes anything - it only reads and reports. Invoke as /cc-ci-status. +--- + +# cc-ci-status + +One comprehensive, **read-only** status pass over the cc-ci system. Output ends with either +**`ALL HEALTHY`** or a prioritized findings list, each finding paired with the skill to invoke. +Nothing here mutates state — no restarts, no deploys, no merges. + +## Checks (run all; collect findings, don't stop at the first) + +### 1. Weekly upgrade runs — recency + outcome + +``` +ls -t /srv/cc-ci/.cc-ci-logs/upgrades/upgrade-all-*.md | head -3 +head -20 # the Summary block: Considered/green/stale/Failed/Skipped +systemctl list-timers cc-ci-upgrade-all.timer --no-pager | head -3 +``` +- **Overdue** if the newest report is >8 days old, or the timer is inactive/missing → recommend + `systemctl start cc-ci-upgrade-all.service` (or investigate the timer) / `/upgrade-all`. +- **Failed entries** in the latest report → recommend `/recipe-upgrade ` per entry (or + `/ci-test-review` if the failure is harness-side). +- A run currently in flight (tmux `cc-ci-upgrader` session live) is NOT a finding — report it as + in-progress and skip staleness checks that depend on its output. + +### 2. Report publishing — report.ci.commoninternet.net + +``` +curl -s -o /dev/null -w '%{http_code}' https://report.ci.commoninternet.net/ +ls -t /var/lib/cc-ci-reports/week-*.html | head -2 # via ssh cc-ci +``` +- Index must be 200 and there must be a `week-*.html` at least as new as the last **completed** + upgrade run (a completed run without a matching page = report generation broke → recommend + `/recipe-report` / inspect `launch-report.py`). + +### 3. Stale recipes / stale tests + +- Latest report's "PRs where a test looks stale" section + carry-over notes. +- Any entries → recommend `/cc-ci-tests-update` (fleet) or `/recipe-upgrade + --with-tests` (single). +- Also check `/srv/cc-ci/.cc-ci-logs/tests-update-*.md` recency — if stale tests were reported + weeks ago and no tests-update run since, say so. + +### 4. Open recipe PRs — especially CVE-carrying ones that linger + +Enumerate open PRs across `recipe-maintainers/*` (Gitea API, creds in `/srv/cc-ci/.testenv`): +``` +GET /repos/recipe-maintainers//pulls?state=open # repos = the used-recipes.md inventory + cc-ci +``` +For each open PR: age (now − created_at), and whether the PR title/body/report row mentions +**CVE** / security patch. +- **CVE-carrying PR open >14 days** → HIGH-priority finding: name the PR, the CVE context, and + the blocker (commonly a stale test — check the report row) → recommend the unblocking skill + (`/cc-ci-tests-update`) plus "operator: review + merge ". +- Non-CVE PRs open >30 days → low-priority note ("operator review backlog: N PRs"). +- Verified-green PRs awaiting operator merge are normal — list them as "ready to merge", not + as failures. + +### 5. Server + orchestrator host update recency + +For BOTH hosts: +``` +# cc-ci server: ssh cc-ci 'nixos-version; cd /root/cc-ci-deploy && nix flake metadata --json' (or builder-clone) +# orchestrator: nixos-version; cd /srv/cc-ci-orch && nix flake metadata --json +git ls-remote https://github.com/NixOS/nixpkgs # current tip +ls -t /srv/cc-ci-orch/.cc-ci-logs/server-update-*.md /srv/cc-ci-orch/.cc-ci-logs/orchestrator-update-*.md | head -2 +``` +- Report: days since last update log + how far the running nixpkgs rev lags the channel tip. +- **Lagging >30 days** (or a NixOS release behind) → recommend `/cc-ci-server-update` / + `/cc-ci-orchestrator-update`. + +### 6. Host + service health (both machines) + +``` +ssh cc-ci 'systemctl --failed --no-legend; df -h / | tail -1; docker service ls --format "{{.Name}} {{.Replicas}}"' +systemctl --failed --no-legend; df -h / | tail -1; tmux ls +``` +- Failed units, core swarm services not 1/1 (warm-* spares flapping is a known benign pattern — + note, don't page), disk >80% (server) / >85% (orchestrator) → findings. Server unreachable → + HIGH: recommend `hetzner-server-recovery`. +- **Bridge / !testme path**: `docker service ls` shows `ccci-bridge_app 1/1` AND the bridge log + has no auth errors (`docker service logs --since 24h ccci-bridge_app 2>&1 | grep -ci "401\|user does not exist"` == 0). + A silently-401ing bridge drops `!testme` (seen 2026-08-03, stale rotated Gitea secret) → + recommend refreshing the bridge secret + redeploy (test-before-switch). + +### 7. Maintained-set consistency (quick) + +- Every `weekly` row in `used-recipes.md` has `tests//` on cc-ci AND a + `recipe-maintainers/` mirror AND is in bridge `POLL_REPOS`; mismatches → recommend + finishing enrollment (`/recipe-enroll ` covers all touchpoints). + +## Output format + +``` +# cc-ci status — +## Verdict: ALL HEALTHY | N findings (M high-priority) +## Weekly upgrades: +## Report site: +## Stale tests: +## Open PRs: +## Host updates: server · orchestrator +## Health: server · orchestrator · bridge +## Recommended next steps +1. → / (or operator action) +… +``` +`ALL HEALTHY` requires: recent successful weekly run + published report, no stale tests, no +CVE PR open >14 days, both hosts <30 days behind their channel, zero failed units, disk under +thresholds, bridge clean, maintained-set consistent. Anything else is a finding — even minor +ones get a recommended next step. Order findings by priority (CVE/unreachable-host first). + +## Guardrails + +- **Read-only.** This skill diagnoses and recommends; it invokes nothing and changes nothing. +- Use the pseudo-TTY wrap for any abra call; plain ssh for everything else. +- Don't double-count: a finding that explains another (bridge 401 → !testme "failures") gets + reported once, at the root cause.