ci-test-review: close the loop — author + open + cc-ci-verify fix PRs (never merge)

Per operator: the skill should not just propose, it should CREATE the fix PR
(recipe repo or cc-ci repo) and VERIFY it green on its own CI server — but not
merge. It drives cc-ci like the loops do.

- SKILL.md: diagnose+classify (recipe vs CI-server) -> author the fix + open a
  PR (recipe-create-pr for recipe PRs; Gitea API for cc-ci PRs, dedicated branch
  in a separate clone, single-writer safe) -> VERIFY on cc-ci (full suite cold
  against the PR head = the !testme dogfood path) -> report a verified,
  ready-to-merge PR. Never merges; never weakens a test; flake != bug. General
  bar = one cold green; repeated-green (REPEAT=3) only for a known-flaky recipe.
  Adds coordination/single-writer guardrails (shared Swarm is stateful; tear
  down deploys; never push main or touch the loops' clones).
- verify-pr.sh: deterministic recipe-PR gate — RECIPE + REF -> cold full suite
  on cc-ci, green iff every repeat exits 0. CI-server-PR verification stays
  bespoke (branch checkout + rebuild + regression sample) per SKILL.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-05-29 17:04:35 +01:00
co-authored by Claude Opus 4.8
parent 2530845e50
commit cbe1406bce
2 changed files with 126 additions and 30 deletions
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
# ci-test-review :: deterministic PR verification on the CI server (NO AI)
# -------------------------------------------------------------------------
# Verifies a RECIPE PR the same way `!testme` does: deploy + run the FULL suite
# COLD against the PR head on cc-ci. The harness decides pass/fail — this script
# only runs it and reports the exit code + RUN SUMMARY. AI never judges here.
#
# RECIPE=ghost REF=my-fix-branch verify-pr.sh # 1 cold green = working
# RECIPE=lasuite-drive REF=sha REPEAT=3 verify-pr.sh # repeated-green: ONLY
# # for a known-flaky recipe
#
# REF is the recipe PR head (branch name or sha) on the recipe mirror that abra
# fetches from. Green iff EVERY repeat exits 0.
#
# (CI-SERVER PRs are verified differently — check the branch out in a clone on
# cc-ci, rebuild, re-run the failing recipe(s) + a regression sample — see
# SKILL.md step 5; that path is bespoke and not scripted here.)
set -o errexit -o nounset -o pipefail
SSH="${SSH:-cc-ci}"
REPEAT="${REPEAT:-1}"
: "${RECIPE:?set RECIPE (e.g. ghost)}"
: "${REF:?set REF (the PR head branch or sha)}"
RUNID="$(date -u +%Y%m%dT%H%M%SZ)"
REMOTE_LOG="/root/cc-ci-review-logs/verify-${RECIPE}-${RUNID}"
ssh "$SSH" "mkdir -p /root/cc-ci-review-logs"
echo "verify-pr: RECIPE=$RECIPE REF=$REF cold full-suite x${REPEAT} on ${SSH}" >&2
green=1
for i in $(seq 1 "$REPEAT"); do
log="${REMOTE_LOG}.${i}.log"
rc=0
# Real harness, cold (no --quick), against the PR head — same path as !testme.
ssh "$SSH" "cd /root/cc-ci && RECIPE='${RECIPE}' REF='${REF}' cc-ci-run runner/run_recipe_ci.py >'${log}' 2>&1" || rc=$?
echo "--- pass ${i}/${REPEAT}: exit ${rc} (log ${SSH}:${log}) ---" >&2
ssh "$SSH" "awk '/===== RUN SUMMARY =====/{f=1} f{print}' '${log}'" >&2 || true
[ "$rc" = "0" ] || green=0
done
if [ "$green" = "1" ]; then
echo "VERDICT: GREEN — ${RECIPE} PR (REF=${REF}) passed cold full-suite x${REPEAT}. Ready for operator merge (NOT merged)." >&2
exit 0
else
echo "VERDICT: RED — ${RECIPE} PR (REF=${REF}) did not pass. Iterate the fix (<=3 attempts) or report needs-work. Do NOT weaken tests; do NOT merge." >&2
exit 2
fi