recipe-upgrade: !testme-on-PR verification + make test PRs opt-in (--with-tests)

Per operator:
- Verify via `!testme` posted ON the recipe PR (the real CI path) so results are
  viewable in the PR; iterate up to 3 !testme runs (fix a real regression + re-test).
  New helper testme-on-pr.sh posts !testme and polls the PR head commit status
  for the verdict (POST=0 to keep polling without re-triggering).
- Test updates are now OPT-IN via `--with-tests`. DEFAULT: recipe PR only using
  existing tests; if a test fails and is genuinely stale, leave an explanatory
  COMMENT on the PR (upgrade looks correct; re-run --with-tests to update tests)
  and do NOT touch any test. --with-tests keeps the verified cc-ci test-update PR
  path (verified via the branch-checkout harness run, since !testme uses prod tests).
- upgrade-all (weekly cron) calls the DEFAULT — never auto-edits tests unattended;
  surfaces "tests look stale" PRs in the summary for the operator to opt in per-recipe.
- New RESULT: SUCCESS-PENDING-TESTS for the recipe-green-but-test-stale default case.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-05-29 20:18:59 +01:00
co-authored by Claude Opus 4.8
parent c7da03fa6c
commit 4a1da1dd60
3 changed files with 134 additions and 38 deletions
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env bash
# recipe-upgrade :: trigger cc-ci on a recipe PR via `!testme` and read the verdict FROM the PR.
# -------------------------------------------------------------------------
# Runs on the ORCHESTRATOR (just hits the Gitea API with /srv/cc-ci/.testenv creds). Posting
# `!testme` as a PR comment is the REAL CI path — the bridge runs the harness on cc-ci and the
# run + its results are posted back to the PR, so the operator can see them in the PR itself.
#
# testme-on-pr.sh <recipe> <pr-index>
# env: POST=1 post a fresh `!testme` first (default). POST=0 = poll an in-flight run only,
# WITHOUT re-triggering (use on follow-up calls so you don't launch duplicate runs).
# MAX_WAIT=480 INTERVAL=30 poll window per call, in seconds. 480s stays under the 10-min
# tool cap — if it prints VERDICT=PENDING, just run again with POST=0 to keep polling.
#
# Prints: VERDICT=GREEN|RED|PENDING and BUILD=<drone-build-url>
set -o errexit -o nounset -o pipefail
RECIPE="${1:?usage: testme-on-pr.sh <recipe> <pr-index>}"
PRIDX="${2:?usage: testme-on-pr.sh <recipe> <pr-index>}"
TESTENV="${TESTENV:-/srv/cc-ci/.testenv}"
set -a; . "$TESTENV"; set +a
: "${GITEA_USERNAME:?}"; : "${GITEA_PASSWORD:?}"; : "${GITEA_URL:?}"
NS="${GITEA_NAMESPACE:-recipe-maintainers}"
API="https://${GITEA_URL}/api/v1"; AUTH=(-u "${GITEA_USERNAME}:${GITEA_PASSWORD}")
POST="${POST:-1}"; MAX_WAIT="${MAX_WAIT:-480}"; INTERVAL="${INTERVAL:-30}"
SHA="$(curl -s "${AUTH[@]}" "${API}/repos/${NS}/${RECIPE}/pulls/${PRIDX}" \
| python3 -c "import json,sys;print(json.load(sys.stdin)['head']['sha'])" 2>/dev/null || true)"
[ -n "$SHA" ] || { echo "ERROR: could not resolve ${NS}/${RECIPE} PR #${PRIDX} head sha"; exit 1; }
if [ "$POST" = "1" ]; then
curl -s -o /dev/null "${AUTH[@]}" -H "Content-Type: application/json" -X POST \
"${API}/repos/${NS}/${RECIPE}/issues/${PRIDX}/comments" -d '{"body":"!testme"}'
echo "→ posted !testme on ${NS}/${RECIPE}#${PRIDX} (head ${SHA:0:8}) — results will appear in the PR"
sleep 5
fi
# Poll the Drone-reported combined commit status on the PR head (terminal = success/failure/error).
deadline=$(( $(date +%s) + MAX_WAIT ))
url=""
while [ "$(date +%s)" -lt "$deadline" ]; do
read -r state url < <(curl -s "${AUTH[@]}" "${API}/repos/${NS}/${RECIPE}/commits/${SHA}/status" 2>/dev/null \
| python3 -c "import json,sys
try: d=json.load(sys.stdin)
except Exception: d={}
sts=d.get('statuses') or []
print(d.get('state','') or 'none', (sts[0].get('target_url') if sts else '') or '')" 2>/dev/null || echo "none ")
case "$state" in
success) echo "VERDICT=GREEN"; echo "BUILD=${url}"; exit 0;;
failure|error) echo "VERDICT=RED"; echo "BUILD=${url}"; exit 2;;
esac
sleep "$INTERVAL"
done
echo "VERDICT=PENDING"; echo "BUILD=${url:-?}"
echo "(run still in flight — re-run with POST=0 to keep polling without re-triggering)"
exit 3