diff --git a/.claude/skills/recipe-report/SKILL.md b/.claude/skills/recipe-report/SKILL.md new file mode 100644 index 0000000..d51ff93 --- /dev/null +++ b/.claude/skills/recipe-report/SKILL.md @@ -0,0 +1,50 @@ +--- +name: recipe-report +description: Generate the weekly public "Recipe Report" after the cc-ci /upgrade-all run. Reviews how the upgrade went and the state of every recipe + open PR, classifies what needs attention vs routine, and publishes a self-contained HTML page to report.ci.commoninternet.net (one page per week + a home index). Read-only — reports, never merges or edits PRs. Runs as its own agent (model configured separately from the upgrader; default opus). Invoke as /recipe-report [YYYY-MM-DD]. +--- + +# recipe-report + +Produce the weekly **"The Recipe Report"** — a public HTML page reviewing the latest `/upgrade-all` run +and the live state of every recipe + open PR. Served at **https://report.ci.commoninternet.net** (one +`week-.html` per run + a home-page index). You are **read-only**: you report, you never merge, +edit, or comment on PRs. The page is **public + unauthenticated** — include only public-safe data +(recipe names, version bumps, PR titles/links, CI verdicts, concise error summaries); **never** secrets, +tokens, internal IPs, or raw logs. + +Helper: `python3 /srv/cc-ci/cc-ci-plan/recipe-report.py {survey|render|publish}` (see its header). + +## Procedure + +1. **Pick the date.** `DATE` = the argument, else today (UTC). The `/upgrade-all` summary for that run is + `/srv/cc-ci/.cc-ci-logs/upgrades/upgrade-all-.md`. + +2. **Survey.** `python3 /srv/cc-ci/cc-ci-plan/recipe-report.py survey > /tmp/survey.json`. This + gives you the raw `/upgrade-all` summary markdown plus, for every recipe mirror, its open PRs and the + `cc-ci/testme` CI verdict on each PR head. Read it all. + +3. **Review & classify.** Judge the run and the recipe/PR state. Sort everything into: + - **Needs attention** — PRs that are GREEN and ready to merge, and errors/failures to investigate + (RED `!testme`, recipe bugs, anything blocking). Lead with these. Short, specific prose per item. + - **Routine** — minor/clean bumps, stale-test PRs (need operator `--with-tests`), up-to-date or + skipped-by-design. Brief. + Also flag cross-cutting issues (e.g. a recipe ending with two open PRs to reconcile). + +4. **Write the spec** `/tmp/report-spec.json` (shape in the helper header): `date`, `subtitle` + ("Week of "), a one-line `headline`, `needs_attention[]`, `routine[]`, and a + **comprehensive `table[]`** with EVERY recipe (`recipe`, `change` "a → b", `status` + GREEN/STALE/FAILED/SKIPPED/UPTODATE, `ci` as a level/result number+info string e.g. `build 154 ✓` + or `RED · restore` with `ci_url`, `pr`/`pr_url`, `notes`). **CI = link + number/info only; no images.** + +5. **Render & publish.** + `python3 .../recipe-report.py render /tmp/report-spec.json /tmp/week-.html` + `python3 .../recipe-report.py publish /tmp/week-.html ` + (publish copies the page to `cc-ci:/var/lib/cc-ci-reports/` and regenerates the index.) + +6. **Verify & stop.** `curl -fsS https://report.ci.commoninternet.net/week-.html` renders and the + index lists it. Print the report URL and `RECIPE REPORT COMPLETE`, then go idle (one-shot — do not loop). + +## Guardrails +- **Read-only.** Never merge/edit/comment on PRs or touch recipes/tests. You only describe + link. +- **Public-safe.** No secrets/tokens/internal hostnames/raw logs on the page. Summarize errors. +- One page per run; always regenerate the index so all weeks are listed (newest first). diff --git a/.claude/skills/upgrade-all/SKILL.md b/.claude/skills/upgrade-all/SKILL.md index 0a32fff..ca2bf70 100644 --- a/.claude/skills/upgrade-all/SKILL.md +++ b/.claude/skills/upgrade-all/SKILL.md @@ -115,6 +115,12 @@ PR list** (the actionable output): ``` End with the report path and a reminder that **nothing was merged**. +## 6. Launch the public Recipe Report +Once the summary is written, kick off the weekly public report (its own agent, separate model — default +opus): `python3 /srv/cc-ci/cc-ci-plan/launch-report.py start`. It runs `/recipe-report`, reviews this +run + the live recipe/PR state, and publishes to https://report.ci.commoninternet.net. Fire-and-forget +— it runs independently; you can then go idle. + ## Safety / coordination (this matters — shared host with the build loops) - **Sequential is the default for a reason.** Recipe deploys are **stateful on the shared Swarm** and parallel deploys can OOM/collide. Between sequential recipes, the per-recipe `recipe-upgrade` tears diff --git a/cc-ci-plan/launch-report.py b/cc-ci-plan/launch-report.py new file mode 100755 index 0000000..def3347 --- /dev/null +++ b/cc-ci-plan/launch-report.py @@ -0,0 +1,98 @@ +#!/usr/bin/env python3 +"""cc-ci recipe-report launcher — one-shot agent that runs /recipe-report after the weekly upgrade. + +The report agent's model is configured SEPARATELY from the upgrader (so e.g. the upgrader runs on +sonnet while the report is written by opus). Defaults: REPORT_MODEL=opus, REPORT_BACKEND=claude. + +Usage: + launch-report.py start [DATE] use-or-create the session; runs /recipe-report [DATE] + launch-report.py fresh [DATE] always start a new session + launch-report.py stop kill the session + launch-report.py status show session state + +Env: REPORT_MODEL (default opus), REPORT_BACKEND (default claude), REPORT_SESSION, REPORT_DIR. +""" +import os, subprocess, sys, time +from pathlib import Path +from datetime import datetime, timezone + +SESSION = os.environ.get("REPORT_SESSION", "cc-ci-report") +WORKDIR = os.environ.get("REPORT_DIR", "/srv/cc-ci") +LOG_DIR = os.environ.get("LOG_DIR", "/srv/cc-ci/.cc-ci-logs") +BACKEND = os.environ.get("REPORT_BACKEND", "claude") +MODEL = os.environ.get("REPORT_MODEL", "opus") +CLAUDE_BIN = os.environ.get("CLAUDE_BIN", "claude") +CLAUDE_FLAGS = os.environ.get("CLAUDE_FLAGS", "--dangerously-skip-permissions") +OPENCODE_BIN = os.environ.get("OPENCODE_BIN", "/home/loops/.local/bin/opencode") +OPENCODE_SERVER = os.environ.get("OPENCODE_SERVER", "http://127.0.0.1:4096") + + +def log(m): print(f"[report {datetime.now(timezone.utc):%H:%M:%S}] {m}", flush=True) +def die(m): log(f"ERROR: {m}"); sys.exit(1) +def _sh(c): return subprocess.run(c, capture_output=True, text=True) +def session_alive(): return _sh(["tmux", "has-session", "-t", SESSION]).returncode == 0 +def kill_session(): subprocess.run(["tmux", "kill-session", "-t", SESSION], capture_output=True) + + +def build_kickoff(date): + arg = f" {date}" if date else "" + return ( + f"*** cc-ci RECIPE-REPORT — one-shot ***\n" + f"You generate the public weekly \"Recipe Report\". Run the /recipe-report skill now:\n" + f" invoke /recipe-report{arg}\n" + f"Full spec: {WORKDIR}/.claude/skills/recipe-report/SKILL.md. Creds in {WORKDIR}/.testenv; " + f"reach the CI server with `ssh cc-ci`.\n" + f"You are READ-ONLY: review the latest /upgrade-all run + every recipe's open PRs + CI verdicts, " + f"classify needs-attention vs routine, and publish one HTML page per run to " + f"report.ci.commoninternet.net (+ regenerate the index). Public page — NO secrets/tokens/raw logs. " + f"Never merge/edit/comment on PRs. When done, print the report URL + 'RECIPE REPORT COMPLETE' and " + f"go idle (do NOT loop)." + ) + + +def start(mode, date): + import shutil + if not shutil.which("tmux"): + die("tmux not found") + Path(LOG_DIR).mkdir(parents=True, exist_ok=True) + if session_alive(): + if mode == "use-or-create": + log(f"{SESSION} already running — leaving it"); return + log(f"{SESSION} exists — killing (fresh)"); kill_session(); time.sleep(1) + + kf = Path(LOG_DIR) / f".kickoff-{SESSION}.txt" + kf.write_text(build_kickoff(date)) + model_flag = f"--model '{MODEL}'" if MODEL else "" + if BACKEND == "claude": + cmd = f"{CLAUDE_BIN} --remote-control '{SESSION}' {model_flag} {CLAUDE_FLAGS} \"$(cat '{kf}')\"" + cwd = WORKDIR + elif BACKEND == "opencode": + cwd = "/srv/cc-ci-orch/cc-ci" + cmd = f"set -a; . /srv/cc-ci/.testenv; set +a; NO_COLOR=1 {OPENCODE_BIN} attach {OPENCODE_SERVER} --dir {cwd}" + else: + die(f"unknown REPORT_BACKEND '{BACKEND}'") + log(f"starting {SESSION} (backend={BACKEND}, model={MODEL}, date={date or 'today'})") + subprocess.run(["tmux", "new-session", "-d", "-s", SESSION, "-c", cwd, cmd]) + subprocess.run(["tmux", "pipe-pane", "-o", "-t", SESSION, f"cat >> '{LOG_DIR}/{SESSION}.log'"]) + if BACKEND == "opencode": + time.sleep(12) + subprocess.run(["tmux", "send-keys", "-t", SESSION, "-l", "--", f"Read {kf} and follow it."]) + time.sleep(0.5); subprocess.run(["tmux", "send-keys", "-t", SESSION, "C-m"]) + log(f"started. attach: tmux attach -t {SESSION}") + + +def main(): + a = sys.argv[1:] + cmd = a[0] if a else "start" + date = a[1] if len(a) > 1 else "" + if cmd == "start": start("use-or-create", date) + elif cmd == "fresh": start("fresh", date) + elif cmd == "stop": kill_session(); log("stopped.") + elif cmd == "status": + log(f"{SESSION}: {'RUNNING' if session_alive() else 'stopped'} (backend={BACKEND} model={MODEL})") + else: + print(__doc__); sys.exit(2) + + +if __name__ == "__main__": + main() diff --git a/cc-ci-plan/plan-recipe-report-skill.md b/cc-ci-plan/plan-recipe-report-skill.md index 75f39a4..6a3cae3 100644 --- a/cc-ci-plan/plan-recipe-report-skill.md +++ b/cc-ci-plan/plan-recipe-report-skill.md @@ -1,6 +1,8 @@ # Plan — `/recipe-report` skill + report.ci.commoninternet.net -**Status:** PROPOSED. **Created:** 2026-06-02. +**Status:** IMPLEMENTED 2026-06-02 — serving live at https://report.ci.commoninternet.net; skill +`recipe-report` + helper `cc-ci-plan/recipe-report.py` + launcher `cc-ci-plan/launch-report.py` +(REPORT_MODEL default opus); upgrade-all launches the report as its closing step. **Created:** 2026-06-02. **Goal:** after the weekly `/upgrade-all`, generate a public HTML **"Recipe Report"** that reviews how the upgrade went and the state of every recipe + open PR — served at **report.ci.commoninternet.net**, one page per week, with a home-page index of all reports. diff --git a/cc-ci-plan/recipe-report.py b/cc-ci-plan/recipe-report.py new file mode 100755 index 0000000..3635421 --- /dev/null +++ b/cc-ci-plan/recipe-report.py @@ -0,0 +1,198 @@ +#!/usr/bin/env python3 +"""recipe-report — data + HTML helper for the weekly "Recipe Report" (/recipe-report skill). + +Three subcommands; the /recipe-report agent runs them around its own review/classification: + survey [DATE] print structured JSON of the run + every recipe's open PRs + CI verdict + (DATE defaults to today UTC; reads the dated /upgrade-all summary if present) + render SPEC.json OUT.html render the agent's report spec -> a self-contained HTML page + publish OUT.html DATE copy the page to cc-ci:/var/lib/cc-ci-reports/week-DATE.html and regen index + +The agent: `survey` -> review the data, write a SPEC (see SPEC SHAPE below) -> `render` -> `publish`. + +SPEC SHAPE (the agent writes this JSON): + {"date":"YYYY-MM-DD","subtitle":"Week of ","headline":"one line", + "needs_attention":[{"title":"...","body":"","links":[{"text":"PR #4","url":"..."}]}], + "routine":[ {same shape} ], + "table":[{"recipe":"x","change":"a → b","status":"GREEN","ci":"level 8 ✓","ci_url":"...", + "pr":"#4","pr_url":"...","notes":"..."}]} +PUBLIC PAGE — include only public-safe data (no secrets/tokens/raw logs). +""" +import html, json, os, subprocess, sys, urllib.request, urllib.parse +from datetime import datetime, timezone + +LOGDIR = "/srv/cc-ci/.cc-ci-logs" +TESTENV = "/srv/cc-ci/.testenv" +INFRA = {"cc-ci", "cc-ci-orchestrator", "cc-ci-secrets"} +HOST_REPORTS = "/var/lib/cc-ci-reports" + + +def _env(): + e = {} + try: + for ln in open(TESTENV): + ln = ln.strip() + if "=" in ln and not ln.startswith("#"): + k, v = ln.split("=", 1) + e[k] = v.strip().strip('"').strip("'") + except FileNotFoundError: + pass + return e + + +def _gitea_get(path, env): + url = f"https://{env['GITEA_URL']}/api/v1{path}" + auth = "%s:%s" % (env["GITEA_USERNAME"], env["GITEA_PASSWORD"]) + import base64 + req = urllib.request.Request(url, headers={"Authorization": "Basic " + base64.b64encode(auth.encode()).decode()}) + try: + with urllib.request.urlopen(req, timeout=20) as r: + return json.load(r) + except Exception: + return None + + +def survey(date): + env = _env() + out = {"date": date, "generated": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")} + # The /upgrade-all dated summary (raw markdown; the agent reads its narrative). + sm = os.path.join(LOGDIR, "upgrades", f"upgrade-all-{date}.md") + out["upgrade_summary_md"] = open(sm).read() if os.path.exists(sm) else None + # Every recipe mirror -> open PRs + the cc-ci/testme verdict on each PR head. + repos = _gitea_get("/orgs/recipe-maintainers/repos?limit=100", env) or [] + recipes = sorted(r["name"] for r in repos if r["name"] not in INFRA and not r["name"].startswith("archived-")) + out["recipes"] = [] + for r in recipes: + prs = _gitea_get(f"/repos/recipe-maintainers/{r}/pulls?state=open&limit=50", env) or [] + rec = {"recipe": r, "open_prs": []} + for p in prs: + sha = p["head"]["sha"] + st = _gitea_get(f"/repos/recipe-maintainers/{r}/commits/{sha}/status", env) or {} + ci = {"state": st.get("state")} + for s in (st.get("statuses") or []): + if s.get("context") == "cc-ci/testme": + ci = {"state": s.get("state"), "url": s.get("target_url"), "desc": s.get("description")} + rec["open_prs"].append({"number": p["number"], "title": p["title"], + "url": p["html_url"], "head": sha[:10], "ci": ci}) + out["recipes"].append(rec) + print(json.dumps(out, indent=2)) + + +# ---------- HTML rendering ---------- +CSS = """ +:root{--fg:#1a1a1a;--mut:#666;--bd:#e3e3e3;--bg:#fafafa;--card:#fff;--accent:#c9760a;--red:#c0392b;--green:#218a4e} +*{box-sizing:border-box}body{font:16px/1.6 system-ui,-apple-system,Segoe UI,sans-serif;color:var(--fg); +background:var(--bg);margin:0;padding:2.5rem 1rem 4rem}.wrap{max-width:56rem;margin:0 auto} +h1{font-size:2.4rem;margin:0;letter-spacing:-.02em}.sub{color:var(--mut);font-size:1.15rem;margin:.2rem 0 .2rem} +.headline{font-size:1.1rem;margin:.6rem 0 2rem;padding:.7rem 1rem;background:#fff7ec;border-left:4px solid var(--accent);border-radius:4px} +h2{font-size:1.45rem;margin:2.4rem 0 .8rem;border-bottom:2px solid var(--bd);padding-bottom:.3rem} +.item{background:var(--card);border:1px solid var(--bd);border-radius:8px;padding:.9rem 1.1rem;margin:.7rem 0} +.item.attn{border-left:4px solid var(--red)}.item .t{font-weight:600}.item .b{color:#333;margin:.25rem 0 .4rem} +.links a{margin-right:.8rem;font-size:.92rem}a{color:#0b62c4;text-decoration:none}a:hover{text-decoration:underline} +table{border-collapse:collapse;width:100%;font-size:.93rem;margin-top:.6rem} +th,td{text-align:left;padding:.5rem .6rem;border-bottom:1px solid var(--bd);vertical-align:top} +th{color:var(--mut);font-weight:600;font-size:.82rem;text-transform:uppercase;letter-spacing:.03em} +.s-GREEN{color:var(--green);font-weight:600}.s-RED,.s-FAILED{color:var(--red);font-weight:600} +.s-STALE{color:var(--accent);font-weight:600}.s-SKIPPED,.s-UPTODATE{color:var(--mut)} +footer{margin-top:3rem;padding-top:1rem;border-top:1px solid var(--bd);color:var(--mut);font-size:.9rem} +.idx li{margin:.4rem 0}.idx .d{color:var(--mut);font-size:.9rem} +""" + + +def _esc(s): + return html.escape(str(s or "")) + + +def _links(links): + if not links: + return "" + a = " · ".join(f'{_esc(l["text"])}' for l in links) + return f'
{a}
' + + +def _items(items, attn=False): + if not items: + return "

None.

" + out = [] + for it in items: + out.append( + f'
' + f'
{_esc(it.get("title"))}
' + f'
{_esc(it.get("body"))}
{_links(it.get("links"))}
') + return "\n".join(out) + + +def _table(rows): + if not rows: + return "" + head = "RecipeChangeStatusCIPRNotes" + trs = [] + for r in rows: + scls = "s-" + str(r.get("status", "")).upper().replace("-", "").replace(" ", "") + ci = _esc(r.get("ci")) + if r.get("ci_url"): + ci = f'{ci}' + pr = _esc(r.get("pr")) + if r.get("pr_url"): + pr = f'{pr}' + trs.append(f"{_esc(r.get('recipe'))}{_esc(r.get('change'))}" + f'{_esc(r.get("status"))}{ci}{pr}' + f"{_esc(r.get('notes'))}") + return f"{head}{''.join(trs)}
" + + +def _page(title, body): + return (f'' + f'' + f"{_esc(title)}
{body}
") + + +def render(spec_path, out_path): + s = json.load(open(spec_path)) + gen = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC") + body = ( + f'

🌻 The Recipe Report

' + f'
{_esc(s.get("subtitle", "Week of " + s["date"]))}
' + f'
{_esc(s.get("headline"))}
' + f'

⚑ Needs attention

{_items(s.get("needs_attention"), attn=True)}' + f'

Routine

{_items(s.get("routine"))}' + f'

All recipes

{_table(s.get("table"))}' + f'') + open(out_path, "w").write(_page("The Recipe Report — " + s["date"], body)) + print("wrote", out_path) + + +def publish(html_path, date): + # Copy the page to the host, then rebuild index.html from the host's week-*.html list. + page = f"week-{date}.html" + data = open(html_path, "rb").read() + subprocess.run(["ssh", "cc-ci", f"cat > {HOST_REPORTS}/{page}"], input=data, check=True) + # Rebuild index: list week-*.html, newest first, with each page's /subtitle hint = date. + listing = subprocess.run(["ssh", "cc-ci", f"ls -1 {HOST_REPORTS}/week-*.html 2>/dev/null"], + capture_output=True, text=True).stdout.split() + dates = sorted({os.path.basename(p)[5:-5] for p in listing}, reverse=True) + lis = "\n".join(f'<li><a href="week-{d}.html">Week of {d}</a> <span class="d">— {d}</span></li>' for d in dates) + idx = _page("The Recipe Report", f'<h1>🌻 The Recipe Report</h1>' + f'<div class="sub">Weekly review of Co-op Cloud recipe upgrades + CI</div>' + f'<ul class="idx">{lis or "<li><em>No reports yet.</em></li>"}</ul>') + subprocess.run(["ssh", "cc-ci", f"cat > {HOST_REPORTS}/index.html"], input=idx.encode(), check=True) + print(f"published https://report.ci.commoninternet.net/{page} (+ index)") + + +def main(): + a = sys.argv[1:] + cmd = a[0] if a else "survey" + today = datetime.now(timezone.utc).strftime("%Y-%m-%d") + if cmd == "survey": + survey(a[1] if len(a) > 1 else today) + elif cmd == "render": + render(a[1], a[2]) + elif cmd == "publish": + publish(a[1], a[2]) + else: + print(__doc__); sys.exit(2) + + +if __name__ == "__main__": + main()