feat(recipe-report): /recipe-report skill + helper + launcher (default opus); wire into upgrade-all

- recipe-report.py: survey (run + per-recipe PRs + CI verdicts) / render (spec->HTML) / publish (copy to cc-ci:/var/lib/cc-ci-reports + regen index). - skill .claude/skills/recipe-report: review the weekly run, classify needs-attention vs routine, publish one public HTML page per week + index at report.ci.commoninternet.net. Read-only. - launch-report.py: one-shot cc-ci-report agent, REPORT_MODEL default opus (separate from the sonnet upgrader), REPORT_BACKEND default claude. - upgrade-all SKILL: closing step launches the report agent. Serving (nix/modules/reports.nix) already deployed + live. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 23:02:22 +00:00
parent 73aa20e8ab
commit c7301a9e39
5 changed files with 355 additions and 1 deletions
--- a/.claude/skills/recipe-report/SKILL.md
+++ b/.claude/skills/recipe-report/SKILL.md
@ -0,0 +1,50 @@
+---
+name: recipe-report
+description: Generate the weekly public "Recipe Report" after the cc-ci /upgrade-all run. Reviews how the upgrade went and the state of every recipe + open PR, classifies what needs attention vs routine, and publishes a self-contained HTML page to report.ci.commoninternet.net (one page per week + a home index). Read-only — reports, never merges or edits PRs. Runs as its own agent (model configured separately from the upgrader; default opus). Invoke as /recipe-report [YYYY-MM-DD].
+---
+
+# recipe-report
+
+Produce the weekly **"The Recipe Report"** — a public HTML page reviewing the latest `/upgrade-all` run
+and the live state of every recipe + open PR. Served at **https://report.ci.commoninternet.net** (one
+`week-<date>.html` per run + a home-page index). You are **read-only**: you report, you never merge,
+edit, or comment on PRs. The page is **public + unauthenticated** — include only public-safe data
+(recipe names, version bumps, PR titles/links, CI verdicts, concise error summaries); **never** secrets,
+tokens, internal IPs, or raw logs.
+
+Helper: `python3 /srv/cc-ci/cc-ci-plan/recipe-report.py {survey|render|publish}` (see its header).
+
+## Procedure
+
+1. **Pick the date.** `DATE` = the argument, else today (UTC). The `/upgrade-all` summary for that run is
+   `/srv/cc-ci/.cc-ci-logs/upgrades/upgrade-all-<DATE>.md`.
+
+2. **Survey.** `python3 /srv/cc-ci/cc-ci-plan/recipe-report.py survey <DATE> > /tmp/survey.json`. This
+   gives you the raw `/upgrade-all` summary markdown plus, for every recipe mirror, its open PRs and the
+   `cc-ci/testme` CI verdict on each PR head. Read it all.
+
+3. **Review & classify.** Judge the run and the recipe/PR state. Sort everything into:
+   - **Needs attention** — PRs that are GREEN and ready to merge, and errors/failures to investigate
+     (RED `!testme`, recipe bugs, anything blocking). Lead with these. Short, specific prose per item.
+   - **Routine** — minor/clean bumps, stale-test PRs (need operator `--with-tests`), up-to-date or
+     skipped-by-design. Brief.
+   Also flag cross-cutting issues (e.g. a recipe ending with two open PRs to reconcile).
+
+4. **Write the spec** `/tmp/report-spec.json` (shape in the helper header): `date`, `subtitle`
+   ("Week of <human date>"), a one-line `headline`, `needs_attention[]`, `routine[]`, and a
+   **comprehensive `table[]`** with EVERY recipe (`recipe`, `change` "a → b", `status`
+   GREEN/STALE/FAILED/SKIPPED/UPTODATE, `ci` as a level/result number+info string e.g. `build 154 ✓`
+   or `RED · restore` with `ci_url`, `pr`/`pr_url`, `notes`). **CI = link + number/info only; no images.**
+
+5. **Render & publish.**
+   `python3 .../recipe-report.py render /tmp/report-spec.json /tmp/week-<DATE>.html`
+   `python3 .../recipe-report.py publish /tmp/week-<DATE>.html <DATE>`
+   (publish copies the page to `cc-ci:/var/lib/cc-ci-reports/` and regenerates the index.)
+
+6. **Verify & stop.** `curl -fsS https://report.ci.commoninternet.net/week-<DATE>.html` renders and the
+   index lists it. Print the report URL and `RECIPE REPORT COMPLETE`, then go idle (one-shot — do not loop).
+
+## Guardrails
+- **Read-only.** Never merge/edit/comment on PRs or touch recipes/tests. You only describe + link.
+- **Public-safe.** No secrets/tokens/internal hostnames/raw logs on the page. Summarize errors.
+- One page per run; always regenerate the index so all weeks are listed (newest first).
--- a/.claude/skills/upgrade-all/SKILL.md
+++ b/.claude/skills/upgrade-all/SKILL.md
@ -115,6 +115,12 @@ PR list** (the actionable output):
 ```
 End with the report path and a reminder that **nothing was merged**.

+## 6. Launch the public Recipe Report
+Once the summary is written, kick off the weekly public report (its own agent, separate model — default
+opus): `python3 /srv/cc-ci/cc-ci-plan/launch-report.py start`. It runs `/recipe-report`, reviews this
+run + the live recipe/PR state, and publishes to https://report.ci.commoninternet.net. Fire-and-forget
+— it runs independently; you can then go idle.
+
 ## Safety / coordination (this matters — shared host with the build loops)
 - **Sequential is the default for a reason.** Recipe deploys are **stateful on the shared Swarm** and
  parallel deploys can OOM/collide. Between sequential recipes, the per-recipe `recipe-upgrade` tears
--- a/cc-ci-plan/launch-report.py
+++ b/cc-ci-plan/launch-report.py
@ -0,0 +1,98 @@
+#!/usr/bin/env python3
+"""cc-ci recipe-report launcher — one-shot agent that runs /recipe-report after the weekly upgrade.
+
+The report agent's model is configured SEPARATELY from the upgrader (so e.g. the upgrader runs on
+sonnet while the report is written by opus). Defaults: REPORT_MODEL=opus, REPORT_BACKEND=claude.
+
+Usage:
+  launch-report.py start [DATE]   use-or-create the session; runs /recipe-report [DATE]
+  launch-report.py fresh [DATE]   always start a new session
+  launch-report.py stop           kill the session
+  launch-report.py status         show session state
+
+Env: REPORT_MODEL (default opus), REPORT_BACKEND (default claude), REPORT_SESSION, REPORT_DIR.
+"""
+import os, subprocess, sys, time
+from pathlib import Path
+from datetime import datetime, timezone
+
+SESSION = os.environ.get("REPORT_SESSION", "cc-ci-report")
+WORKDIR = os.environ.get("REPORT_DIR", "/srv/cc-ci")
+LOG_DIR = os.environ.get("LOG_DIR", "/srv/cc-ci/.cc-ci-logs")
+BACKEND = os.environ.get("REPORT_BACKEND", "claude")
+MODEL   = os.environ.get("REPORT_MODEL", "opus")
+CLAUDE_BIN   = os.environ.get("CLAUDE_BIN", "claude")
+CLAUDE_FLAGS = os.environ.get("CLAUDE_FLAGS", "--dangerously-skip-permissions")
+OPENCODE_BIN    = os.environ.get("OPENCODE_BIN", "/home/loops/.local/bin/opencode")
+OPENCODE_SERVER = os.environ.get("OPENCODE_SERVER", "http://127.0.0.1:4096")
+
+
+def log(m): print(f"[report {datetime.now(timezone.utc):%H:%M:%S}] {m}", flush=True)
+def die(m): log(f"ERROR: {m}"); sys.exit(1)
+def _sh(c): return subprocess.run(c, capture_output=True, text=True)
+def session_alive(): return _sh(["tmux", "has-session", "-t", SESSION]).returncode == 0
+def kill_session(): subprocess.run(["tmux", "kill-session", "-t", SESSION], capture_output=True)
+
+
+def build_kickoff(date):
+    arg = f" {date}" if date else ""
+    return (
+        f"*** cc-ci RECIPE-REPORT — one-shot ***\n"
+        f"You generate the public weekly \"Recipe Report\". Run the /recipe-report skill now:\n"
+        f"  invoke /recipe-report{arg}\n"
+        f"Full spec: {WORKDIR}/.claude/skills/recipe-report/SKILL.md. Creds in {WORKDIR}/.testenv; "
+        f"reach the CI server with `ssh cc-ci`.\n"
+        f"You are READ-ONLY: review the latest /upgrade-all run + every recipe's open PRs + CI verdicts, "
+        f"classify needs-attention vs routine, and publish one HTML page per run to "
+        f"report.ci.commoninternet.net (+ regenerate the index). Public page — NO secrets/tokens/raw logs. "
+        f"Never merge/edit/comment on PRs. When done, print the report URL + 'RECIPE REPORT COMPLETE' and "
+        f"go idle (do NOT loop)."
+    )
+
+
+def start(mode, date):
+    import shutil
+    if not shutil.which("tmux"):
+        die("tmux not found")
+    Path(LOG_DIR).mkdir(parents=True, exist_ok=True)
+    if session_alive():
+        if mode == "use-or-create":
+            log(f"{SESSION} already running — leaving it"); return
+        log(f"{SESSION} exists — killing (fresh)"); kill_session(); time.sleep(1)
+
+    kf = Path(LOG_DIR) / f".kickoff-{SESSION}.txt"
+    kf.write_text(build_kickoff(date))
+    model_flag = f"--model '{MODEL}'" if MODEL else ""
+    if BACKEND == "claude":
+        cmd = f"{CLAUDE_BIN} --remote-control '{SESSION}' {model_flag} {CLAUDE_FLAGS} \"$(cat '{kf}')\""
+        cwd = WORKDIR
+    elif BACKEND == "opencode":
+        cwd = "/srv/cc-ci-orch/cc-ci"
+        cmd = f"set -a; . /srv/cc-ci/.testenv; set +a; NO_COLOR=1 {OPENCODE_BIN} attach {OPENCODE_SERVER} --dir {cwd}"
+    else:
+        die(f"unknown REPORT_BACKEND '{BACKEND}'")
+    log(f"starting {SESSION} (backend={BACKEND}, model={MODEL}, date={date or 'today'})")
+    subprocess.run(["tmux", "new-session", "-d", "-s", SESSION, "-c", cwd, cmd])
+    subprocess.run(["tmux", "pipe-pane", "-o", "-t", SESSION, f"cat >> '{LOG_DIR}/{SESSION}.log'"])
+    if BACKEND == "opencode":
+        time.sleep(12)
+        subprocess.run(["tmux", "send-keys", "-t", SESSION, "-l", "--", f"Read {kf} and follow it."])
+        time.sleep(0.5); subprocess.run(["tmux", "send-keys", "-t", SESSION, "C-m"])
+    log(f"started. attach: tmux attach -t {SESSION}")
+
+
+def main():
+    a = sys.argv[1:]
+    cmd = a[0] if a else "start"
+    date = a[1] if len(a) > 1 else ""
+    if cmd == "start":   start("use-or-create", date)
+    elif cmd == "fresh": start("fresh", date)
+    elif cmd == "stop":  kill_session(); log("stopped.")
+    elif cmd == "status":
+        log(f"{SESSION}: {'RUNNING' if session_alive() else 'stopped'}  (backend={BACKEND} model={MODEL})")
+    else:
+        print(__doc__); sys.exit(2)
+
+
+if __name__ == "__main__":
+    main()
--- a/cc-ci-plan/plan-recipe-report-skill.md
+++ b/cc-ci-plan/plan-recipe-report-skill.md
@ -1,6 +1,8 @@
 # Plan — `/recipe-report` skill + report.ci.commoninternet.net

-**Status:** PROPOSED. **Created:** 2026-06-02.
+**Status:** IMPLEMENTED 2026-06-02 — serving live at https://report.ci.commoninternet.net; skill
+`recipe-report` + helper `cc-ci-plan/recipe-report.py` + launcher `cc-ci-plan/launch-report.py`
+(REPORT_MODEL default opus); upgrade-all launches the report as its closing step. **Created:** 2026-06-02.
 **Goal:** after the weekly `/upgrade-all`, generate a public HTML **"Recipe Report"** that reviews how
 the upgrade went and the state of every recipe + open PR — served at **report.ci.commoninternet.net**,
 one page per week, with a home-page index of all reports.
--- a/cc-ci-plan/recipe-report.py
+++ b/cc-ci-plan/recipe-report.py
@ -0,0 +1,198 @@
+#!/usr/bin/env python3
+"""recipe-report — data + HTML helper for the weekly "Recipe Report" (/recipe-report skill).
+
+Three subcommands; the /recipe-report agent runs them around its own review/classification:
+  survey [DATE]            print structured JSON of the run + every recipe's open PRs + CI verdict
+                           (DATE defaults to today UTC; reads the dated /upgrade-all summary if present)
+  render SPEC.json OUT.html   render the agent's report spec -> a self-contained HTML page
+  publish OUT.html DATE    copy the page to cc-ci:/var/lib/cc-ci-reports/week-DATE.html and regen index
+
+The agent: `survey` -> review the data, write a SPEC (see SPEC SHAPE below) -> `render` -> `publish`.
+
+SPEC SHAPE (the agent writes this JSON):
+  {"date":"YYYY-MM-DD","subtitle":"Week of <human date>","headline":"one line",
+   "needs_attention":[{"title":"...","body":"<plain text>","links":[{"text":"PR #4","url":"..."}]}],
+   "routine":[ {same shape} ],
+   "table":[{"recipe":"x","change":"a → b","status":"GREEN","ci":"level 8 ✓","ci_url":"...",
+             "pr":"#4","pr_url":"...","notes":"..."}]}
+PUBLIC PAGE — include only public-safe data (no secrets/tokens/raw logs).
+"""
+import html, json, os, subprocess, sys, urllib.request, urllib.parse
+from datetime import datetime, timezone
+
+LOGDIR = "/srv/cc-ci/.cc-ci-logs"
+TESTENV = "/srv/cc-ci/.testenv"
+INFRA = {"cc-ci", "cc-ci-orchestrator", "cc-ci-secrets"}
+HOST_REPORTS = "/var/lib/cc-ci-reports"
+
+
+def _env():
+    e = {}
+    try:
+        for ln in open(TESTENV):
+            ln = ln.strip()
+            if "=" in ln and not ln.startswith("#"):
+                k, v = ln.split("=", 1)
+                e[k] = v.strip().strip('"').strip("'")
+    except FileNotFoundError:
+        pass
+    return e
+
+
+def _gitea_get(path, env):
+    url = f"https://{env['GITEA_URL']}/api/v1{path}"
+    auth = "%s:%s" % (env["GITEA_USERNAME"], env["GITEA_PASSWORD"])
+    import base64
+    req = urllib.request.Request(url, headers={"Authorization": "Basic " + base64.b64encode(auth.encode()).decode()})
+    try:
+        with urllib.request.urlopen(req, timeout=20) as r:
+            return json.load(r)
+    except Exception:
+        return None
+
+
+def survey(date):
+    env = _env()
+    out = {"date": date, "generated": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")}
+    # The /upgrade-all dated summary (raw markdown; the agent reads its narrative).
+    sm = os.path.join(LOGDIR, "upgrades", f"upgrade-all-{date}.md")
+    out["upgrade_summary_md"] = open(sm).read() if os.path.exists(sm) else None
+    # Every recipe mirror -> open PRs + the cc-ci/testme verdict on each PR head.
+    repos = _gitea_get("/orgs/recipe-maintainers/repos?limit=100", env) or []
+    recipes = sorted(r["name"] for r in repos if r["name"] not in INFRA and not r["name"].startswith("archived-"))
+    out["recipes"] = []
+    for r in recipes:
+        prs = _gitea_get(f"/repos/recipe-maintainers/{r}/pulls?state=open&limit=50", env) or []
+        rec = {"recipe": r, "open_prs": []}
+        for p in prs:
+            sha = p["head"]["sha"]
+            st = _gitea_get(f"/repos/recipe-maintainers/{r}/commits/{sha}/status", env) or {}
+            ci = {"state": st.get("state")}
+            for s in (st.get("statuses") or []):
+                if s.get("context") == "cc-ci/testme":
+                    ci = {"state": s.get("state"), "url": s.get("target_url"), "desc": s.get("description")}
+            rec["open_prs"].append({"number": p["number"], "title": p["title"],
+                                    "url": p["html_url"], "head": sha[:10], "ci": ci})
+        out["recipes"].append(rec)
+    print(json.dumps(out, indent=2))
+
+
+# ---------- HTML rendering ----------
+CSS = """
+:root{--fg:#1a1a1a;--mut:#666;--bd:#e3e3e3;--bg:#fafafa;--card:#fff;--accent:#c9760a;--red:#c0392b;--green:#218a4e}
+*{box-sizing:border-box}body{font:16px/1.6 system-ui,-apple-system,Segoe UI,sans-serif;color:var(--fg);
+background:var(--bg);margin:0;padding:2.5rem 1rem 4rem}.wrap{max-width:56rem;margin:0 auto}
+h1{font-size:2.4rem;margin:0;letter-spacing:-.02em}.sub{color:var(--mut);font-size:1.15rem;margin:.2rem 0 .2rem}
+.headline{font-size:1.1rem;margin:.6rem 0 2rem;padding:.7rem 1rem;background:#fff7ec;border-left:4px solid var(--accent);border-radius:4px}
+h2{font-size:1.45rem;margin:2.4rem 0 .8rem;border-bottom:2px solid var(--bd);padding-bottom:.3rem}
+.item{background:var(--card);border:1px solid var(--bd);border-radius:8px;padding:.9rem 1.1rem;margin:.7rem 0}
+.item.attn{border-left:4px solid var(--red)}.item .t{font-weight:600}.item .b{color:#333;margin:.25rem 0 .4rem}
+.links a{margin-right:.8rem;font-size:.92rem}a{color:#0b62c4;text-decoration:none}a:hover{text-decoration:underline}
+table{border-collapse:collapse;width:100%;font-size:.93rem;margin-top:.6rem}
+th,td{text-align:left;padding:.5rem .6rem;border-bottom:1px solid var(--bd);vertical-align:top}
+th{color:var(--mut);font-weight:600;font-size:.82rem;text-transform:uppercase;letter-spacing:.03em}
+.s-GREEN{color:var(--green);font-weight:600}.s-RED,.s-FAILED{color:var(--red);font-weight:600}
+.s-STALE{color:var(--accent);font-weight:600}.s-SKIPPED,.s-UPTODATE{color:var(--mut)}
+footer{margin-top:3rem;padding-top:1rem;border-top:1px solid var(--bd);color:var(--mut);font-size:.9rem}
+.idx li{margin:.4rem 0}.idx .d{color:var(--mut);font-size:.9rem}
+"""
+
+
+def _esc(s):
+    return html.escape(str(s or ""))
+
+
+def _links(links):
+    if not links:
+        return ""
+    a = " · ".join(f'<a href="{_esc(l["url"])}">{_esc(l["text"])}</a>' for l in links)
+    return f'<div class="links">{a}</div>'
+
+
+def _items(items, attn=False):
+    if not items:
+        return "<p><em>None.</em></p>"
+    out = []
+    for it in items:
+        out.append(
+            f'<div class="item{" attn" if attn else ""}">'
+            f'<div class="t">{_esc(it.get("title"))}</div>'
+            f'<div class="b">{_esc(it.get("body"))}</div>{_links(it.get("links"))}</div>')
+    return "\n".join(out)
+
+
+def _table(rows):
+    if not rows:
+        return ""
+    head = "<tr><th>Recipe</th><th>Change</th><th>Status</th><th>CI</th><th>PR</th><th>Notes</th></tr>"
+    trs = []
+    for r in rows:
+        scls = "s-" + str(r.get("status", "")).upper().replace("-", "").replace(" ", "")
+        ci = _esc(r.get("ci"))
+        if r.get("ci_url"):
+            ci = f'<a href="{_esc(r["ci_url"])}">{ci}</a>'
+        pr = _esc(r.get("pr"))
+        if r.get("pr_url"):
+            pr = f'<a href="{_esc(r["pr_url"])}">{pr}</a>'
+        trs.append(f"<tr><td>{_esc(r.get('recipe'))}</td><td>{_esc(r.get('change'))}</td>"
+                   f'<td class="{scls}">{_esc(r.get("status"))}</td><td>{ci}</td><td>{pr}</td>'
+                   f"<td>{_esc(r.get('notes'))}</td></tr>")
+    return f"<table>{head}{''.join(trs)}</table>"
+
+
+def _page(title, body):
+    return (f'<!doctype html><html lang="en"><head><meta charset="utf-8">'
+            f'<meta name="viewport" content="width=device-width,initial-scale=1">'
+            f"<title>{_esc(title)}</title><style>{CSS}</style></head><body><div class=wrap>{body}</div></body></html>")
+
+
+def render(spec_path, out_path):
+    s = json.load(open(spec_path))
+    gen = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
+    body = (
+        f'<h1>🌻 The Recipe Report</h1>'
+        f'<div class="sub">{_esc(s.get("subtitle", "Week of " + s["date"]))}</div>'
+        f'<div class="headline">{_esc(s.get("headline"))}</div>'
+        f'<h2>⚑ Needs attention</h2>{_items(s.get("needs_attention"), attn=True)}'
+        f'<h2>Routine</h2>{_items(s.get("routine"))}'
+        f'<h2>All recipes</h2>{_table(s.get("table"))}'
+        f'<footer>Generated {gen} · '
+        f'<a href="https://ci.commoninternet.net/">dashboard</a> · '
+        f'<a href="./">all reports</a></footer>')
+    open(out_path, "w").write(_page("The Recipe Report — " + s["date"], body))
+    print("wrote", out_path)
+
+
+def publish(html_path, date):
+    # Copy the page to the host, then rebuild index.html from the host's week-*.html list.
+    page = f"week-{date}.html"
+    data = open(html_path, "rb").read()
+    subprocess.run(["ssh", "cc-ci", f"cat > {HOST_REPORTS}/{page}"], input=data, check=True)
+    # Rebuild index: list week-*.html, newest first, with each page's <title>/subtitle hint = date.
+    listing = subprocess.run(["ssh", "cc-ci", f"ls -1 {HOST_REPORTS}/week-*.html 2>/dev/null"],
+                             capture_output=True, text=True).stdout.split()
+    dates = sorted({os.path.basename(p)[5:-5] for p in listing}, reverse=True)
+    lis = "\n".join(f'<li><a href="week-{d}.html">Week of {d}</a> <span class="d">— {d}</span></li>' for d in dates)
+    idx = _page("The Recipe Report", f'<h1>🌻 The Recipe Report</h1>'
+                f'<div class="sub">Weekly review of Co-op Cloud recipe upgrades + CI</div>'
+                f'<ul class="idx">{lis or "<li><em>No reports yet.</em></li>"}</ul>')
+    subprocess.run(["ssh", "cc-ci", f"cat > {HOST_REPORTS}/index.html"], input=idx.encode(), check=True)
+    print(f"published https://report.ci.commoninternet.net/{page}  (+ index)")
+
+
+def main():
+    a = sys.argv[1:]
+    cmd = a[0] if a else "survey"
+    today = datetime.now(timezone.utc).strftime("%Y-%m-%d")
+    if cmd == "survey":
+        survey(a[1] if len(a) > 1 else today)
+    elif cmd == "render":
+        render(a[1], a[2])
+    elif cmd == "publish":
+        publish(a[1], a[2])
+    else:
+        print(__doc__); sys.exit(2)
+
+
+if __name__ == "__main__":
+    main()