121 lines
8.9 KiB
Markdown
121 lines
8.9 KiB
Markdown
# Plan — Recipe Report: TESTS + live PR-STATUS column
|
||
|
||
Goal (operator request): in the weekly Recipe Report table,
|
||
1. **rename the current `Status` column → `TESTS`** (it shows the test/CI verdict: GREEN/FAILED/STALE/SKIPPED/UPTODATE — unchanged content, new label),
|
||
2. **add a new `STATUS` column** showing the live state of the recipe's PR — **`open`** (still to review) vs a **✓** for **any state that is not open** (merged OR closed-without-merge both show ✓; no separate "closed unmerged" treatment — operator-decided),
|
||
3. the PR state is fetched **client-side with JavaScript so it updates in realtime** (a visitor sees the current state, even on an archived past report),
|
||
4. to support (3), **make the recipe repo mirrors public** (operator: this is fine).
|
||
|
||
> **Operator decisions (locked):** (a) STATUS is binary — `open` vs ✓ (not-open). (b) The realtime
|
||
> proxy goes on the **cc-ci server via nix**, in the SAME stack/way the report itself is served
|
||
> (`reports.nix`) — NOT a separate service. (c) Exclusion set for the public flip confirmed: make the
|
||
> recipe mirrors public; keep `cc-ci-secrets`, `cc-ci-orchestrator`, `archived-cc-ci-orchestrator`,
|
||
> and `cc-ci` private.
|
||
|
||
## Findings that shape the plan
|
||
- The table is built by `cc-ci-plan/recipe-report.py::_table`. Current columns:
|
||
`Recipe | Change | Status | CVEs | CI | PR | Notes`. The `Status` cell carries a CSS class
|
||
`s-GREEN`/`s-FAILED`/… (colour); the `CI` cell is the build number+link; `PR` is `#n`+link.
|
||
The page is a **self-contained HTML** file (`_page`) served statically by the `ccci-reports`
|
||
nginx stack at `https://report.ci.commoninternet.net`.
|
||
- **Gitea CORS:** git.autonomic.zone has CORS middleware enabled, but it does **not** return
|
||
`access-control-allow-origin` for `report.ci.commoninternet.net` (origin not in the instance's
|
||
`[cors] ALLOW_DOMAIN`). That allow-list is autonomic.zone **instance config we don't control**, so a
|
||
direct browser→Gitea API fetch will be **blocked by CORS**. → use a **same-origin proxy** (below).
|
||
- **Mirror repos:** 25 repos under `recipe-maintainers`, **all private**. This set includes repos that
|
||
must **NOT** be made public:
|
||
- `cc-ci-secrets` (secrets!), `cc-ci-orchestrator` + `archived-cc-ci-orchestrator` (operational
|
||
plans/journal), and arguably `cc-ci` (the harness/infra). **Exclude these.**
|
||
- The **recipe mirrors** that SHOULD go public (they mirror public coopcloud recipes): bluesky-pds,
|
||
cryptpad, custom-html, custom-html-tiny, discourse, ghost, immich, keycloak, lasuite-docs,
|
||
lasuite-drive, lasuite-meet, mailu, matrix-synapse, mattermost-lts, mumble, n8n, plausible,
|
||
uptime-kuma. (Decide on the test fixtures `custom-html-bkp-bad`/`custom-html-rst-bad` and
|
||
`hedgedoc` — fine to include or skip; not in the 18-recipe fleet.)
|
||
|
||
## Realtime-status architecture (chosen: same-origin proxy)
|
||
Browser JS calls a **same-origin** endpoint on the reports host; nginx proxies it to the Gitea API.
|
||
No cross-origin request → no dependency on the co-op's CORS allow-list. Public mirrors mean the proxy
|
||
needs no token (and the PRs are publicly viewable, which is the point).
|
||
|
||
- Add an nginx `location` to the `ccci-reports` stack (cc-ci repo, `reports.nix` / its nginx conf):
|
||
```
|
||
location ~ ^/pr/([a-z0-9._-]+)/([0-9]+)$ {
|
||
resolver 127.0.0.11 ipv6=off; # docker DNS; or a public resolver
|
||
proxy_ssl_server_name on;
|
||
proxy_set_header Host git.autonomic.zone;
|
||
proxy_pass https://git.autonomic.zone/api/v1/repos/recipe-maintainers/$1/pulls/$2;
|
||
proxy_intercept_errors off;
|
||
add_header Cache-Control "no-store";
|
||
}
|
||
```
|
||
So `GET https://report.ci.commoninternet.net/pr/cryptpad/5` → the PR JSON (`{state, merged, …}`).
|
||
(Restrict the regex to recipe-maintainers + a recipe-name charset; only proxies public PR reads.)
|
||
- **Alternative (if the operator prefers no proxy):** ask the autonomic.zone Gitea admins to add
|
||
`report.ci.commoninternet.net` (or `*`) to `[cors] ALLOW_DOMAIN`; then the JS can hit the Gitea API
|
||
directly. Same JS, just a different base URL. Documented as a fallback; the proxy is the default
|
||
because it has no external dependency.
|
||
|
||
## Work items
|
||
|
||
### A. Make the recipe mirrors public (scoped + safe)
|
||
1. **Secret-scan** each recipe mirror before flipping: `git log -p`/`git grep` for tokens, real `.env`
|
||
(not `.env.sample`), private keys; confirm `.drone.yml` uses secret *refs* not plaintext. (Low risk
|
||
— they mirror public upstream — but verify; the flip is hard to walk back once indexed/cloned.)
|
||
2. Confirm **no secrets in bot-authored PR comments / `!testme` output** posted on these repos (the PRs
|
||
become publicly readable). Reports are already public-safe; spot-check a couple of PR threads.
|
||
3. Flip visibility via the Gitea API (per repo): `PATCH /api/v1/repos/recipe-maintainers/<r>` with
|
||
`{"private": false}`, for the recipe-mirror set ONLY (never the excluded repos above). Verify the
|
||
`recipe-maintainers` org itself is visible enough that public repos are reachable.
|
||
4. Verify an **unauthenticated** read now works: `curl https://git.autonomic.zone/api/v1/repos/recipe-maintainers/cryptpad/pulls/5` → 200 with `state`/`merged`.
|
||
|
||
### B. Reports nginx proxy — in `reports.nix`, via nix (cc-ci repo, /ci-dev-workflow)
|
||
The proxy lives in the SAME place and is shipped the SAME way as the report serving itself: in the
|
||
cc-ci repo's `reports.nix` (the `ccci-reports` nginx stack), deployed via the nix rebuild — NOT a
|
||
separate service or a hand-edited container.
|
||
1. Locate how `reports.nix` defines the nginx config for `ccci-reports` (the static `/var/lib/cc-ci-reports`
|
||
server). Add the `/pr/<recipe>/<n>` `location` (above) into that same nginx config, so it's built
|
||
and deployed by the same mechanism. Resolver/SNI as needed for the `git.autonomic.zone` https upstream.
|
||
2. Ship via `/ci-dev-workflow` (branch cc-ci → adversary verify → merge → `nixos-rebuild` deploy).
|
||
Verify `curl https://report.ci.commoninternet.net/pr/cryptpad/5` returns the PR JSON same-origin.
|
||
3. (Optional hardening) a small `proxy_cache` (e.g. 30–60 s) so many visitors don't hammer Gitea.
|
||
|
||
### C. Report generator (`cc-ci-plan/recipe-report.py` — orchestrator repo)
|
||
1. `_table` header: rename `Status` → `TESTS`; add a `STATUS` column. Proposed order:
|
||
`Recipe | Change | TESTS | CVEs | CI | PR | STATUS | Notes`.
|
||
2. TESTS cell: unchanged (keep the `s-<STATUS>` colour class + the GREEN/FAILED/… text).
|
||
3. STATUS cell: emit a JS hook carrying the repo + PR number from the existing row fields
|
||
(`recipe`, `pr` like `#5`):
|
||
`<td class="pr-status" data-repo="{recipe}" data-pr="{n}">…</td>` — blank cell when the row has no
|
||
PR (e.g. merged-upstream custom-html, mumble). **No spec-shape change** — derives from existing
|
||
`recipe` + `pr`.
|
||
4. Add a small `<script>` (in `_page`, so every report + archive page gets it) that, on
|
||
`DOMContentLoaded` and then on a ~30 s `setInterval` (true realtime while the tab is open), for each
|
||
`.pr-status[data-pr]`:
|
||
- `fetch('/pr/' + repo + '/' + pr)` → JSON;
|
||
- render **binary**: `state === 'open'` → an **`open`** badge; **any other state** (closed/merged) →
|
||
a green **✓** (title shows merged vs closed for hover detail, but the glyph is ✓ either way);
|
||
network/parse error → a muted "?" (never break the page).
|
||
5. Add CSS for the two states (`.pr-open` badge, `.pr-done` ✓) + reuse the muted style.
|
||
6. Keep it dependency-free (vanilla fetch), CSP-safe (inline script in the self-contained page), and
|
||
resilient (try/catch per cell; the page renders fully even if the proxy is down).
|
||
|
||
### D. Verify + ship
|
||
1. Render a report locally and publish to a **preview path** (`/badge-preview`-style, not the live
|
||
index) to eyeball: TESTS column reads right, STATUS column populates live (open vs ✓) from real PRs.
|
||
2. Cross-check a known-merged PR shows ✓ and a known-open PR shows "open"; flip one to confirm realtime
|
||
(re-fetch updates without regenerating the page).
|
||
3. Ship the generator change (orchestrator repo: commit + push). Ship the nginx proxy via
|
||
`/ci-dev-workflow` (cc-ci PR → adversary verify → merge → deploy). Regenerate the current week's
|
||
report so the live page gains the new columns.
|
||
4. Update `recipe-report/SKILL.md` + the `_table`/page docstring to describe the new columns and the
|
||
`/pr/<recipe>/<n>` dependency.
|
||
|
||
## Risks / notes
|
||
- **Public exposure is one-way in practice** (mirrors get cloned/indexed). The secret-scan (A.1/A.2)
|
||
is the hard gate before any flip — do not flip a repo that doesn't pass. Excluded set locked:
|
||
`cc-ci-secrets`, `cc-ci-orchestrator`, `archived-cc-ci-orchestrator`, `cc-ci` stay private.
|
||
- **Load:** ~20 unauth PR reads per page view. With proxy caching (B.3) this is negligible.
|
||
- **Order of ops:** the STATUS column is useless until the proxy is live AND the repos are public, but
|
||
it degrades gracefully (muted "?") if either is missing — so the generator change can ship first and
|
||
light up once B + A land. Recommended order: A (public) → B (proxy) → C (generator) → D (verify).
|