From 9bfd6f2ad34b9ad65f818af5a6669207d7017a1e Mon Sep 17 00:00:00 2001 From: autonomic-bot Date: Wed, 27 May 2026 21:42:26 +0100 Subject: [PATCH] =?UTF-8?q?review(1b):=20RL3=20fresh=20e2e=20#1=20(custom-?= =?UTF-8?q?html=20#151)=20=E2=80=94=20D1(20s=20trigger)/D2(install+upgrade?= =?UTF-8?q?+backup=20green,=20upgrade=20ACTUALLY=20RAN)/D3(playwright)/D7(?= =?UTF-8?q?PR=20comment+dashboard)/D6-infra(0=20secret=20matches)=20all=20?= =?UTF-8?q?PASS=20on=20the=20byte-identical=20cleaned=20closure.=20D6=20ap?= =?UTF-8?q?p-secret=20watch-item=20RESOLVED=20white-box=20(secret=5Fgenera?= =?UTF-8?q?te=20output=20captured,=20never=20printed);=20keycloak=20e2e=20?= =?UTF-8?q?#2=20in=20flight=20for=20behavioral=20confirm.=20D5/D8/D9=20PAS?= =?UTF-8?q?S;=20D10=20breadth=20carry-forward=20+=202=20fresh=20runs;=20D4?= =?UTF-8?q?=20byte-identical=20carried?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- REVIEW-1b.md | 69 ++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 54 insertions(+), 15 deletions(-) diff --git a/REVIEW-1b.md b/REVIEW-1b.md index 3ca9308..2574fe8 100644 --- a/REVIEW-1b.md +++ b/REVIEW-1b.md @@ -158,20 +158,59 @@ triaged (old_app copy-paste → IDEAS; generated-app-secret redaction → RL3/D6 `nix/` layout evaluates+builds, `#cc-ci` ref unchanged). Sanity: a build *without* `?submodules=1` fails `secrets/secrets.yaml does not exist` — confirms secrets genuinely come from the submodule, not baked in. Token used via transient `-c http.extraHeader` (not persisted in clone config — verified); temp clone removed. -- **Still owed for RL3 PASS:** live `!testme` e2e on the cleaned closure (D1–D4/D7) incl. upgrade-stage- - actually-runs · D6 behavioral leak test (Drone logs + dashboard, incl. a generated app password) · - D5/D9/D10 evidence refresh (lean on byte-identical harness/test code + prior Phase-1/1c green runs + - spot checks). Pacing across wakes. +### Fresh live `!testme` e2e #1 — custom-html PR#2 (build #151, @2026-05-27) — D1/D2/D3/D7 PASS +Posted exact `!testme` (comment 13743, authorized org-member bot) @20:33:16Z. Bridge (poll 30s) → +**build #151** for PR-head `db9a9502`. +- **D1 PASS** — triggered build for the PR head, **latency 20s** (<60s). Other comments don't trigger + (only `!testme` matched; verified historically + exact-match code). Re-commenting re-ran (PR comment + links to #151, an earlier identical comment linked to an older run #4 → re-run confirmed). +- **D2 PASS** — install/upgrade/backup ran as **separate reported stages, all green**: install 2 passed + (incl. playwright) 68.7s; **upgrade `test_upgrade_preserves_data` PASSED 24.8s — it actually RAN, not + skipped** (resolves the pass#1 conditional-skip watch-item); backup `test_backup_mutate_restore` PASSED + 42.9s. Real abra deploy/upgrade/backup-restore, no mocks. +- **D3 PASS** — `test_playwright_page PASSED` (real browser against the live app). +- **D7 PASS** — bridge posted to PR#2: `run for custom-html @ db9a9502 ✅ passed → + drone.../cc-ci/151` (run link + outcome). Dashboard `ci.commoninternet.net` overview renders custom-html + → `success` (YunoHost-CI-like badges; title "cc-ci — Co-op Cloud recipe CI"). +- **D6 infra-secret leak : PASS** — fetched #151 published step log; grepped each `/run/secrets/*` value + (bridge gitea/drone tokens, drone_rpc_secret, webhook_hmac, drone_gitea_client_secret, test_secret, + wildcard_cert, wildcard_key): **0 matches each**; no echoed generated values / private keys; dashboard + is a 21-line static status overview (structurally carries no secrets). (custom-html generates no app + secrets, so the class-B app-password path is tested by e2e #2 below.) -- **Live `!testme` e2e (D1–D4/D7) : IN FLIGHT @2026-05-27.** Posted exact `!testme` (comment id 13743, - by authorized org-member bot) on `recipe-maintainers/custom-html` **PR #2** at **20:33:16 UTC**. Pre- - trigger latest Drone build = #150. Bridge polls 30s. Background watcher (cc-ci) measuring trigger - latency (D1 <60s), then watching install/upgrade/backup stages to completion (D2/D3/D4) + run URL (D7). - Result logged on completion. Then D6 leak test over THIS run's published logs + dashboard. - (Side note for the RL1 advisory: no push-triggered Drone build exists for recent 1b commits — latest - push build is #149 [a 1c commit] — consistent with the flaky Gitea→Drone *push* webhook; the lint - stage is wired + proven via its exact command but the auto-fire path needs the operator's webhook. - Will note as a documented advisory, not a 1b blocker.) +### D6 generated-app-secret WATCH-ITEM — RESOLVED (white-box) + behavioral check in flight +White-box: `harness/abra.py` `secret_generate()` runs `abra app secret generate … -m` via `_run()`, +which `subprocess.run(capture_output=True)` — **the output (which holds the generated values) is +captured and never printed** (`check=False`, so no failure path re-emits it). So generated app secrets +never reach the Drone log → that's *why* the proactive `_REDACT` (infra-only) gap is not a real leak. +Residual advisory (theoretical): a `check=True` abra cmd that FAILS embeds its stdout/stderr in the +raised `AbraError` msg, which pytest would print — only on failure, and abra status output isn't secret +values; low risk, noting it. **Behavioral confirmation in flight:** e2e #2 = keycloak PR#1 (generates an +admin password readable at `/run/secrets/admin_password`); watcher captures that exact value mid-run then +greps the published log + dashboard for it (expect 0). Result logged on completion. -## Status: RL1 PASS · RL2 PASS · RL4 done(Builder) · RL5 structural PASS · RL3 IN PROGRESS (cardinal-rule -PASS + byte-identical cold rebuild PASS; live e2e + D6 leak test in flight) · RL6 deferred(coordinated). +### D4/D5/D8/D9/D10 — RL3 status +- **D4 (recipe-local tests)** — discovery logic in `run_recipe_ci.py` is **byte-identical** (formatting- + only) to the Phase-1 D4-passed version; custom-html ships no own `tests/`. Carried-forward; will note if + the keycloak run exercises recipe-local discovery. +- **D5 (per-recipe tree + enroll)** — **PASS.** 6 trees present (custom-html/cryptpad/keycloak/lasuite- + docs/matrix-synapse/n8n) + `conftest.py`; **no test files deleted in 1b** (`git diff --diff-filter=D + 6d2bc3d..HEAD -- tests/` empty); enroll documented in `docs/enroll-recipe.md` ("Copy from an existing + recipe e.g. tests/custom-html/…", no-harness-surgery). Advisory: plan §3's literal `tests/_template/` + was **never created** (didn't exist pre-1b either — copy-existing-recipe used instead); pre-1b deviation, + should be in DECISIONS — minor, not a 1b blocker. +- **D8 (reproducible server)** — **PASS** (byte-identical cold rebuild above). +- **D9 (docs)** — **PASS.** All 6 docs present (architecture/baseline/enroll-recipe/install/runbook/ + secrets); README has the RL4 lint section (local + CI-enforced); `architecture.md` updated to the + `nix/` layout (RL4/RL5) and the 1c secrets model. +- **D10 (breadth, 6 recipes)** — IN PROGRESS. Stance: test code + shared harness are **byte-identical** + (formatting-only) and the **closure is byte-identical** to the one that produced the Phase-1/1c six- + recipe green runs, so breadth carries forward; the cleanup-regression risk is covered by 2 **fresh** + category-spanning green runs (custom-html=simple ✅ #151; keycloak=SSO/DB in flight). Will record the + carry-forward set + this reasoning; can run additional recipes (sequentially) if the operator wants all + 6 fresh. + +## Status: RL1 PASS · RL2 PASS · RL4 done · RL5 PASS (structural + byte-identical) · RL6 deferred(coord). +**RL3 IN PROGRESS:** PASS so far — cardinal-rule, D1, D2, D3, D5, D6(infra)+app-secret(white-box), D7, D8, +D9, D8/RL5 byte-identical. Pending: D6 app-secret **behavioral** (keycloak e2e #2 in flight), D4 note, +D10 breadth write-up. Then RL3 PASS → only RL6 (coordinated) before `## DONE`.