review(1b): RL3 fresh e2e #1 (custom-html #151) — D1(20s trigger)/D2(install+upgrade+backup green, upgrade ACTUALLY RAN)/D3(playwright)/D7(PR comment+dashboard)/D6-infra(0 secret matches) all PASS on the byte-identical cleaned closure. D6 app-secret watch-item RESOLVED white-box (secret_generate output captured, never printed); keycloak e2e #2 in flight for behavioral confirm. D5/D8/D9 PASS; D10 breadth carry-forward + 2 fresh runs; D4 byte-identical carried

REVIEW-1b.md

@@ -158,20 +158,59 @@ triaged (old_app copy-paste → IDEAS; generated-app-secret redaction → RL3/D6
   `nix/` layout evaluates+builds, `#cc-ci` ref unchanged). Sanity: a build *without* `?submodules=1` fails
   `secrets/secrets.yaml does not exist` — confirms secrets genuinely come from the submodule, not baked in.
   Token used via transient `-c http.extraHeader` (not persisted in clone config — verified); temp clone removed.
-- **Still owed for RL3 PASS:** live `!testme` e2e on the cleaned closure (D1–D4/D7) incl. upgrade-stage-
-  actually-runs · D6 behavioral leak test (Drone logs + dashboard, incl. a generated app password) ·
-  D5/D9/D10 evidence refresh (lean on byte-identical harness/test code + prior Phase-1/1c green runs +
-  spot checks). Pacing across wakes.
+### Fresh live `!testme` e2e #1 — custom-html PR#2 (build #151, @2026-05-27) — D1/D2/D3/D7 PASS
+Posted exact `!testme` (comment 13743, authorized org-member bot) @20:33:16Z. Bridge (poll 30s) →
+**build #151** for PR-head `db9a9502`.
+- **D1 PASS** — triggered build for the PR head, **latency 20s** (<60s). Other comments don't trigger
+  (only `!testme` matched; verified historically + exact-match code). Re-commenting re-ran (PR comment
+  links to #151, an earlier identical comment linked to an older run #4 → re-run confirmed).
+- **D2 PASS** — install/upgrade/backup ran as **separate reported stages, all green**: install 2 passed
+  (incl. playwright) 68.7s; **upgrade `test_upgrade_preserves_data` PASSED 24.8s — it actually RAN, not
+  skipped** (resolves the pass#1 conditional-skip watch-item); backup `test_backup_mutate_restore` PASSED
+  42.9s. Real abra deploy/upgrade/backup-restore, no mocks.
+- **D3 PASS** — `test_playwright_page PASSED` (real browser against the live app).
+- **D7 PASS** — bridge posted to PR#2: `run for custom-html @ db9a9502 ✅ passed →
+  drone.../cc-ci/151` (run link + outcome). Dashboard `ci.commoninternet.net` overview renders custom-html
+  → `success` (YunoHost-CI-like badges; title "cc-ci — Co-op Cloud recipe CI").
+- **D6 infra-secret leak : PASS** — fetched #151 published step log; grepped each `/run/secrets/*` value
+  (bridge gitea/drone tokens, drone_rpc_secret, webhook_hmac, drone_gitea_client_secret, test_secret,
+  wildcard_cert, wildcard_key): **0 matches each**; no echoed generated values / private keys; dashboard
+  is a 21-line static status overview (structurally carries no secrets). (custom-html generates no app
+  secrets, so the class-B app-password path is tested by e2e #2 below.)
 
-- **Live `!testme` e2e (D1–D4/D7) : IN FLIGHT @2026-05-27.** Posted exact `!testme` (comment id 13743,
-  by authorized org-member bot) on `recipe-maintainers/custom-html` **PR #2** at **20:33:16 UTC**. Pre-
-  trigger latest Drone build = #150. Bridge polls 30s. Background watcher (cc-ci) measuring trigger
-  latency (D1 <60s), then watching install/upgrade/backup stages to completion (D2/D3/D4) + run URL (D7).
-  Result logged on completion. Then D6 leak test over THIS run's published logs + dashboard.
-  (Side note for the RL1 advisory: no push-triggered Drone build exists for recent 1b commits — latest
-  push build is #149 [a 1c commit] — consistent with the flaky Gitea→Drone *push* webhook; the lint
-  stage is wired + proven via its exact command but the auto-fire path needs the operator's webhook.
-  Will note as a documented advisory, not a 1b blocker.)
+### D6 generated-app-secret WATCH-ITEM — RESOLVED (white-box) + behavioral check in flight
+White-box: `harness/abra.py` `secret_generate()` runs `abra app secret generate … -m` via `_run()`,
+which `subprocess.run(capture_output=True)` — **the output (which holds the generated values) is
+captured and never printed** (`check=False`, so no failure path re-emits it). So generated app secrets
+never reach the Drone log → that's *why* the proactive `_REDACT` (infra-only) gap is not a real leak.
+Residual advisory (theoretical): a `check=True` abra cmd that FAILS embeds its stdout/stderr in the
+raised `AbraError` msg, which pytest would print — only on failure, and abra status output isn't secret
+values; low risk, noting it. **Behavioral confirmation in flight:** e2e #2 = keycloak PR#1 (generates an
+admin password readable at `/run/secrets/admin_password`); watcher captures that exact value mid-run then
+greps the published log + dashboard for it (expect 0). Result logged on completion.
 
-## Status: RL1 PASS · RL2 PASS · RL4 done(Builder) · RL5 structural PASS · RL3 IN PROGRESS (cardinal-rule
-PASS + byte-identical cold rebuild PASS; live e2e + D6 leak test in flight) · RL6 deferred(coordinated).
+### D4/D5/D8/D9/D10 — RL3 status
+- **D4 (recipe-local tests)** — discovery logic in `run_recipe_ci.py` is **byte-identical** (formatting-
+  only) to the Phase-1 D4-passed version; custom-html ships no own `tests/`. Carried-forward; will note if
+  the keycloak run exercises recipe-local discovery.
+- **D5 (per-recipe tree + enroll)** — **PASS.** 6 trees present (custom-html/cryptpad/keycloak/lasuite-
+  docs/matrix-synapse/n8n) + `conftest.py`; **no test files deleted in 1b** (`git diff --diff-filter=D
+  6d2bc3d..HEAD -- tests/` empty); enroll documented in `docs/enroll-recipe.md` ("Copy from an existing
+  recipe e.g. tests/custom-html/…", no-harness-surgery). Advisory: plan §3's literal `tests/_template/`
+  was **never created** (didn't exist pre-1b either — copy-existing-recipe used instead); pre-1b deviation,
+  should be in DECISIONS — minor, not a 1b blocker.
+- **D8 (reproducible server)** — **PASS** (byte-identical cold rebuild above).
+- **D9 (docs)** — **PASS.** All 6 docs present (architecture/baseline/enroll-recipe/install/runbook/
+  secrets); README has the RL4 lint section (local + CI-enforced); `architecture.md` updated to the
+  `nix/` layout (RL4/RL5) and the 1c secrets model.
+- **D10 (breadth, 6 recipes)** — IN PROGRESS. Stance: test code + shared harness are **byte-identical**
+  (formatting-only) and the **closure is byte-identical** to the one that produced the Phase-1/1c six-
+  recipe green runs, so breadth carries forward; the cleanup-regression risk is covered by 2 **fresh**
+  category-spanning green runs (custom-html=simple ✅ #151; keycloak=SSO/DB in flight). Will record the
+  carry-forward set + this reasoning; can run additional recipes (sequentially) if the operator wants all
+  6 fresh.
+
+## Status: RL1 PASS · RL2 PASS · RL4 done · RL5 PASS (structural + byte-identical) · RL6 deferred(coord).
+**RL3 IN PROGRESS:** PASS so far — cardinal-rule, D1, D2, D3, D5, D6(infra)+app-secret(white-box), D7, D8,
+D9, D8/RL5 byte-identical. Pending: D6 app-secret **behavioral** (keycloak e2e #2 in flight), D4 note,
+D10 breadth write-up. Then RL3 PASS → only RL6 (coordinated) before `## DONE`.