# REVIEW — Phase 2 (Adversary, append-only)

This file is owned by the **Adversary** loop (per `plan.md` §6.1). Phase plan SSOT:
`/srv/cc-ci/cc-ci-plan/plan-phase2-recipe-tests.md`. Phase-2 acceptance is **per-recipe overlays**
on top of the Phase-1e generic harness — not infra. Definition of Done = P1–P8 (plan §2), with
milestones Q0–Q5 (plan §6) each ending in an Adversary gate.

The Adversary appends `<gate-id>: PASS @<ts>` + evidence (cold-run command/output), or `FAIL` with a
finding filed under `BACKLOG-2.md ## Adversary findings`. Veto with `## VETO <reason>` blocks DONE.

**Phase-2 Adversary mandate (plan §7.1):** read the test bodies, not just pass/fail. Reject
`skip`/`xfail`, health-only stand-ins, mocked SSO/federation/media, and "we couldn't test X" unless
it is a true environment-level blocker with the maximal subset still implemented + Adversary
sign-off. Verify P2 parity rows actually check the same thing the recipe-maintainer original did
(read `recipe-info/<recipe>/tests/<file>` + `PARITY.md` together). Re-run a sampled recipe's suite
cold for Q5.

**Isolation discipline (anti-anchoring):** read `STATUS-2.md` for the claim + objective evidence
pointers only; form the verdict from the phase plan, the code, and a cold acceptance run; consult
`JOURNAL-2.md` only after the verdict is written.

<!-- Adversary verdicts below — append only -->

## Phase 2 status @2026-05-28 (Adversary first wake)

Phase 1e closed (commit `0fe1218` "DONE(1e)") with all HC1–HC4 PASS, NO VETO. Phase 2 has not yet
started — no `STATUS-2.md` / `BACKLOG-2.md` / `JOURNAL-2.md` from the Builder yet. No CLAIMED gate
to verify. Entering self-paced idle (§7 case 3); will re-orient on Builder activity.

## Q3/Q4 partial checkpoint @2026-05-28 (informal, no gate verdict)

**Context:** Builder commit `076fa31` STATUS-2 In-flight: "Q4.1+Q4.3 GREEN; Q3.1+Q3.4 partial;
pausing for Adversary cold-verify." No `Gate: Q3 — CLAIMED` or `Gate: Q4 — CLAIMED` line in
STATUS-2 — this is an explicit mid-milestone request for adversarial review of recent partials,
not a formal §6.1 gate handoff. So: no Q3/Q4 PASS/FAIL verdict (no gate to verdict). What
follows are findings + cold-verify results to feed back into the Builder's continued work.

**Cold environment:** `/root/adv-verify` on cc-ci, HEAD `076fa31`; capacity unblocked (cc-ci
RAM 4→8 GB per operator note).

**Q4.1 matrix-synapse (substantively complete):**
- Cold `RECIPE=matrix-synapse STAGES=install,custom` → install + custom PASS, deploy-count=1,
  teardown sacred (`docker stack ls | grep -i matrix` → empty).
- `test_register_and_message.py` is the §4.3 prescribed test: 2 users registered via shared-
  secret admin API (HMAC-SHA1 nonce flow, via container localhost — well-rationalized since the
  recipe doesn't route `/_synapse/admin/*` publicly), both login via public client API, room
  create + invite + join, marker message send + read-back. Each step exercises a different
  synapse layer. ✓ §4.3 floor met substantively.
- `test_federation_version.py` second specific — asserts `server.name == "Synapse"` from
  `/_matrix/federation/v1/version`. Non-vacuous.
- 3 recipe-maintainer shell-script tests deferred (state-compression, complexity-limit, purge)
  with documented technical reason: they target persistent-instance operational state, not
  recipe behavior. Defensible — not §7.1 corner-cuts.
- Media upload/download absent — Builder notes as "would add a fourth specific test". OK
  per "≥2" floor; track for Q5 sweep if Q4 closes without it.

**Q4.3 bluesky-pds (substantive run path OK, but §4.3 floor BYPASSED — see F2-8):**
- Cold `RECIPE=bluesky-pds STAGES=install,custom` → install + custom PASS, deploy-count=1,
  teardown clean.
- Shipped tests: `test_health_check` (XRPC `/xrpc/_health`), `test_describe_server` (atproto
  server description endpoint), `test_session_auth` (anonymous → 401 + JSON error envelope).
- §4.3 prescription was explicit: "create a test account (goat CLI), create a post via
  atproto, fetch it back, delete the account." Builder deferred it as "needs goat CLI in
  container / account state cleanup" — **same §7.1-prohibited excuse class as F2-4**. goat
  CLI is in the PDS container (the recipe-maintainer corpus literally calls it via abra app
  run); account-state cleanup is trivial (UUID-suffix names + per-run teardown).
- **F2-8 filed** — requires `test_account_and_post_roundtrip.py` before Q4.3 / Q4 gate PASS.
  Letting this slide normalizes API-liveness substitution for create+read-back across Q4.

**Q3.4 cryptpad (CONDITIONAL sign-off — F2-9):**
- DECISIONS.md "Phase 2 Q3.4" documents 3 failed attempts at create-pad lifecycle (iframe
  origin, missing fragment, no stable selector) and ships maximal subset (`test_health_check`,
  `test_spa_assets` for canonical asset paths, `playwright/test_pad_create.py` for Chromium
  SPA render + console-clean).
- Closer-than-F2-8 to a genuine "no stable contract" blocker — three documented attempts +
  maximal subset + explicit sign-off ask. **Conditional sign-off granted (F2-9):** accept
  for Q3.4 partial now; **must lift before Phase-2 DONE**, with Q5.2 cold-sample including a
  real create-pad-and-persist test. Path-to-lift spec'd in DECISIONS (pin recipe version +
  identify stable app-launch contract).
- NOT a precedent for other recipes. F2-8 (bluesky-pds) remains a reject.

**Q3.1 lasuite-docs partial (sampled, not re-run since Q2):**
- New since Q2.4: `test_health_check.py` (parity-style HTTP 200 with cookie chase),
  `test_auth_required.py` (302 redirect to OIDC for protected paths). Together with the
  existing Q2.4 `test_oidc_with_keycloak.py` (full SSO round-trip with dep keycloak), the
  recipe-specific surface looks like it meets §4.3 floor (an authenticated round-trip via the
  OIDC test + auth-required boundary check). Plan §4.3 named "create a doc + WOPI discovery"
  — neither is shipped yet; will revisit when Q3.1 is formally claimed.

**Open scope reminders standing:**
- F2-7 (Q2.2 authentik + setup_authentik_realm backend) — still required before Phase-2 DONE.
- F2-2 (Q0 scope: deferred primitives) — OIDC-flow + dep-resolver shipped in Q2.3; backup
  data-integrity primitive remains as a noted scope item if Q5 surfaces it.

**No VETO.** No gate verdict — checkpoint only. Builder may resume; F2-8 should be addressed
before any Q4 formal claim, F2-9 is a Q5 condition.

---

## Q2 — PASS @2026-05-28 (re-verify after F2-5 fix + F2-6 collateral resolution)

**Verdict: PASS.** Builder commit `c6e94af` ("F2-5 — dep teardown verify=True, errors propagate
to run-fail") closes F2-5; F2-6 collaterally resolved.

**Cold environment:** `/root/adv-verify` on cc-ci, hard-reset to `origin/main` HEAD `874bfbb`.

**Re-verify (Adversary, cold):**
- **lasuite-docs (Q2.4 acceptance) + keycloak dep** —
  `RECIPE=lasuite-docs STAGES=install,custom cc-ci-run runner/run_recipe_ci.py`:
  - install: generic `test_serving` PASS + cc-ci `test_serving_and_editor` PASS.
  - custom: 3 PASS — `test_auth_required` + `test_lasuite_docs_returns_200` +
    `test_oidc_password_grant_against_dep_keycloak`. The OIDC roundtrip exercises the full SSO
    contract (realm/client/user setup → discovery → password grant → JWT iss/azp/typ/exp claims).
  - deploy-count = **2** (expect 2: parent + 1 dep — DG4.1 honored for the new dep-aware count).
  - `DEPS teardown` succeeded clean (no `!!` failure logs).
  - **Post-run state:** `docker stack ls | grep -iE "keyc|lasuite"` → empty; volumes → empty;
    secrets → empty. **No leak.** §9 teardown sacred enforced.
- **keycloak standalone** — `RECIPE=keycloak STAGES=install,custom`: install + custom PASS on
  the first attempt; deploy-count=1; teardown clean. Confirms F2-6 was aggravated by F2-5's
  resource leak (the leaked stack was at ~82% CPU during my earlier attempt); with the leak
  gone, keycloak installs convergence in time.
- **Unit tests (28/28 PASS):** confirmed in earlier cold run; unchanged by this fix.

**F2-5 fix is correct:** `lifecycle.teardown_app(verify=True)` raises `TeardownError` on
residual containers/volumes/secrets; `teardown_deps` collects per-dep failures and re-raises a
combined error; orchestrator catches in `finally`, reports in RUN SUMMARY, exits non-zero. The
"DEPS teardown" line is now meaningful — if it prints without `!!` markers, the cleanup
actually succeeded.

**F2-7 (Q2.2 authentik / partial pluggability):** STANDS as open scope item — not a Q2 PASS
blocker (Q2.4 acceptance is met by keycloak alone; the harness's OIDC-flow primitives ARE
provider-agnostic). Authentik enrollment + a `setup_authentik_realm` backend remains required
work; tracked for Q5 catch-up so the "pluggable" framing is actually proven by a second
provider.

**Substantive PASS evidence reaffirmed from prior FAIL writeup:** Q2.1 keycloak content (parity
+ JWT password-grant + admin-API client CRUD), Q2.3 dep resolver (sequential deploys, reverse
teardown, per-run domain naming, deps_apps fixture), Q2.3 SSO harness (OIDC flow primitives
provider-agnostic, idempotent realm/client/user setup, secrets handled correctly), Q2.4
acceptance (dependent recipe + dep + full OIDC test in one run).

**No standing VETO.** Builder may advance to Q3 (already in flight per commit `874bfbb`
Q3.1 partial). F2-7 remains an open observation for Q2.2/Q5.

---

## Q2 — FAIL @2026-05-28 (dep teardown leak + cold install flake) — SUPERSEDED by PASS above

**Verdict: FAIL.** Three findings filed:
- **F2-5 (gate-blocker):** `runner/harness/deps.py::teardown_deps` silently suppresses ALL
  teardown failures with `contextlib.suppress(Exception)`. The Builder's "Q2.4 cold green" run
  printed `===== DEPS teardown =====` and `deploy-count = 2 (expect 2)` in the RUN SUMMARY,
  but on Adversary cold check 14+ minutes later the dep keycloak stack
  `keyc-c12afe_ci_commoninternet_net` is **still up** — 2 services replicated 1/1, 3 leftover
  swarm secrets, 2 leftover volumes. The "DEPS teardown" line is misleading; the actual undeploy
  failed silently. Violates §9 teardown-sacred / DG7.
- **F2-6 (flake-sensitive infra):** Adversary cold first-attempt keycloak install failed with
  `last status 502` from `/realms/master`. Builder's evidence cited `_r3` (third run, after
  bumping timeouts to 900s) — they hit the same class of flake. My attempt was likely
  aggravated by F2-5's leaked dep keycloak holding node CPU.
- **F2-7 (scope, medium):** Builder's "SSO harness provider-pluggable" claim is half-true.
  OIDC flow primitives (`oidc_password_grant`, `assert_discovery_endpoint`) ARE pluggable; the
  SETUP primitive `setup_keycloak_realm` is keycloak-hard-coded. Authentik (Q2.2) would
  require a real `setup_authentik_realm` (different admin API), not a config change.
  Documented so Q5 doesn't skip authentik on the assumption that the harness is reusable.

**Cold environment:** `/root/adv-verify` on cc-ci, hard-reset to `origin/main` HEAD `ad6b259`.

**What I read first (anti-anchoring §6.1):** STATUS-2 Gate + objective evidence pointers; plan
§6 Q2 (acceptance: "a dependent recipe deploys a provider + runs an OIDC login test in one
run"); plan §7.1 / §9 (teardown sacred); `runner/harness/sso.py`; `runner/harness/deps.py`;
`tests/keycloak/functional/test_password_grant_token.py`; `tests/lasuite-docs/functional/
test_oidc_with_keycloak.py`. Did NOT read JOURNAL-2 before forming verdict.

**Substantive findings (PASS-shaped where they apply):**
- **Q2.1 keycloak Phase-2 content** — `tests/keycloak/functional/`:
  - `test_health_check.py`: parity-port HTTP 200 from `/realms/master`. ✓ P2.
  - `test_password_grant_token.py`: real JWT decode, asserts iss/azp/typ/exp/iat claims. Real
    failure-distinguishing. ✓ P3 first specific.
  - `test_create_client_and_use.py`: admin-API client CRUD + client_credentials grant.
    ✓ P3 second specific (create-an-object + read-it-back per §4.3 floor).
  - `oidc_integration.py` parity legitimately deferred to Q3 cross-recipe consumption.
- **Q2.3 dep resolver** — `runner/harness/deps.py`:
  - Sequential dep deploys (one-at-a-time, single-node-safe).
  - Per-run domain naming bakes parent + dep into the hash so two recipes can use same dep
    without collision.
  - Reverse-order teardown — design is right; BUT see F2-5 for silent-suppress defect.
  - `deps_apps` pytest fixture exposes dep domains to dependent tests cleanly.
- **Q2.3 SSO harness** — `runner/harness/sso.py`:
  - Reads abra-generated `admin_password` secret directly from container (clean — no plaintext
    in repo/logs).
  - Generates `client_secret` + test-user password as class-B run-scoped secrets per §4.4-B.
  - Idempotent on realm/client/user (409 → reset to known values).
  - OIDC discovery + password grant primitives are provider-agnostic.
  - **Gap:** see F2-7 — only keycloak setup is implemented; authentik would need parallel
    backend.
- **Q2.4 lasuite-docs OIDC test** — `tests/lasuite-docs/functional/test_oidc_with_keycloak.py`:
  - Reads `deps_apps["keycloak"]` (dep domain), runs full realm/client/user setup via the
    harness, asserts OIDC discovery `issuer == https://<kc>/realms/lasuite-docs`, performs
    password grant, decodes JWT, asserts `iss`/`azp`/`typ`/`exp` claims.
  - Non-vacuous: real end-to-end. The acceptance criterion (dependent recipe deploys provider
    + OIDC login test in one run) is **substantively met** in the test's success case.
  - **Caveat:** PASS only if the dep teardown leak (F2-5) is resolved — a green run that
    leaks state is not "green" per §9.
- **F2-3 systemic fix (commit `47f7cb4`)** — `runner/harness/browser.py::goto_with_retry`
  centralizes the F2-3 try/except PlaywrightError pattern across all install overlays. Bonus
  hardening; appreciated.
- **Unit tests cold (28/28 PASS):** matches Builder's claim; new `test_deps.py` (7 tests) +
  prior 21 all green.

**Cold e2e (Adversary, HEAD `ad6b259`):**
- `RECIPE=keycloak cc-ci-run runner/run_recipe_ci.py` → install FAILED (F2-6, 502, log
  `/root/adv-q2-keycloak.log`). Parent (keyc-c1ffca) torn down cleanly post-failure.
  Pre-existing leaked dep keycloak (F2-5) `keyc-c12afe` still running independent of my
  attempt — discovered via `docker stack ls` + `docker secret ls` + `docker volume ls`.
- `RECIPE=lasuite-docs STAGES=install,custom` — NOT yet run (would deploy a fresh dep keycloak
  on top of the leaked one; defer pending F2-5 fix to avoid compounding the leak).

**What unblocks Q2:**
1. **F2-5 (required):** stop silently suppressing teardown errors; surface them; root-cause
   the underlying undeploy failure; the leaked `keyc-c12afe` stack on cc-ci should be torn
   down properly (either by fixing the leak + re-running cleanup, or by the Builder cleaning
   up manually + documenting the abra-side issue).
2. **F2-6 (strongly recommended):** make the install readiness check tolerant of the cold-boot
   502 window — either add 502 to a retry-on-transient list, or extend the timeout further, or
   diagnose what's making keycloak's HTTP layer respond before the realm is ready.
3. **F2-7 (acknowledge for Q5):** keep Q2.2 authentik genuinely open; the "pluggable" framing
   needs the work, not just the intention.

**NO VETO at this time** — F2-5 is a mechanical fix (replace `contextlib.suppress(Exception)`
with explicit logging) + a root-cause hunt on the underlying teardown failure. The dependent
recipe + OIDC harness end-to-end IS sound; the gap is honest teardown reporting.

---

## Q1 — PASS @2026-05-28 (re-verify after F2-3 + F2-4 fixes)

**Verdict: PASS.** Both findings closed by Builder commit `fc89552`:
- **F2-4 (CLOSED):** `tests/n8n/functional/test_workflow_roundtrip.py` added. Owner setup via
  `POST /rest/owner/setup` with per-run generated email + 25-char alphanumeric password (class-B
  run-scoped per §4.4-B), capture auth cookie, `POST /rest/workflows` with a Manual-Trigger
  workflow, `GET /rest/workflows/<id>`, assert id+name+nodes[0].type+nodes[0].name all round-trip.
  This IS the plan §4.3 prescribed test (create + read-back). The "execute" step is deferred with
  documented technical rationale (manual-trigger needs separate webhook activation + async polling
  fragility) — that's a defensible scope decision (a real technical reason, not a §7.1 "needs X"
  excuse), and create+read-back exercises the same persistence/retrieval surface that execution
  would use.
- **F2-3 (CLOSED):** `tests/n8n/test_install.py` wraps `page.goto(...)` in `try/except
  PlaywrightError` inside the retry loop, captures `last_err` into the failure message. Same
  pattern as F1e-1's `exec_in_app` poll+raise hardening.

**Cold environment:** `/root/adv-verify` on cc-ci, hard-reset to `origin/main` HEAD `fc89552`.
Independent of Builder's `/root/cc-ci`.

**Cold e2e on Adversary clone (first attempt, no retry):**
```
ssh cc-ci 'cd /root/adv-verify && RECIPE=n8n cc-ci-run runner/run_recipe_ci.py'
```
- **install:** generic `test_serving` PASS + cc-ci `test_serving_and_editor` PASS (no flake, but
  the F2-3 hardening is now in place for future runs).
- **upgrade:** generic `test_upgrade_reconverges` PASS + cc-ci `test_upgrade_preserves_data` PASS.
  HC1 non-vacuous: `head_ref=63dd3e0f == chaos-version=63dd3e0f`, version `3.1.0+2.9.4 →
  3.2.0+2.20.6`. Marker `upgrade-survives` written by `ops.pre_upgrade` survived the chaos
  redeploy.
- **backup:** generic `test_backup_artifact` PASS + cc-ci `test_backup_captures_state` PASS
  (marker `original` captured).
- **restore:** generic `test_restore_healthy` PASS + cc-ci `test_restore_returns_state` PASS
  (marker mutated to `mutated` pre-restore; restore returned it to `original` — real backup
  data-integrity P4).
- **custom:** 4/4 PASS:
  - `test_n8n_returns_200` (parity port, SOURCE comment)
  - `test_login_endpoint_returns_json` (auth subsystem alive)
  - `test_rest_settings_returns_json_with_known_keys` (bootstrap surface intact)
  - `test_workflow_create_and_read_back` (§4.3 prescribed; full round-trip)
- **deploy-count = 1** (DG4.1).
- **Teardown sacred:** `docker stack ls | grep -i n8n` → none; `docker volume ls | grep n8n` →
  none.

**custom-html (Q1.1):** unchanged since Q0 PASS; still good. Both recipes green; both PARITY.md
complete; data-integrity proven via the lifecycle overlay pattern.

**No new findings.**

**NO VETO.** Q1 PASS — Builder may advance to Q2 (keycloak + authentik + SSO-setup/OIDC-flow
harness primitive). F2-2 (Q0 deferred primitives) carries over — Q2 is where OIDC-flow primitive
ships, so I'll checkpoint that finding then.

---

## Q1 — FAIL @2026-05-28 (n8n specific tests fall short of plan §4.3 P3 floor) — SUPERSEDED by PASS above

**Verdict: FAIL.** Two findings filed in BACKLOG-2 ## Adversary findings:
- **F2-3 (flake / hardening gap):** the "robust install" poll loop in `tests/n8n/test_install.py`
  added by commit `2f3d5aa` doesn't catch `page.goto` exceptions (network-level errors escape the
  retry loop). Cold first-run from `/root/adv-verify` @ HEAD `df28cef` FAILED with
  `playwright.Error: net::ERR_NETWORK_CHANGED`; retry passed. Builder's evidence log filename
  `_r3` (third run) consistent with the same flake pattern.
- **F2-4 (P3 / §7.1 / §4.3 floor) — the gate-blocker:** Plan §4.3 explicitly defines the ≥2-floor
  as "create-an-object + read-it-back, and one more that touches a distinctive feature", and
  names "create a workflow via API, execute it, assert the result" as the n8n example. Builder
  shipped two API-liveness shape tests (`/rest/settings` JSON-keys; `/rest/login` JSON-shape) and
  bypassed workflow create/read-back. PARITY.md's stated reason — "n8n's REST API requires owner
  setup" — is the exact §7.1 prohibited "needs SSO setup" excuse class. Owner setup is a routine
  `POST /rest/owner/setup` with a generated class-B run-scoped secret.

**Cold environment:** `/root/adv-verify` on cc-ci @ HEAD `df28cef` (Q1 CLAIMED main).

**What I read first (anti-anchoring §6.1):** STATUS-2 Gate + objective evidence pointers; plan §6
Q1 acceptance; plan §4.3 (n8n example); plan §7.1 (Adversary mandate — "needs SSO setup" not a
valid reason); PARITY.md; the three n8n functional test bodies; ops.py; the install-overlay diff.
Did NOT read JOURNAL-2 before forming this verdict.

**Substantive findings (PASS-shaped where they apply):**
- **custom-html Q1.1:** already cold-PASSed at Q0 — re-stated, still good. No additional work
  needed; PARITY.md + functional/ + playwright/ + 2 specific tests + real backup data-integrity
  are all in place. Specifically: `test_content_roundtrip.py` writes a UUID marker into the served
  volume and fetches it back — that IS create-an-object + read-it-back per §4.3 floor. ✓ P3 met.
- **n8n parity port (test_health_check.py):** matches `recipe-info/n8n/tests/health_check.py`
  shape (HTTP 200 from `/`); SOURCE comment present. ✓ P2 met for parity row.
- **n8n PARITY.md:** mapping table present; non-ports section says none (the recipe-maintainer
  corpus for n8n contains only health_check.py — verified). ✓
- **n8n lifecycle / backup data-integrity (P4):** `ops.py` writes `original` to
  `/home/node/.n8n/ci-marker.txt` pre-backup, `mutated` pre-restore; the restore overlay reads
  the marker via `lifecycle.exec_in_app` and asserts it returned to `original`. **Real
  data-integrity**, not health-only. Cold verified: backup PASS + restore PASS at HEAD `df28cef`.
- **n8n upgrade (HC1 non-vacuous):** Builder log evidence `head_ref=63dd3e0f ==
  chaos-version=63dd3e0f`, version `3.1.0+2.9.4 → 3.2.0+2.20.6`. Marker `upgrade-survives`
  written pre-upgrade survives the chaos redeploy. ✓ HC1 honored.
- **Cold e2e (Adversary):** retry-2 → **all 5 stages PASS**, deploy-count=1, teardown sacred
  (`docker stack ls | grep n8n` → none, `docker volume ls | grep n8n` → none). Retry-1 hit F2-3.
- **Discovery + harness from Q0:** `runner/harness/http.py` + `discovery.custom_tests` (which
  recurses into functional/playwright/) flow through to n8n correctly — visible in the
  per-tier log lines `custom (cc-ci): tests/n8n/functional/test_*.py`. ✓

**Why FAIL (F2-4 detail):**

The plan's §4.3 P3 floor — "create-an-object + read-it-back, and one more that touches a
distinctive feature" — is a CONTRACT, not a guideline. Both of n8n's specific tests are
endpoint-shape liveness checks. Neither creates anything, neither reads back. Neither exercises
n8n's distinctive workflow-automation surface. Per §7.1 the Adversary "reads the test bodies, not
just pass/fail":

- `test_rest_settings.py` proves `/rest/settings` is alive and returns the bootstrap key set the
  editor SPA needs. Real failure-distinguishing assertion (the placeholder HTML 200 fails this).
  But this is "the API layer is alive", not "the workflow engine works".
- `test_login_state.py` proves `/rest/login` is alive with JSON shape — even weaker than the
  settings test (only asserts the response is dict/list, no content-shape check).

The Builder's PARITY.md justifies skipping the workflow-create test:
> "n8n's REST API requires owner setup before workflows are creatable, and the simpler /rest/
>  settings + /rest/login JSON-shape tests are equally non-vacuous"

Per §7.1 verbatim:
> "Reject 'we couldn't test X' unless it is a genuine *environment-level* limitation ... 'It's
>  hard', 'needs a browser', 'needs SSO setup', **'needs another app deployed'** are **not** valid
>  reasons — Playwright, the SSO-setup harness (§4.2), and the dependency resolver exist precisely
>  to remove those excuses."

"Owner setup needed" is in the prohibited class. Owner setup is one POST with a generated email/
password (class-B run-scoped per §4.4-B); the resulting cookie authorizes `POST /rest/workflows`
and `GET /rest/workflows/:id`. That's the test plan §4.3 prescribed.

Letting this PASS sets a low precedent: every Q2/Q3 recipe could substitute "API-liveness with
keys" for "characteristic behavior." Especially harmful for Q3 (SSO-dependent suite), where the
SSO-setup harness primitive is the whole point.

**What unblocks Q1:**
1. **F2-4 (required):** add `tests/n8n/functional/test_workflow_roundtrip.py` — owner setup via
   API with a generated password (class-B run secret), `POST /rest/workflows` (create), `GET
   /rest/workflows/:id` (read back), assert the round-trip. `test_login_state.py` can stay as a
   complement, OR be replaced; what matters is that the ≥2 specific floor contains a real
   create-and-read-back per §4.3.
2. **F2-3 (strongly recommended):** wrap `page.goto(...)` in the install poll loop in try/except
   so `playwright.Error` triggers a retry rather than test failure. Without this, every cold
   `!testme` run has a non-trivial chance of failing on the first try and needing a retry — that's
   a flaky CI signal, not a "robust install."

**Scope reminders standing:** F2-2 (Q0 deferred primitives) — OIDC-flow + dep resolver + dedicated
backup-data-integrity primitive deferred to Q2/Q3 when their consuming recipe lands. Not a Q1
gate-blocker on its own.

**NO VETO at this time** — both findings are fixable without architectural change. Builder fixes
F2-4 (and ideally F2-3), re-claims Q1; Adversary re-runs the e2e on a fresh `/root/adv-verify`
HEAD and re-PASSes.

---

## Q0 — PASS @2026-05-28 (re-verify after F2-1 fix)

**Verdict: PASS.** F2-1 fixed by Builder commit `5741e88` ("synthetic recipe + monkeypatched
`cc_ci_dir`") — exactly the prescribed pattern. Cold re-run on `/root/adv-verify` @ HEAD `0b834e9`
(Q0 RE-CLAIMED): `cc-ci-run -m pytest tests/unit -v` → **21 passed in 4.69s**. Previously-failing
`test_custom_tests_repo_local_gated` now PASSes; no other regression. E2E PASS from prior verdict
at HEAD `d480411` still stands (only `tests/unit/test_discovery.py` + `tests/n8n/PARITY.md` changed
since; no harness/lifecycle code touched between Q0-CLAIMED and Q0-RE-CLAIMED).

F2-1 **CLOSED** in BACKLOG-2 ## Adversary findings.

F2-2 (scope observation: §6 lists 5 primitives, only HTTP + TTY abra reused shipped in Q0; OIDC +
deps + dedicated backup-data-integrity primitive deferred to Q2/Q3) stands as an open observation —
not a Q0 gate-blocker; will checkpoint at Q2/Q3 verdict that the deferred primitives ship.
Builder's BACKLOG-2 Q0.4 update explicitly defers dep-resolver to Q2 — fine, transparent.

**NO VETO.** Builder may advance from Q0 → Q1 (custom-html stays green; n8n Q1.2/Q1.3 next).

---

## Q0 — FAIL @2026-05-28 (regression in test suite) — SUPERSEDED by PASS above

**Verdict: FAIL.** One real defect (F2-1) blocks PASS. Substantive Q0 work is sound — e2e cold runs
green, harness additions are real and used by the reference recipe — but a unit-test regression in
the changeset means `cc-ci-run -m pytest tests/unit -v` exits non-zero, contradicting the Builder's
"21 passed" evidence claim.

**Cold environment:** `/root/adv-verify` on cc-ci, hard-reset to `origin/main` HEAD `d480411`
(`status(2): Q0 CLAIMED — harness additions + custom-html parity reference proven`). Independent
of the Builder's `/root/cc-ci` working tree.

**What I read first (anti-anchoring §6.1):** STATUS-2 Gate + Objective evidence pointers; the
plan §6 Q0 acceptance clause; the Phase-2 plan §4.1/§4.3 contract; the four new test files; the
recipe-maintainer source `recipe-info/custom-html/tests/health_check.py`; the new unit test
`tests/unit/test_discovery_phase2.py`. Did NOT read `JOURNAL-2.md` before forming this verdict.

**Substantive findings (PASS-shaped, but gated by F2-1):**
- **Harness additions land in code (Q0.1 partial / Q0.2):**
  - `runner/harness/http.py` (233 lines) vendors `http_get` / `http_post` / `http_request` /
    `retry_http_get` / `retry_http_post` / `wait_for_http` / `assert_converges` with the same shape
    as `references/recipe-maintainer/utils/tests/helpers.py`. TLS hostname-check disabled (the
    `generic.served_cert` assertion does the real-cert sanity check once per install).
  - `runner/harness/discovery.custom_tests` (lines 102–128) recurses into `functional/` +
    `playwright/` subdirs (Phase-2 §4.1 layout) and excludes lifecycle `test_<op>.py` names; HC2
    repo-local default-deny gate still applied to subdirs (verified by `test_discovery_phase2.py::
    test_custom_tests_repo_local_subdirs_gated`).
  - TTY abra wrapper reused from Phase-1d `runner/harness/abra.py::_run_pty` (no Q0 change).
- **Per-recipe contract artifact (Q0.3 / Q1.1):**
  - `tests/custom-html/PARITY.md` records the parity row + the two recipe-specific test rationales
    + the data-integrity + playwright sections — readable, not a hollow rename.
  - Parity port `tests/custom-html/functional/test_health_check.py`: asserts HTTP 200 from
    `https://<live_app>/` via `harness.http.retry_http_get` — preserves the assertion shape of
    `recipe-info/custom-html/tests/health_check.py` (HTTP 200), adapted to the ephemeral per-run
    domain via `live_app`. SOURCE comment present for audit. P2-compliant.
  - Specific test `test_content_roundtrip.py`: writes a UUID-marked file into `/usr/share/nginx/
    html/` via `lifecycle.exec_in_app`, fetches `https://<live_app>/<filename>`, asserts the exact
    bytes round-trip. **Non-vacuous**: a stale-page or misrouted backend would fail. Validates the
    recipe's defining behavior (serving the volume).
  - Specific test `test_content_type_header.py`: writes `.html` and `.txt` files with the same
    body bytes, fetches each, asserts `Content-Type` reflects the MIME mapping (`text/html` vs
    `text/plain`). **Non-vacuous**: a misconfigured nginx falling back to
    `application/octet-stream` would fail even with HTTP 200.
  - Playwright `test_browser_smoke.py`: launches Chromium, asserts response status==200, HTML
    document present, no console errors.
- **End-to-end PASS on Adversary clone, cold:**
  - `ssh cc-ci 'cd /root/adv-verify && RECIPE=custom-html cc-ci-run runner/run_recipe_ci.py'`
    → install/upgrade/backup/restore/custom **all PASS**; deploy-count=**1** (DG4.1).
  - Custom-stage executed all 4 cc-ci-side tests: `test_content_roundtrip` PASSED,
    `test_content_type_html_and_txt` PASSED, `test_custom_html_returns_200` PASSED,
    `test_browser_renders_html` PASSED.
  - Teardown sacred: `docker stack ls | grep -i custom` → none, `docker volume ls | grep custom`
    → none. No leftover apps/volumes.
  - Log retained at cc-ci `/root/adv-q0-customhtml.log`.

**Why FAIL (filed F2-1):**
- `cc-ci-run -m pytest tests/unit -v` from `/root/adv-verify` (Q0-CLAIMED HEAD) → **1 failed,
  20 passed**. The failing test is `test_discovery.py::test_custom_tests_repo_local_gated`
  (introduced Phase-1e HC2, commit `d38a695`). Its assertion
  `discovery.custom_tests("custom-html", str(rl)) == []` is broken by Phase-2 commit `bec9265`
  adding 4 non-lifecycle `test_*.py` files under `tests/custom-html/{functional,playwright}/`.
  Behavior is correct — those files ARE legitimate cc-ci-side custom tests — but the test fixture
  used the real recipe name `"custom-html"` instead of a synthetic one. Builder's STATUS-2
  "21 passed in 4.93s" evidence does not reproduce on cold re-run.
- The fix is mechanical (~5 lines): switch the fixture to a synthetic recipe name + monkeypatch
  `discovery.cc_ci_dir`, the same pattern already used in the Phase-2 sibling
  `tests/unit/test_discovery_phase2.py`.

**Scope observation (F2-2, NOT a gate-blocker):** Plan §6 Q0 enumerates 5 primitives; Q0
changeset ships 2 (HTTP/convergence + TTY abra reused). OIDC-flow + dep resolver + dedicated
backup-data-integrity primitive remain to be implemented when their consuming recipe (Q2 keycloak/
authentik for OIDC; Q3 SSO-dependent for deps) lands. BACKLOG-2 Q0.4 is still `[ ]` open.
Custom-html (no SSO, no deps) cannot exercise those primitives, so the literal "uses them" clause
holds for the subset that applies — but Q0 is not "complete" in the broad §6 sense until Q2/Q3
fills in the rest. Filed for transparency; will check off when Q2/Q3 ships.

**Next:** Builder fixes F2-1 (test rewrite), re-claims Q0; Adversary re-runs `pytest tests/unit -v`
(expect 21/21) and the e2e PASS already stands. NO VETO at this time — F2-1 is a small,
mechanical fix, not a fundamental design issue.

## Watchdog ping @~2026-05-28 07:xxZ — FALSE POSITIVE (no verdict)

Watchdog claimed Builder CLAIMED `[D5 F3 N8 Q1]`. Cold check after `git pull --rebase`:
- STATUS-2 Gate section still shows the **old** "Q0 — RE-CLAIMED" text (stale w.r.t. my Q0 PASS
  in commit `5ab25c3`). No Q1 claim line, no `Gate: Q1 — CLAIMED` marker, no commit-evidence
  pointer.
- Builder commit `2f3d5aa` ("feat(2): Q1.2 — n8n Phase-2 parity + functional + robust install (full
  e2e green)") is **in-progress Q1 work** — n8n PARITY.md + 3 new `functional/test_*.py` files +
  install hardening. No Q1 gate claim accompanies it.
- "Q1" appears only in the "In flight" section header. D5/F3/N8 don't map to any Phase-2 gate
  identifier (Phase 2 milestones are Q0–Q5; findings are F2-N).

No verdict written — nothing CLAIMED to verify. Held anti-anchoring: did NOT read the new n8n test
bodies before a Q1 claim arrives. Returning to idle.

## Watchdog ping @~2026-05-28 04:35Z — FALSE POSITIVE (no verdict)

Watchdog claimed Builder CLAIMED `[C6 D0 Q0 Q1]`. Cold check after `git pull --rebase`:
- Builder commit `8f5df6d` bootstraps `STATUS-2.md` / `BACKLOG-2.md` / `JOURNAL-2.md` (+ Phase-2
  section in `DECISIONS.md`). Nothing more.
- `STATUS-2.md` "Gate:" line literally reads `(none yet — Q0 has not been claimed)`.
- `STATUS-2.md` "In flight:" reads `Q0 — Harness additions. Bootstrap … begin porting helpers`.
- Q0/Q1 appear only as headings under "Milestones" and `## Build backlog` (open `[ ]` items, no
  CLAIMED marker). C6 and D0 are not Phase-2 identifiers at all (C6 was the Phase-1c throwaway-VM
  decision; D0 is nowhere in any phase plan).
- Verbatim grep: `grep -n -E '(CLAIMED|VETO)' machine-docs/STATUS-2.md` → no match.

No gate is actually claimed. The watchdog likely string-matched on milestone identifiers anywhere
in the file. **No verdict written** (nothing to verify). Held discipline: did NOT read `JOURNAL-2.md`
to avoid anchoring on the Builder's Q0 reasoning before a real claim arrives. Returning to idle.


---

## Idle-wake checkpoint @2026-05-28T18:58Z (no gate claimed)

**Cold access re-verified:** dashboard `https://ci.commoninternet.net/` HTTP 200 via SOCKS proxy
(127.0.0.1:1055); `ssh cc-ci` ok (root, NixOS 24.11 Vicuna). Proxy healthy.

**State:** HEAD `f59d8e6`. No `Gate: <Mn> CLAIMED` line in STATUS-2. Q0/Q1/Q2 PASS stand;
Builder mid-sprint (Q3/Q4 partials, already checkpointed). Latest landed = Q3.2 lasuite-drive
**base enrollment** (`f59d8e6`). No verdict written (nothing claimed). JOURNAL-2 not read.

**lasuite-drive Q3.2 (in-flight, NOT a claim — observations for when it IS claimed):**
- Honest base-only: `recipe_meta.py` keeps `DEPS=["keycloak"]` commented OFF until base deploy is
  cold-green; only `functional/test_health_check.py` shipped; SSO + §4.3 specifics explicitly
  deferred to the SSO iteration. Transparent, well-documented (nested-subdomain flatten +
  DEPLOY/HTTP/TIMEOUT bumps rationalised in recipe_meta + DECISIONS). No finding — partial WIP.
- **When Q3.2 is formally claimed it must show (plan §4.3 lasuite-drive line):** keycloak dep
  auto-deployed; OIDC functional test; **≥2 specific incl. create-an-object+read-back** = upload a
  file to a workspace + list/download it back, and MinIO bucket present; real backup data-integrity
  (P4); PARITY.md mapping. Base health-only will NOT satisfy P3 at gate.

**Standing §4.3-floor audit (forward-looking DONE conditions — NOT reopening closed findings).**
Read the shipped functional bodies for the recipes whose create-and-read-back is parked in
DEFERRED.md:
- **ghost** — specific tests are `test_admin_redirect` (route 200/302 + body contains "ghost") and
  `test_content_api` which **accepts 401/403/400 as PASS** → asserts ~nothing material about app
  behaviour (P7 concern: liveness/route-existence stand-in, no object created/read). create-post
  deferred (DEFERRED.md, reason = "owner-setup + JWT" — a §7.1-disallowed "needs setup" excuse, NOT
  operator-confirmed). **At DONE I will require ghost's §4.3 create-an-object+read-back implemented,
  OR an explicit operator DoD amendment.**
- **uptime-kuma** — `test_socketio_handshake` (sid+pingInterval) IS distinctive/non-vacuous (good);
  `test_spa_branding` is thin; create-monitor deferred (F2-10, closed via DEFERRED.md route on
  operator-confirmed framing). I will hold to that closure, but the create-monitor §4.3 floor
  remains unmet — surfaced for the Phase-4/operator review the DEFERRED.md preamble mandates.
- **cryptpad** — create-pad deferred; **F2-9 conditional sign-off already requires this lifts
  before Phase-2 DONE** (Q5.2 cold-sample MUST include a real create-pad-and-persist test).
- **matrix-synapse** — its three operational-script deferrals (compress_state/complexity/purge) are
  PARITY (P2), operator-confirmed heavy, and §4.3 floor is independently met by
  `test_register_and_message` (create-room+message+read-back). Defensible; not in scope of this audit.

**Consolidated Phase-2 DONE-blocking conditions (what a `## DONE` claim must clear):**
1. **F2-7** — authentik (Q2.2) enrolled + `setup_authentik_realm` SSO backend (proves the SSO
   harness is *pluggable*, not keycloak-only). Currently in DEFERRED.md, open.
2. **F2-9** — cryptpad real create-pad-and-persist test (conditional sign-off, must lift).
3. **§4.3 create-an-object+read-back floor** for **ghost** (and any other recipe shipping only
   liveness/route specifics) — implement, or carry an explicit operator DoD amendment. ghost's
   `test_content_api` accepting 401/403 as PASS is the weakest current specimen.
4. **P1 coverage** — the remaining §5 recipes (lasuite-drive full, lasuite-meet, immich,
   mattermost-lts, discourse, mailu, drone, plausible) each green via the run path.
5. Full P1–P8 cold re-verify (Q5) against the literal plan §2 checklist — DoD boxes must reflect
   reality (no box ticked while its §4.3 floor sits unimplemented in DEFERRED.md).

**No VETO** (no DONE claim to block yet). No new blocking finding filed on unclaimed WIP. Returning
to self-paced idle; will verify promptly when a gate is claimed (watchdog edge-ping) or re-verify a
stale D-gate >24h.

## Idle break-it probe @2026-05-28 — F2-11 filed (SSO-skip-goes-green); git host outage noted

**Git coordination host down.** `git.autonomic.zone` returns a bare Go `404 page not found`
(text/plain, 19 bytes) on EVERY path incl. root `/` — the Gitea app is down behind its proxy
(not a deleted repo: my local clone still tracks `origin/main` and is ahead 1 with my prior
review checkpoint). `git fetch/push` both fail. External, transient infra. **Test infra is up**
(`ssh cc-ci` OK, dashboard 200 via SOCKS, load avg ~8 → a run likely in flight). No gate is
CLAIMED. Verdicts/commits accumulate locally and push when the host recovers.

**Independent probe (no git needed):** read the SSO-dep skip path end-to-end and cold-proved the
hazard. Filed **F2-11** in BACKLOG-2 (full detail there). Summary:
- `setup_custom_tests` failure → `CCCI_DEPS_READY=0` (`run_recipe_ci.py:528`) →
  `conftest.py:98` skips every `@pytest.mark.requires_deps` test → a skip-only pytest file exits
  **0** (cold-proven on cc-ci: `1 skipped`, `PYTEST_EXIT=0`) → `run_custom` returns `"pass"`
  (`run_recipe_ci.py:372`) → `overall=0` → **`!testme` reports GREEN while the only SSO test for
  that recipe never ran.** Counter-signal is one conditional `deps-not-ready:` line; no skip count
  in the summary, no effect on the green/exit signal.
- **Does NOT compromise Q2 PASS** — Q2.4's `test_oidc_password_grant_against_dep_keycloak`
  actually PASSED (deps were ready), per the recorded evidence. Latent hazard for future Q3
  SSO-dep gates + the standing `!testme` signal.
- **Binding on my future verdicts:** no SSO-dep recipe gate accepted on a green exit alone — I
  will grep the run log for `SKIPPED`/`deps-not-ready` on `requires_deps` tests and require the
  OIDC/SSO test to have actually PASSED.
- Recommended (not a VETO): surface skipped `requires_deps` tests in RUN SUMMARY + make an
  unexpected deps-not-ready skip gate-blocking for the declaring recipe, while preserving
  generic-tier failure-isolation.

**No VETO.** No gate claimed. Returning to self-paced idle; will retry the git host and re-orient
on Builder activity on next wake.

## F2-11 re-verify @2026-05-28 — FIXED (deploy-free cold proof); inbox consumed

Builder commit `5b34496` fixes F2-11 (SSO-dep deps-not-ready SKIP no longer yields a GREEN run).
Consumed `ADVERSARY-INBOX.md` (F2-11 fixed + deploy work paused on Docker Hub rate limit) — deleted
to mark consumed. Read the fix code + the 7 new unit-test bodies (not just pass/fail).

**Cold re-verify on `/root/adv-verify` HEAD `0d6cd05` (deploy-free — rate-limit-independent):**
- `cc-ci-run -m pytest tests/unit -q` → **35 passed** (28 prior + 7 new `test_f211_sso_skip.py`).
- Real signal: `tests/lasuite-docs/functional/test_oidc_with_keycloak.py` (DEPS=["keycloak"]) with
  `CCCI_DEPS_READY=0` → `1 skipped`, **pytest-exit=0** (hazard) BUT `$CCCI_DEPS_SKIP_REPORT` == `1`.
- Stitched to the real predicate: `sso_dep_unverified(["keycloak"], False, 1) = True` → `overall=1`
  (RED). Negatives: `deps_ready=True → False`, `no-deps → False`. Generic-tier isolation preserved
  (predicate only flips `overall`; tier results untouched), no false-fail.
- Runtime wiring confirmed by code-read (`main():445` sets the report path before the custom tier;
  `_tier_env` = `dict(os.environ,…)` propagates to the pytest subprocess; orchestrator sums the
  same `skipfile` at `:582-585` and applies the predicate at `:633`).

**Verdict: F2-11 CLOSED** (BACKLOG-2 marked `[x]`). NO VETO. F2-11 was a finding, not a gate — no
gate is CLAIMED. **Residual (non-blocking):** the live-deploy e2e (forced `setup_custom_tests`
failure on a real recipe → `overall=1` end-to-end) is Builder-deferred behind the Docker Hub pull
rate limit; the logic + signal it exercises are proven here. I'll confirm the live path on the next
SSO-dep deploy once pulls flow.

Standing DONE-gate conditions unchanged (F2-7 authentik, F2-9 cryptpad create-pad, ghost §4.3 floor,
P1 coverage of remaining §5 recipes, full P1–P8 Q5 cold re-verify) — all deploy-gated, awaiting the
rate-limit unblock. Returning to self-paced idle; watchdog edge-pings on the next gate claim.

## Rate-limit fix — pre-wiring baseline @2026-05-28 (operator provided Docker Hub creds, Class A1)

Operator provided `DOCKERHUB_USERNAME=nptest2` + `DOCKERHUB_TOKEN` (read-only PAT) in
`/srv/cc-ci/.testenv` to clear the `toomanyrequests` blocker. Builder will wire it (sops PAT into
`secrets/`, declarative NixOS docker auth, `--with-registry-auth` for swarm service pulls). My job:
verify AFTER wiring. Captured the **"before" baseline** now for contrast (cc-ci):
- Anonymous manifest HEAD → `ratelimit-limit: 100;w=21600` (100/6h), `ratelimit-remaining: 4`
  (window nearly exhausted — blocker confirmed real), `docker-ratelimit-source: 68.14.43.142`
  (the shared IP).
- `/root/.docker/config.json` → no `auths` yet (unwired).

**Verification I'll run once Builder signals wiring done:**
1. Authenticated pull from cc-ci → expect `ratelimit-limit: 200;w=21600` and
   `docker-ratelimit-source` = an ACCOUNT hash, NOT `68.14.43.142`.
2. A real recipe deploy no longer hits `toomanyrequests` (and swarm SERVICE task pulls authenticate
   — the `--with-registry-auth` / daemon-config subtlety the orchestrator flagged; a bare node
   `docker login` is NOT sufficient).
3. Persistence across a 1c rebuild: PAT sops-encrypted in `secrets/` (never plaintext) + the auth
   wired declaratively in NixOS (not just an imperative `docker login`); wiring recorded in
   DECISIONS.md. Rate-limit finding closed only when 1–3 hold.

Not wiring it myself (Builder owns code/config). Idling until the Builder signals.

## Rate-limit fix — PARTIAL verify @2026-05-28 (immediate relief confirmed; persistence + swarm pulls pending)

Builder has done the immediate-relief node `docker login` (orchestrator-sanctioned). State on cc-ci:
- `docker info` → `Username: nptest2`; `/root/.docker/config.json` has an `index.docker.io` auths
  entry.
- **Authenticated ratelimit (via cc-ci's OWN stored cred — PAT never exposed in my commands):**
  `ratelimit-limit: 200;w=21600` (vs anon 100), `docker-ratelimit-source:
  b662dd8b-81ac-4b81-bf8a-a9c0a466ad4e` — an ACCOUNT hash, NOT the shared IP `68.14.43.142`.
  ✓ **Condition 1 (authenticated 200-limit from account source) — CONFIRMED.**

**Rate-limit finding NOT yet closeable — two conditions remain:**
2. **Swarm SERVICE-task pulls authenticate** — a node `docker login` does NOT guarantee swarm
   service pulls carry the cred (orchestrator's explicit subtlety: need `docker stack deploy
   --with-registry-auth` or daemon-level config). Verify with a REAL deploy that clears
   `toomanyrequests` — and guard against a false pass from already-cached base images (prefer a
   recipe whose images aren't cached, or inspect the abra/stack deploy path for `--with-registry-auth`).
   Deploy-gated; verify when the Builder runs the next recipe deploy.
3. **Declarative persistence across a 1c rebuild** — currently only an IMPERATIVE `docker login`
   (survives reboot but NOT a NixOS rebuild that re-provisions the node). Operator requires: PAT
   sops-encrypted in `secrets/` (no plaintext), docker auth wired declaratively in NixOS, recorded
   in DECISIONS.md. None present yet (no docker secret in `/root/cc-ci/secrets/`, origin/main has no
   wiring commit).

Verdict: immediate relief WORKS (deploys can proceed now); the finding stays OPEN until 2 + 3 hold.
No VETO. Idling for the Builder's declarative wiring + next deploy.

## Rate-limit fix — VERIFIED / finding CLOSED @2026-05-28 (all 3 conditions, cold)

Builder commits `5e14963` (sops dockerhub_auth + config.json template), `7a337f5` (STATUS RESOLVED +
DECISIONS), secrets submodule `cdd5e0a`. Consumed `ADVERSARY-INBOX.md` (deleted = consumed). All
three conditions independently re-verified cold on cc-ci — NOT taken on the Builder's word:

1. **Authenticated 200-limit from account source — CONFIRMED** (prior tick + re-confirmed):
   `ratelimit-limit: 200;w=21600`, `docker-ratelimit-source: b662dd8b-…` (account UUID, NOT shared
   IP `68.14.43.142`). Account remaining moved 197→195 across ticks → real authenticated activity.

2. **Swarm SERVICE-task pulls authenticate — CONFIRMED by my OWN uncached-image test** (not the
   Builder's deploy): created a throwaway `docker service create traefik/whoami:latest` with the
   image VERIFIED uncached (`docker images | grep -c whoami` → 0). Task reached `Running` in ~5s,
   **error column empty — no `toomanyrequests`/rejected/failed**; service removed clean. Decisive on
   authentication by architecture: **single-node swarm** (`docker node ls` → only `nixos`), so
   service tasks pull via the same local daemon whose `/root/.docker/config.json` is the
   sops-rendered auth — no anonymous worker path exists; `--with-registry-auth` is a multi-node
   concern that doesn't arise here. (Honest caveat: the `ratelimitpreview` HEAD counter didn't tick
   down across my single pull — a known real-time-fidelity quirk of that endpoint within a short
   window; it moves over longer spans as the cross-tick 197→195 shows. Not evidence against auth.)

3. **Declarative persistence across a 1c rebuild — CONFIRMED cold:**
   - `/root/.docker/config.json` → symlink to `/run/secrets/rendered/docker-config.json`
     (sops-rendered at NixOS activation, not an imperative `docker login`).
   - `nix/modules/secrets.nix:69-74` — `sops.templates."docker-config.json"` renders the auths block
     from `${config.sops.placeholder.dockerhub_auth}` → re-rendered every rebuild/reboot.
   - `secrets/secrets.yaml` — `dockerhub_auth: ENC[AES256_GCM,…]` (encrypted; no plaintext PAT in git).

**Verdict: rate-limit blocker RESOLVED; finding CLOSED. NO VETO.** Deploys can proceed; Builder is
resuming Q3.2 (lasuite-drive base now converges per their note — I'll verify Q3.2 specifics when
claimed). NOTE (not a blocker): 200/6h may still be tight for a full ~18-recipe sweep — the
pull-through cache (Phase 2b) is the structural fix; flagging so a future broad sweep doesn't silently
re-hit `toomanyrequests`.

## Idle break-it probe @2026-05-29 — cross-phase: 2w WC5 canonical-promotion × F2-11 SSO-skip — NO regression

Independent probe (no gate pending in Phase 2; Phase 2 dormant while 2w ran to DONE). Phase 2w added
**WC5 promote-on-green-cold** — a green cold run on LATEST advances/seeds a recipe's warm canonical.
Adversarial question: can that NEW promotion path resurrect the **F2-11** hazard (a deps-not-ready SSO
recipe whose `@requires_deps` tests SKIP, formerly going GREEN) by promoting a recipe as canonical
whose SSO/OIDC was never actually verified? Verified COLD against origin/main HEAD `aebb28d` (my clone)
+ live host:

1. **Promotion is strictly gated on the fully-computed `overall`.** `should_promote_canonical`
   (`runner/run_recipe_ci.py:606-611`) returns true iff `is_enrolled ∧ overall==0 ∧ ¬quick ∧ ¬ref`.
   In `main()` the F2-11 flip `sso_dep_unverified(declared, deps_ready, requires_deps_skipped)` sets
   `overall=1` at line 942-949 — **before** the promote check at line 958. So a deps-not-ready SSO run
   has `overall=1` → `should_promote_canonical` False → NOT promoted. Same ordering in the `--quick`
   path (which never promotes regardless).
2. **No alternate promotion path.** `seed_canonical` is reached ONLY via `promote_canonical`
   (run_recipe_ci.py:637), itself called ONLY behind the gate at :958. The WC6 nightly sweep
   (`nightly_sweep.py:62-67`) drives each recipe via `RECIPE=<r> run_recipe_ci.py` with **no REF** —
   the same `main()` gate, not a direct promote. Grep across `runner/**.py` confirms no other call site.
3. **Unit-level coverage of both halves.** `tests/unit/test_promote.py::test_no_promote_when_red`
   asserts `should_promote_canonical(...,1,quick=False) is False`; `test_f211_sso_skip.py` asserts the
   SSO-skip→`overall=1` half. Full unit suite re-run cold on the host: **72 passed in 4.84s**
   (`ssh cc-ci 'cd /root/cc-ci && cc-ci-run -m pytest tests/unit -q'`).

**Result: NO regression — F2-11 stays CLOSED under 2w's WC5 promotion. No finding, NO VETO.** A
nightly-sweep run whose warm keycloak is down (deps-not-ready) fails (`overall=1`) and does NOT
advance the canonical to an SSO-unverified version — the desired safety property holds.

## Disk-blocker LIFTED — cold-verified @2026-05-29; lasuite-drive upgrade tier now REQUIRED (not deferrable)

Orchestrator resized cc-ci 30→70GB (VM restart). Independently re-verified post-restart (did NOT take
the orchestrator's word):
- `ssh cc-ci df -h /` → **64G total, 44G free (30% used)** (was ~11G free). 44G free ≫ the ~10GB
  transient onlyoffice+collabora upgrade crossover → the disk-exhaustion blocker is genuinely gone.
- Public `https://ci.commoninternet.net/` → **HTTP 200** (via SOCKS proxy).
- Infra all up: `docker stack ls` = traefik(2) + ccci-dashboard + ccci-bridge + drone + backups
  (backup-bot-two) + warm-keycloak(2); `warm-keycloak …_app 1/1`, `…_db 1/1` converged. Single-node
  swarm Leader Ready.

**Adversary stance:** the disk-blocker deferral basis is now VOID. The lasuite-drive Q3.2 **upgrade
tier** (prev→PR-head in-place `deploy --chaos`, the office-image crossover) — and any other heavy
upgrade tier parked on disk — is **no longer validly deferrable**. To sign off Q3.2 (and before
Phase-2 `## DONE`) I REQUIRE that upgrade tier to run **GREEN** and I will **cold-verify it myself**
(real prev→PR-head upgrade, app healthy after; no health-only stand-in). A claim that still defers it
= FAIL. **I hold this as an OPEN, veto-eligible obligation** until cold-verified.

**On DEFERRED.md:** the orchestrator noted the disk-blocker DEFERRED entry can be closed. I am
deliberately **NOT** editing DEFERRED.md — (a) it is the Builder's single-writer registry (ownership
discipline; the Builder received the same orchestrator signal), and (b) "closing" it now would
misstate the truth: the disk *constraint* is lifted, but the upgrade *test* is still UNPROVEN. The
entry should convert from "deferred (disk)" to active required work, which only becomes truly closed
when the tier runs green and I verify it. Builder owns the file edit; I hold the verification gate.

## (forward-looking) Adversary cold-verify criteria for lasuite-drive Q3.2 rework @2026-05-29

Orchestrator queued `cc-ci-plan/plan-lasuite-drive-oidc-robustness.md` (skimmed — disk lift noted in
it). NOT active yet (Builder finishing current unit). When the lasuite-drive Q3.2 rework is claimed I
will enforce, cold:
1. **Step 0 evidence** — real captured failure logs (collabora WOPI-discovery timing, backend log at
   the 404, exact gunicorn-perms error) exist before any "fix"; not a guessed root cause.
2. **Part A — wire-OIDC-at-INSTALL, deploy ONCE.** No mid-run `abra app deploy --chaos` reconverge.
   **ENFORCE REAL-abra-only (operator rule):** grep `setup_custom_tests`/harness for
   `docker service update`/`docker service scale` surgical patches → any such bypass = FAIL (CI must
   exercise the real abra path). Deploy-count discipline still holds (install = 1 deploy).
3. **Part B — root-cause recipe PR** (collabora WOPI healthcheck-gating + backend retry, gunicorn-perms
   startup race, lazy/retrying OIDC discovery). RULE (operator): the recipe change counts as "working"
   ONLY when cc-ci runs the **full suite on that PR repeatedly GREEN + Adversary cold-verified**, then
   the operator merges. So I require **repeat green** (not a one-off) + my own cold re-run + read the
   assertions, **including the now-required upgrade tier** (disk lifted).
This extends the open, veto-eligible obligation recorded above (disk-blocker LIFTED entry). DEFERRED.md
plan-link + entry update is the Builder's (its single writer).

## @2026-05-29 — Cross-phase regression probe (2pc→Phase-2 boundary): warm infra INTACT — no finding
Phase 2pc (`## DONE`, my PASS `486d162`) replaced the daily `docker system prune --all`/`autoPrune`
with the gated `ci-docker-prune`. Phase 2w (`## DONE`, my PASS `2822d60`) relies on warm volumes
surviving any prune (WC8: prune must NOT carry `--volumes`). Adversarial concern: did the 2pc
nixos-rebuild + prune-policy change regress the 2w warm foundation that Phase 2 now resumes on?
Cold-checked on cc-ci:
- system `running`, **0 failed units**.
- 2pc state intact: `ci-docker-prune.timer` **active**; old `docker-prune.timer` **not-found**.
- 2w state intact: `nightly-sweep.timer` **active**; `warm-keycloak.service` **active**.
- **Warm volumes SURVIVED the prune-policy change** (the real test): `warm-custom-html…content`,
  `warm-keycloak…mariadb`, `warm-keycloak…providers` all present; `canonical.json` = custom-html
  **idle @ 1.11.0+1.29.0** (commit 8a02606), unchanged.
- disk `/` **27% (45G free)** — healthy; the ≥80%-gated prune correctly no-ops.
**Result: NO regression, NO finding, NO VETO.** 2pc's surgical prune (no `--all`/`--volumes`) preserves
2w's warm cache. Phase 2 resumes on a sound foundation. Standing veto-eligible obligations from the
entries above remain OPEN (lasuite-drive Q3.2 upgrade tier GREEN + cold-verify; cryptpad F2-9 create-pad).

## @2026-05-29 — Pre-claim recon: lasuite-drive Q3.2a Part A (in-flight @f89cf9b, NOT yet claimed — no verdict)
Builder is validating Q3.2a Part A ("wire OIDC at INSTALL, eliminate flaky redeploy"). Read the code
ahead of the claim so my verdict is instant. Findings to carry into the gate (re-verify live then):
- **`setup_custom_tests.sh:26` `docker service scale --detach …_minio-createbuckets=1`** initially
  tripped my real-abra-only grep, but it is **NOT a surgical bypass**. Upstream ships
  `minio-createbuckets` at **`replicas: 0`** (confirmed in the abra recipe cache compose, line 239) —
  a one-shot the deploy intentionally leaves dormant; the hook triggers the *recipe's own* job and
  polls the real bucket. My FAIL trigger is `service update/scale` used to patch a broken deploy into
  false health — this isn't that. ACCEPTABLE pending live re-confirm.
- **`install_steps.sh`** writes OIDC env + inserts the real `oidc_rpcs` client secret (bumped version)
  into `.env` BEFORE the single `abra app deploy` → satisfies Part A deploy-once (no post-deploy
  `--chaos` reconverge). No `docker service update/scale` patching of app state. Clears the
  FranceConnect `acr_values=eidas1` so keycloak can satisfy the flow.
- **`functional/test_minio_storage.py`** is a genuine S3 round-trip (upload via `mc pipe` → list →
  `mc cat` readback → assert marker content survives), runs `mc` inside the real `minio` container.
  ast PARSES_OK, no stub/`pass`/`skip`. Non-vacuous (SPA-200 ≠ pass).
**Still enforced at claim (unchanged from the obligations above):** deploy-count discipline
(install = 1 deploy, no mid-run reconverge), the now-REQUIRED **upgrade tier GREEN** (disk lifted),
repeat-green + my own cold re-run reading the assertions. This note is recon only — NO PASS/FAIL until
the Builder claims the gate.

## Q3.2 lasuite-drive — FAIL @2026-05-29 (cold-verify; gate claim 911680f / code 4b38b66)
Cold-verified from my own clone `/root/adv-verify` synced to origin/main `911680f` (claim commit is
**docs-only** — BACKLOG-2/DEFERRED/STATUS-2; verified *code* == `4b38b66`. git==host confirmed:
Builder `/root/builder-clone` @ 4b38b66, deploy tree clean). Ran `RECIPE=lasuite-drive PR=0 cc-ci-run
runner/run_recipe_ci.py` from /root/adv-verify (log `/root/adv-q32-102348.log`).

**Result — RUN SUMMARY (verbatim):**
```
deploy-count = 1 (expect 1)
  install : pass
  upgrade : fail        <-- FAILS the gate (claim said full lifecycle 3x green)
  backup  : pass
  restore : pass
  custom  : pass
```

**Root cause (from the actual log + abra deploy log — NOT the WOPI gate):** the collabora WOPI-discovery
pre-upgrade gate **worked** — log line 43: `pre_upgrade: collabora WOPI discovery ready (200) on
collabora-lasu-cbcdd6.ci.commoninternet.net`. The failure is the **chaos upgrade deploy itself not
converging**: line 44 `!! upgrade op failed: abra app deploy lasu-cbcdd6.ci.commoninternet.net -o -n -C
failed (1)` → `INFO polling deployment status` → `FATA deploy failed 🛑`
(abra log `/root/.abra/logs/default/lasu-cbcdd6...2026-05-29T103335Z`). This was a real prev→PR-head
crossover with heavy image bumps — collabora/code 25.04.9.1.1→**25.04.9.4.1**, drive-backend
v0.12.0→**v0.18.0**, drive-frontend v0.12.0→**v0.18.0**, onlyoffice 9.2→**9.3.1.2**, nginx 1.29→1.30,
redis 8→8.6.3. The abra deploy log shows the NEW collabora still doing lengthy jail/config init
(`Kit core version …`, hundreds of `Linking file …` lines, `child-roots/.../etc/* needs to be updated`)
when abra's convergence poll gave up. So the upgrade redeploy timed out waiting for the new collabora
to become healthy, not the pre-deploy gate.

**Why FAIL, not a flake-to-retry:**
- The claim is **"flakiness gone, full lifecycle 3× green"** (r2/r3/r4). My **first independent cold
  run** does NOT reproduce green — the upgrade tier fails. That contradicts "reproducibly green."
- Upgrade-tier GREEN is my **standing veto-eligible obligation** (disk lifted; deferral void). My
  stated criteria required **repeat-green + my own cold re-run** of the upgrade tier. It failed on my run.
- The new-collabora-convergence timeout is the *same class* of collabora-timing problem `4b38b66` set
  out to fix; the WOPI pre-gate addresses readiness of the OLD collabora before redeploy, but does not
  ensure the NEW collabora (heavier 25.04.9.4.1) converges within abra's upgrade poll window. The fix
  is incomplete for the crossover it claims to make green.

**What DID verify (fix is partial, not worthless):**
- **Part A install-time OIDC — GREEN & real.** `deploy-count = 1` (single deploy, no post-deploy
  `--chaos` reconverge); log: `using live-warm keycloak … per-run realm`, `install_steps: OIDC env wired
  into .env (… no reconverge)`; `test_oidc_password_grant_against_dep_keycloak` **PASSED, not skipped**
  (real password-grant JWT vs a per-run realm). **Real-abra-only confirmed** — no `docker service
  update/scale` patching of app state (the lone `service scale …minio-createbuckets` triggers the
  recipe's own `replicas:0` one-shot; established acceptable in my pre-claim recon).
- **install + backup + restore + custom all pass**; `test_minio_storage` (S3 round-trip) PASSED.
- **Teardown sacred:** post-run NO `lasu` stacks, NO per-run `lasu` volumes; warm-keycloak + warm
  custom-html canonical volumes intact (prune/teardown didn't touch the cache).

**FILED: F2-12 [adversary] (BLOCKS the Q3.2 gate).** No phase `## VETO`. Q3.2 cannot PASS until the
**upgrade tier runs GREEN on my own cold re-run** (repeat-green). Likely real fixes for the Builder to
consider: raise the abra upgrade convergence timeout for the new-collabora crossover (the recipe-internal
TIMEOUT/`DEPLOY_TIMEOUT` covers the python subprocess, but abra's own per-service convergence poll is
what emitted `FATA deploy failed`), and/or a post-redeploy collabora-health wait before asserting
reconverge. Anti-anchoring honored: verdict formed from the plan + code + my own run's observable log;
I did NOT read JOURNAL-2 before writing this.

## @2026-05-29 — Pre-claim recon: F2-12 fix e1147b5 (NOT re-claimed yet — no verdict)
Builder ACKed F2-12 and pushed fix `e1147b5` ("own convergence wait via abra `-c` + collabora
READY_PROBE"), status `cc4af49` = validating multi-run before RE-CLAIM. Read the fix ahead of the
re-claim. **The adversarial crux: the upgrade redeploy now passes `abra … -c` (`--no-converge-checks`),
which skips abra's own convergence monitor.** Skipping a convergence check is exactly the shape of a
P7 weakening — so I scrutinized whether the replacement is genuinely stronger or a green-washing.
- **Plausibly NOT a weakening (pending cold proof):** `-c` only skips abra's *post-deploy monitor*;
  `docker stack deploy` (the real spec apply) still runs. The harness then owns the verification in
  `generic.perform_upgrade`: `lifecycle.wait_healthy` (= `_wait_services_converged` "every swarm
  service shows running == configured replicas" + HEALTH_PATH) **then** `lifecycle.wait_ready_probes`
  (collabora `/hosting/discovery` → 200), bounded by the generous recipe DEPLOY_TIMEOUT. The READY_PROBE
  loop **raises TimeoutError** if discovery never hits 200 (while/else) → upgrade op fails → tier fails,
  so it's non-vacuous by construction. HC1 (chaos-version label == PR-head) preserved; chaos_redeploy
  still bypasses deploy_app so deploy-count stays 1.
- **MUST cold-verify at re-claim (cannot fully settle by reading):**
  1. **Upgrade tier GREEN on MY own cold run** — the F2-12 close condition (repeat-green, not one-off;
     Builder admits it was 3×green/1×fail before this fix).
  2. **P7 negative:** confirm `_wait_services_converged` truly fails on a stuck `0/1` service (i.e. `-c`
     + owned-wait catches a genuinely broken converge, not just a slow one). I started reading its
     parser (lifecycle.py ~286–328) — finish that read + ideally observe a broken-upgrade-still-RED.
  3. deploy-count == 1; clean teardown.
F2-12 stays OPEN (Adversary-owned). NO verdict until Q3.2 is re-claimed. Anti-anchoring: not reading
JOURNAL before the verdict.

## Q3.2 lasuite-drive — PASS @2026-05-29 (cold re-verify after F2-12 fix; re-claim a13d2ae / code e1147b5+6506c4a)
Cold-verified from my own clone `/root/adv-verify` @ origin/main `a13d2ae` (git==host: Builder
`/root/builder-clone` also a13d2ae). `RECIPE=lasuite-drive PR=0 cc-ci-run runner/run_recipe_ci.py`
(log `/root/adv-q32-reclaim-114620.log`). **F2-12 CLOSED.**

**RUN SUMMARY (verbatim):** `deploy-count = 1 (expect 1)`; **install/upgrade/backup/restore/custom
ALL pass** — the upgrade tier (which FAILed my first cold run, aab77ea) is now GREEN.

**Every per-test PASSED (read the lines — nothing skipped/health-only):**
- install: `test_serving` + `test_serving_and_frontend`.
- **upgrade: `test_upgrade_reconverges` + `test_upgrade_preserves_data`** (ci_marker survives the real
  prev→PR-head chaos crossover — collabora/code 25.04.9.1.1→25.04.9.4.1, drive v0.12→v0.18, onlyoffice
  9.2→9.3).
- backup: `test_backup_artifact` + `test_backup_captures_state`; restore: `test_restore_healthy` +
  `test_restore_returns_state` (real backup data-integrity, P4).
- custom: `test_health_check`, **`test_minio_storage` (real S3 upload→list→cat readback round-trip
  inside the minio container)**, **`test_oidc_password_grant_against_dep_keycloak` PASSED — NOT skipped**
  (real password-grant JWT vs a per-run realm on warm keycloak).
- Log shows `ready-probe OK (200)` **TWICE** — post-install AND post-upgrade — on
  `collabora-lasu-e511fe…/hosting/discovery`.

**F2-12 fix is NOT a P7 weakening (the crux — orchestrator 2026-05-29 requires the probe have teeth):**
the upgrade redeploy is still REAL abra (`abra app deploy … -C -c`); only abra's *impatient converge
monitor* is replaced — `docker stack deploy` still applies the spec. The harness then OWNS a STRICTER
wait, and I verified it is non-vacuous by reading the code AND running the negative tests:
- `services_converged` (lifecycle.py:171) checks **EVERY** stack service `cur==want` (N/N), returns
  False on any `0/1` still-spinning service (correctly treats `replicas:0` one-shots as 0/0 converged).
- `wait_healthy` RAISES `TimeoutError` if services never converge, OR converge but the app never serves
  an OK code. `wait_ready_probes` RAISES if collabora `/hosting/discovery` never returns 200.
- `tests/unit/test_f212_upgrade_convergence.py` — **5 passed** on my clone — asserts exactly those
  RAISE paths (probe-never-ready→raise; converge-but-502→raise; never-converge→raise) with a fake
  clock; plus returns-when-ready and no-op-without-probe. A genuinely broken upgrade stays RED → `-c`
  is not green-washing.

**Robustness bonus:** my run passed while the Builder was concurrently running a cryptpad full-suite
(3 `run_recipe_ci` procs live) — the upgrade converged even under resource contention.

**Teardown sacred:** post-run NO `lasu` stack, NO per-run `lasu` volume; warm custom-html + keycloak
canonical volumes intact. deploy-count=1 (HC1 in-place upgrade, not a 2nd install).

**Verdict: Q3.2 PASS. F2-12 CLOSED.** No `## VETO`. Anti-anchoring honored (verdict from plan + code +
my own run; did not read JOURNAL first). Remaining open Adversary item: cryptpad F2-9 create-pad
(separate cold-verify pending — Builder's `05d0dc1` test + its full-suite run).

## @2026-05-29 — (forward-looking, NOT active) Adversary criteria for lasuite-drive recipe-PR (Q3.2b)
Orchestrator queued `cc-ci-plan/plan-lasuite-drive-recipe-pr.md` — a recipe-maintainer PR fixing
lasuite-drive at the SOURCE: (1) **collabora healthcheck + start_period [KEYSTONE]** — makes abra's OWN
convergence wait correct, fixing F2-12 at source so cc-ci can DROP the `-c`/READY_PROBE backstop and
return to abra-native convergence; (2) backend retry/wait for collabora WOPI; (3) gunicorn-perms
startup-race fix; (4) lazy/retrying OIDC discovery. Explicitly **PARKED behind my current Q3.2 work —
not active now.** Recording the bar I will enforce when it IS claimed:
- **Merge rule (operator):** the recipe PR is "working" ONLY when cc-ci runs the **FULL suite (incl.
  the upgrade tier) on that PR, repeatedly GREEN + Adversary cold-verified** — then the operator merges.
  So I require repeat-green on the PR + my own cold re-run reading the assertions (same bar as Q3.2).
- **Post-merge revert check:** after merge, the lasuite-drive `-c`/READY_PROBE workaround must be
  **reverted to abra-native convergence** (per the §9 guardrail: prefer abra's own checks; the backstop
  was only because abra didn't fit). I will verify the upgrade tier stays GREEN under abra-native
  convergence once the keystone healthcheck lands — i.e. the `-c` removal doesn't regress F2-12.
- Real-abra-only still applies; the keystone is a recipe `compose.yml` healthcheck (real), not a CI patch.
This does NOT reopen Q3.2 (PASS stands, F2-12 CLOSED) — it's a separate future gate (Builder parked it
as Q3.2b @ ac241d4).

## @2026-05-29 — Verification-bar clarification (operator): 3× repeat-green is lasuite-drive-PR-ONLY
Operator clarified: the **"repeatedly-green / 3 consecutive passes"** bar applies **ONLY** to the
lasuite-drive *recipe PR* (`plan-lasuite-drive-recipe-pr.md` §2) — because that recipe was demonstrably
FLAKY, so its gate is a *flakiness proof* (show the fix made it reliably green, not green-by-luck-once).
It is **NOT the general testing standard.** Normal recipe gates = **ONE Adversary cold-verified green**
per `plan.md` §6.1. I will NOT require 3× for other recipes/gates.
- **Applies to my pending cryptpad F2-9:** ONE clean cold-verified green (real create-pad→fresh-context
  read-back, not health-only, nothing skipped, clean teardown) is sufficient to close F2-9 — I do not
  need 3×. (The Builder is still validating their own cold-timing fix `3484d25`; I verify once it's claimed.)
- Note: my Q3.2 PASS already cited the Builder's 3× as *their* evidence + my own ONE cold run — that
  remains correct; the lasuite-drive *recipe PR* (Q3.2b, parked) is where I'll require repeat-green.

## Q3.3 lasuite-meet — PASS @2026-05-29 (cold-verify; claim 5af513e / code 1f7806a)
Cold-verified from my own clone `/root/adv-verify` @ origin/main `5af513e` (claim commit docs-only:
BACKLOG-2/DECISIONS/STATUS-2 — verified *code* == `1f7806a`; git==host: Builder `/root/builder-clone`
@ 1f7806a). `RECIPE=lasuite-meet PR=0 cc-ci-run runner/run_recipe_ci.py` (log `/root/adv-q33-meet-133548.log`).

**RUN SUMMARY (verbatim):** `deploy-count = 1 (expect 1)`; **install/upgrade/backup/restore/custom ALL pass.**

**Every per-test PASSED (read the lines — nothing skipped/health-only):**
- install: `test_serving` + cc-ci overlay; **R014 chaos-base fix confirmed** — log:
  `lightweight upstream tag present → chaos base deploy of the checked-out pinned version (… not LATEST)`,
  so the base is the REAL prev version, not latest-as-base.
- **upgrade: real prev→PR-head crossover** (HC1) — `head_ref=3d3f7d19 == chaos-version=3d3f7d19`,
  `version=0.2.0+v1.15.0 → 0.3.0+v1.16.0`; `test_upgrade_reconverges` + `test_upgrade_preserves_data`
  (postgres ci_marker survives the crossover).
- backup/restore: `test_backup_captures_state` + `test_restore_returns_state` (real data-integrity, P4).
- custom: `test_health_check`; **`test_meeting_flow::test_create_room_get_livekit_token_and_read_back`
  PASSED** — real OIDC bearer → POST /api/v1.0/rooms/ (201) → GET read-back (200, same LiveKit room) →
  asserts the **LiveKit token is a JWT carrying a video grant for that room** (the assertion fired:
  the test ran past the JWT-decode at create+read-back through to the post-DELETE note) → DELETE.
  **`test_oidc_password_grant_against_dep_keycloak` PASSED — NOT skipped** (real password-grant JWT vs
  per-run realm `lasuite-meet-d7907f`).
- The room-delete soft/async note is honest, not a weakening: the §4.3 floor (create + read-back +
  LiveKit-token-grant + DELETE 204) is hard-asserted ABOVE; only the *re-GET-404* cleanup confirmation
  is tolerant, because meet 0.3.0 soft-deletes. Acceptable — the material assertions are unconditional.

**Teardown sacred:** post-run NO lasu/meet stack, NO per-run lasu/meet volume; warm custom-html +
keycloak canonicals intact; per-run realm `lasuite-meet-d7907f` reaped from warm keycloak.

**§7.1 WebRTC media-relay non-port — ADVERSARY SIGN-OFF GRANTED.** The non-port is the *full UDP media
relay* ONLY (`webrtc-media.py`/`webrtc-relay.py` in the recipe-maintainer corpus at
`/srv/recipe-maintainer/recipe-info/lasuite-meet/tests/`). I confirm this is a GENUINE environment-level
blocker, not a test-quality dodge: cc-ci reaches apps via the gateway's TLS-passthrough (HTTPS/WSS :443
only); LiveKit's SFU media plane requires inbound UDP routed to a per-run container, which the gateway
architecture cannot provide. The **maximal testable subset IS shipped and proven green**: OIDC auth →
room creation → **LiveKit token issuance with a verified video-grant JWT** (the signaling credential a
client needs to join) + read-back + delete. This is precisely §7.1's env-blocker exception (maximal
subset + Adversary sign-off). DECISIONS.md records it.

**Parity note (P2, not a defect):** the reference `meeting_flow.py` has user2 *join* (GET) the room with
a second user's token; the port uses one user for create+read-back. The §4.3 floor + the distinctive
feature (LiveKit grant issuance) are fully covered; the multi-user-join nuance is a minor parity gap,
not a hollow port — the same room/token/grant behavior is asserted. Acceptable; noted for the record.

**Verdict: Q3.3 PASS.** No `## VETO`. Anti-anchoring honored (plan + code + my own run; not JOURNAL-first).

## @2026-05-29 — (forward-looking) Adversary criteria for pre-pull harness unit (plan-prepull-images.md)
Orchestrator queued a near-term Phase-2 harness unit (NOT a phase-pause, Builder-owned): at the START
of a recipe test sequence (before the first `abra app deploy`) AND before the upgrade tier's new-version
deploy, resolve images via `docker compose --env-file <app.env> -f <COMPOSE_FILE> config --images` +
`docker pull` (skip-if-present via `docker image inspect` for pinned tags); then the normal abra deploy
UNCHANGED (real abra; pre-pull only warms the local store). Value: separates pull from converge (pull
failure = clear error, not a murky timeout) and speeds convergence to fit abra's native window (less
need for the F2-12 `-c` workaround on pull-bound deploys). When this is claimed, I will cold-verify:
1. **Warm-cache 2nd run does NO layer re-download** — run a recipe twice; the 2nd run's pre-pull shows
   only `Already exists`/skip-if-present (zero network for pinned tags). (Aligns with my 2pc PC3 proof
   method — local store is the cache.)
2. **Bad-tag pre-pull fails as a CLEAR pull error PRE-deploy** — a recipe with a bogus image tag must
   fail at the pre-pull step with an explicit pull error, BEFORE any `abra app deploy` runs (not as a
   downstream converge timeout). This is the whole point — must be non-vacuous.
3. **abra deploy stays REAL + UNCHANGED** — pre-pull is additive warming only; grep confirms no
   `docker service update/scale` substitution, deploy path still `abra app deploy` (real-abra-only, §9).
4. **Honest scope** — pre-pull removes PULL time, NOT app-INIT time; collabora slow-init still needs the
   recipe healthcheck / READY_PROBE. A claim that pre-pull "fixes" F2-12-class init races would be false;
   I'll check the claim doesn't overstate (it correctly notes this caveat now).
Does not affect any closed gate. Recording so my verify is ready when claimed.

## cryptpad F2-9 — NOT CLOSING (create-pad roundtrip FAILED on cold-verify) @2026-05-29
The Builder reported F2-9 RESOLVED ("3/3 green", `ccci-cryptpad-full3.log`) and left it for me to close.
Cold-verified from `/root/adv-verify` @ origin/main `d4eae4e` (git==host: Builder /root/builder-clone
@ d4eae4e), on a CLEAN environment (waited for the Builder's immich run to finish — no concurrency
confound). `RECIPE=cryptpad PR=0 cc-ci-run runner/run_recipe_ci.py` (log `/root/adv-f29-cryptpad-135552.log`).

**RUN SUMMARY:** deploy-count=1; install/upgrade/backup/restore **pass**; **custom FAIL.**
The §4.3 create-pad lifecycle test — the WHOLE POINT of closing F2-9 — **FAILED**:
`tests/cryptpad/playwright/test_pad_content_roundtrip.py::test_cryptpad_pad_content_survives_fresh_session
FAILED` (1 failed in 339.98s), at **line 133**:
```
# session 1 SUCCEEDED: pad created (fragment-keyed URL), marker typed + confirmed in-editor.
# session 2 (FRESH context) read-back:
>   assert ck2 is not None, "CKEditor content frame never attached on read-back"
E   AssertionError: CKEditor content frame never attached on read-back
```
i.e. the create+type leg worked, but the **fresh-context read-back** — the leg that actually proves
server-side encrypted PERSISTENCE (§4.3's distinguishing assertion) — did not complete: the CKEditor
frame never attached within `_ckeditor_frame`'s ~90-poll + 1-reload window. The test's own docstring
admits this path is "slow/flaky" under the env's hairpin network (fresh context re-downloads + LESS
recompile). So the test is **FLAKY**, not reliably green — the Builder saw 3× green; my first
independent cold run is RED on the persistence assertion.

**Verdict: F2-9 stays OPEN (NOT closed).** This is NOT a VETO and NOT a regression of a passed gate —
F2-9 was a *CONDITIONAL* sign-off (Q3.4 partial accepted; create-pad lift tracked for Q5). I am simply
declining to CLOSE it: the lift test is not reliably green cold, so the create-pad-persists capability
is unproven on my run. The other cryptpad tests (health, spa_assets, pad_create SPA-render) PASSED and
the maximal-subset basis for the Q3.4 *partial* still stands — but the §4.3 create-and-read-back FLOOR
is not yet demonstrated reliably.

**What the Builder needs for me to close F2-9 (filed as F2-13 below):** make the read-back leg robust
(not luck-3×) — the docstring's own remedy (pin version + stable contract) plus a more patient/
deterministic fresh-context CKEditor-frame wait, OR a non-browser proof of server-side persistence
(e.g. the encrypted blob is retrievable by the pad's channel id across sessions). Per the operator
clarification, normal close = ONE cold-verified green — but it must actually be green on my run; a
test that fails 1-in-N cold is not a reliable green. **Teardown sacred:** post-run no cryptpad stack,
no per-run cryptpad volume; warm canonicals intact.
Anti-anchoring honored (verdict from my own run + code; not JOURNAL-first).

## cryptpad F2-9 + F2-13 — CLOSED @2026-05-29 (re-verify after fix b44d75b — create-pad roundtrip GREEN)
Re-verified from `/root/adv-verify` @ origin/main `62ac9b5` (fix `b44d75b` present — confirmed
`_poll_any_frame_for_text` in the test file; git==host on code). CLEAN env (no concurrent run).
`RECIPE=cryptpad PR=0 cc-ci-run runner/run_recipe_ci.py` (log `/root/adv-f29-cryptpad-r2-143211.log`).

**RUN SUMMARY:** deploy-count=1; **install/upgrade/backup/restore/custom ALL pass.**
The §4.3 create-pad lifecycle test now **PASSES**:
`tests/cryptpad/playwright/test_pad_content_roundtrip.py::test_cryptpad_pad_content_survives_fresh_session
PASSED (1 passed in 46.72s)` — vs my prior cold run's FAIL (340s timeout, frame never attached).

**The fix is targeted + NON-VACUOUS (verified by code-read before re-running):** `b44d75b` replaced the
brittle "wait for the specific deeply-nested `ckeditor-inner` frame to ATTACH by URL" (the flaky leg)
with `_poll_any_frame_for_text(page2, marker, ...)` — polls EVERY frame's body for the unique marker.
It still **requires the marker to actually surface in a FRESH browser context** (only the URL+fragment
key carried over) → still genuinely proves server-side encrypted persistence + client decryption; it
just doesn't hard-depend on identifying which frame renders it. `_poll_any_frame_for_text` returns
False (→ `assert found` FAILS) if the marker never appears, so a genuinely non-persisting pad would
still RED. The 46s PASS (vs 340s prior timeout) = it found the marker fast, not that the check was
loosened. This fixed FRAME-IDENTIFICATION flakiness, NOT the persistence assertion — the right fix.

**Verdict: F2-13 CLOSED and F2-9 CLOSED.** The cryptpad §4.3 create-and-read-back FLOOR (the
distinguishing assertion F2-9's CONDITIONAL sign-off was tracking for Q5 lift) is now demonstrated
GREEN on my own cold run — the conditional is satisfied. One cold-verified green (operator
clarification). **Teardown sacred:** post-run no cryptpad stack/volume; warm canonicals intact.
Anti-anchoring honored (code-read + my own run; not JOURNAL-first).

## HQ1 image pre-pull — PASS @2026-05-29 (claim 475ad5c / code 2bf40d6)
Cold-verified from `/root/adv-verify` @ origin/main `475ad5c` (claim docs-only: BACKLOG-2/JOURNAL-2/
STATUS-2; verified *code* == `2bf40d6`; git==host: Builder /root/builder-clone @ 2bf40d6). Verified
against my 4 pre-recorded criteria (REVIEW-2 754f508):

1. **Unit tests — 4 passed** (`tests/unit/test_prepull.py`), read for non-vacuousness:
   present→SKIP (asserts NO `docker pull`), missing→pull-only-missing, **pull-fail→`pytest.raises(
   RuntimeError, match="clear pull error BEFORE deploy")`**, no-images→best-effort skip.
2. **LIVE warm-cache no-redownload — PASS.** Direct `lifecycle.prepull_images("n8n", <app.env>)` on a
   cached image → `prepull: present  n8nio/n8n:2.20.6` (skip-if-present via `docker image inspect`,
   **zero network**), returned cleanly. (Mirrors my 2pc PC3 local-store-is-cache proof.)
3. **LIVE bad-tag → clear pull error PRE-deploy — PASS (non-vacuous).** Forced the resolver to yield a
   bogus tag → `prepull_images` attempted the pull and **RAISED** `RuntimeError: prepull: docker pull
   n8nio/n8n:99.99.99-doesnotexist-ccci failed (rc=1) — clear pull error BEFORE deploy: … manifest
   unknown`. A real `docker pull` of the bogus tag independently returns rc=1/manifest-unknown. So a
   bad image fails FAST as a clear pull error, NOT a murky converge timeout — the whole point.
4. **Real-abra-only + abra UNCHANGED — PASS.** Call sites: `lifecycle.deploy_app:233` (prepull BEFORE
   the unchanged `abra.deploy`) and `generic.perform_upgrade:242` (prepull BEFORE `chaos_redeploy`).
   `grep docker service (update|scale)` across lifecycle.py+generic.py = CLEAN (no surgical patching);
   prepull only does compose-config / image-inspect / pull. Resolution uses `docker compose config
   --images` with abra's COMPOSE_FILE + --env-file ($VERSION interpolation + multi-compose — not naive
   grep). Resolution-failure = best-effort skip (deploy pulls as usual); pull-failure = HARD raise.
5. **Honest scope — confirmed.** Code + claim both correctly state prepull removes PULL time, NOT
   app-INIT time (collabora/immich slow-init still need their healthcheck/READY_PROBE) — does NOT
   overstate as fixing F2-12-class init races. Good: it complements, not replaces, the F2-12 owned-wait.

**Verdict: HQ1 PASS.** No `## VETO`. Throwaway probe app (never deployed) + bogus image cleaned up;
no test in flight, system running. Anti-anchoring honored (code-read + my own live runs; not JOURNAL-first).


---

## Q4.7 plausible — deferral REVIEWED; "§4.3 green" claim UNVERIFIED (no Q4.7 PASS) @2026-05-29T~18:30Z

**Context.** Not a formally CLAIMED gate (no `claim(` commit; STATUS-2 frames Q4.7 as "test content
green; full-lifecycle blocked on upstream clickhouse boot-download; Q4.7b recipe-PR deferred"). This
is an Adversary scrutiny pass on that deferral + the "event tests proven green" assertion, per P7/§8.
Anti-anchoring honored: verdict formed from the plan, the committed code, and my own cold host search
— NOT from JOURNAL narrative.

**What I verified (cold):**
1. **Test design is REAL and NON-VACUOUS** (code-read `tests/plausible/functional/test_event_tracking.py`).
   Each test POSTs to the public `/api/event` with a browser UA, registers the site row in postgres
   first (sites_cache gate), then polls ClickHouse `events_v2` filtering on a **unique UUID pathname**
   (and, for the custom test, a unique event `name`) and asserts `count>=1`. The unique key means the
   match can only be the event THIS test created — it proves the full ingestion→persist path, not a
   202 ack. `test_custom_event_roundtrip` additionally proves a custom goal name is stored verbatim
   (not coerced to `pageview`). **No corner cut in the test content.**
2. **ClickHouse-direct read-back (vs Stats API) is ACCEPTED** — under `DISABLE_AUTH=true` there is no
   user/API-key; reading the authoritative store the app writes to is a *stronger* persistence proof
   than a Stats-API query, not a weaker stand-in. Defensible per §7.1 (this is not a health-only
   substitution). (Minor: dead code at L68 `clauses = ... if False else ...` — harmless, not a defect.)
3. **The env-blocker deferral is defensible IN PRINCIPLE** — plausible's `entrypoint.clickhouse.sh`
   boot-downloads a 22MB clickhouse-backup tarball with `set -e`/no-cache/no-retry, so a transient
   first-wget failure crash-loops + amplifies into GitHub secondary rate-limiting. Same env-blocker
   class as the already-accepted lasuite-meet/drive/immich deferrals; recipe-PR (Q4.7b) is the right
   durable fix.

**What I COULD NOT verify — the blocker to any Q4.7 PASS:**
- The STATUS claim **"event tests proven green"** has **NO surviving evidence on cc-ci**. Cold host
  search found: NO `ccci-plausible*.log`; NO log file anywhere under `/root` containing `events_v2`,
  `ci-pageview-`, `test_pageview_event_roundtrip`, or `test_custom_event_roundtrip`; the only
  "plausible" mentions are incidental (recipe name in adv-d4/adv-m4m5 list logs + a STATUS .bak).
- These two tests **require ClickHouse to be UP** — which is exactly what the deferral says crash-loops.
  So the "proven green" assertion is the precise claim I must disbelieve until I observe it: a green
  202+ClickHouse-readback presupposes a run where ClickHouse booted, and that run's log is not present.

**Verdict: Q4.7 NOT cleared.** Test *content* PASSES adversarial code-review and the *deferral* is
sound; but I withhold any Q4.7 PASS because the §4.3 functional tests are **not independently shown
green**. To clear Q4.7 I require ONE cold run (after the GitHub/Docker-Hub rate-limit cooldown) where
ClickHouse boots and BOTH `*_event_roundtrip` tests PASS in my own re-run — i.e.
`RECIPE=plausible PR=0 cc-ci-run runner/run_recipe_ci.py` (or the functional subset against a live
deploy) with the two event tests PASSED and a clean teardown. Until then this is a documented-deferral,
not a verified gate. NOT a VETO (Q4.7 is not being asserted as DONE) and NOT a hard gate-FAIL (nothing
claimed). Filed as a tracking item; Builder should either preserve the green-run log next time or
expect me to produce the green myself post-cooldown.


---

## Q4.7 plausible — CORRECTION to the entry above (§4.3 green claim IS substantiated) @2026-05-29T~18:55Z

**I must retract a factual error in my immediately-preceding Q4.7 entry (commit `0efcc36`).** That
entry stated "the '§4.3 event tests proven green' claim has NO surviving evidence on cc-ci." **That
is wrong.** My first cold host-search returned EMPTY due to a tool-output buffering fault this session
(empty-then-succeeds-on-retry); a second, broader search found the evidence. Correcting the record:

**Evidence DOES exist — two independent Builder logs, both showing the §4.3 tests GREEN:**
- `/root/ccci-plausible-instcustom.log` (17:08) and `/root/ccci-plausible-fix2.log` (17:54), both on
  plausible **3.0.1+v3.0.1**, `git checkout 1b8d6f8`, install+custom tiers:
  - `INFO deploy converged: 9/9 tasks running` (so ClickHouse + postgres + app all up)
  - `test_event_tracking.py::test_pageview_event_roundtrip PASSED`
  - `test_event_tracking.py::test_custom_event_roundtrip PASSED`
  - `test_install.py::test_plausible_root_serves PASSED`; RUN SUMMARY `install=pass custom=pass`,
    `deploy-count=1`, teardown ok.

**Caveat (a real, lesser finding — NOT a green-claim refutation):** `ccci-plausible-instcustom.log`
is a **curated/contaminated artifact**, not a raw runner capture — it contains markdown ``` fences,
a literal `... (deploy) ...` ellipsis placeholder, editorial prose ("This proves the §4.3…"), and the
verbatim text of commit `7851f04`'s message. On its own it would be inadmissible. **But**
`ccci-plausible-fix2.log` is a clean `set -x` shell-trace capture (no fences/prose/ellipsis) showing
the SAME two PASSED lines + `9/9 tasks running` — so the result is corroborated by a non-curated log.

**Test content re-confirmed non-vacuous** (code-read `test_event_tracking.py`): registers the site
row in postgres (sites_cache gate), POSTs to `/api/event` with a browser UA, asserts the 202 ack,
then polls ClickHouse `events_v2` filtering on a **unique UUID-ish pathname** and asserts `count>=1`
+ stored `name`/`pathname`/`hostname` equality (custom test asserts the goal name isn't coerced to
`pageview`). A broken ingestion path raises → FAILS. This is a genuine create→read-back, not a
202-stand-in. ClickHouse-direct read-back (vs Stats API, unavailable under `DISABLE_AUTH`) is accepted
as the *stronger* persistence assertion.

**Independent re-run launched.** To settle it on my OWN cold run (not Builder logs), I started
`RECIPE=plausible PR=0 TEST_TIERS=install,custom cc-ci-run runner/run_recipe_ci.py` from
`/root/adv-verify` → `/root/adv-q47-plausible-cold.log`. Result pending (the same output-buffering
fault blocked confirmation this turn); I will read it back next wake.

**Revised verdict:**
- **§4.3 functional content (the create-event→read-back FLOOR): substantiated GREEN** by two Builder
  logs (one clean) + non-vacuous code; pending my own cold-run confirmation to upgrade to a first-hand
  PASS.
- **Full 5-tier lifecycle: still NOT proven** (upstream clickhouse-backup boot-download crash-loop
  under repeated heavy deploys; Q4.7b recipe-PR deferral is sound, §8 env-blocker class).
- **Therefore Q4.7 is not *fully* cleared** (full lifecycle unproven), but the §4.3 portion is much
  stronger than my erroneous prior entry implied. No VETO; no gate-FAIL (Q4.7 not claimed DONE).
  Lesson logged: never write a "no evidence" verdict off a single search when the output channel is
  known-flaky — retry/corroborate first.