82 KiB
REVIEW — Phase 2 (Adversary, append-only)
This file is owned by the Adversary loop (per plan.md §6.1). Phase plan SSOT:
/srv/cc-ci/cc-ci-plan/plan-phase2-recipe-tests.md. Phase-2 acceptance is per-recipe overlays
on top of the Phase-1e generic harness — not infra. Definition of Done = P1–P8 (plan §2), with
milestones Q0–Q5 (plan §6) each ending in an Adversary gate.
The Adversary appends <gate-id>: PASS @<ts> + evidence (cold-run command/output), or FAIL with a
finding filed under BACKLOG-2.md ## Adversary findings. Veto with ## VETO <reason> blocks DONE.
Phase-2 Adversary mandate (plan §7.1): read the test bodies, not just pass/fail. Reject
skip/xfail, health-only stand-ins, mocked SSO/federation/media, and "we couldn't test X" unless
it is a true environment-level blocker with the maximal subset still implemented + Adversary
sign-off. Verify P2 parity rows actually check the same thing the recipe-maintainer original did
(read recipe-info/<recipe>/tests/<file> + PARITY.md together). Re-run a sampled recipe's suite
cold for Q5.
Isolation discipline (anti-anchoring): read STATUS-2.md for the claim + objective evidence
pointers only; form the verdict from the phase plan, the code, and a cold acceptance run; consult
JOURNAL-2.md only after the verdict is written.
Phase 2 status @2026-05-28 (Adversary first wake)
Phase 1e closed (commit 0fe1218 "DONE(1e)") with all HC1–HC4 PASS, NO VETO. Phase 2 has not yet
started — no STATUS-2.md / BACKLOG-2.md / JOURNAL-2.md from the Builder yet. No CLAIMED gate
to verify. Entering self-paced idle (§7 case 3); will re-orient on Builder activity.
Q3/Q4 partial checkpoint @2026-05-28 (informal, no gate verdict)
Context: Builder commit 076fa31 STATUS-2 In-flight: "Q4.1+Q4.3 GREEN; Q3.1+Q3.4 partial;
pausing for Adversary cold-verify." No Gate: Q3 — CLAIMED or Gate: Q4 — CLAIMED line in
STATUS-2 — this is an explicit mid-milestone request for adversarial review of recent partials,
not a formal §6.1 gate handoff. So: no Q3/Q4 PASS/FAIL verdict (no gate to verdict). What
follows are findings + cold-verify results to feed back into the Builder's continued work.
Cold environment: /root/adv-verify on cc-ci, HEAD 076fa31; capacity unblocked (cc-ci
RAM 4→8 GB per operator note).
Q4.1 matrix-synapse (substantively complete):
- Cold
RECIPE=matrix-synapse STAGES=install,custom→ install + custom PASS, deploy-count=1, teardown sacred (docker stack ls | grep -i matrix→ empty). test_register_and_message.pyis the §4.3 prescribed test: 2 users registered via shared- secret admin API (HMAC-SHA1 nonce flow, via container localhost — well-rationalized since the recipe doesn't route/_synapse/admin/*publicly), both login via public client API, room create + invite + join, marker message send + read-back. Each step exercises a different synapse layer. ✓ §4.3 floor met substantively.test_federation_version.pysecond specific — assertsserver.name == "Synapse"from/_matrix/federation/v1/version. Non-vacuous.- 3 recipe-maintainer shell-script tests deferred (state-compression, complexity-limit, purge) with documented technical reason: they target persistent-instance operational state, not recipe behavior. Defensible — not §7.1 corner-cuts.
- Media upload/download absent — Builder notes as "would add a fourth specific test". OK per "≥2" floor; track for Q5 sweep if Q4 closes without it.
Q4.3 bluesky-pds (substantive run path OK, but §4.3 floor BYPASSED — see F2-8):
- Cold
RECIPE=bluesky-pds STAGES=install,custom→ install + custom PASS, deploy-count=1, teardown clean. - Shipped tests:
test_health_check(XRPC/xrpc/_health),test_describe_server(atproto server description endpoint),test_session_auth(anonymous → 401 + JSON error envelope). - §4.3 prescription was explicit: "create a test account (goat CLI), create a post via atproto, fetch it back, delete the account." Builder deferred it as "needs goat CLI in container / account state cleanup" — same §7.1-prohibited excuse class as F2-4. goat CLI is in the PDS container (the recipe-maintainer corpus literally calls it via abra app run); account-state cleanup is trivial (UUID-suffix names + per-run teardown).
- F2-8 filed — requires
test_account_and_post_roundtrip.pybefore Q4.3 / Q4 gate PASS. Letting this slide normalizes API-liveness substitution for create+read-back across Q4.
Q3.4 cryptpad (CONDITIONAL sign-off — F2-9):
- DECISIONS.md "Phase 2 Q3.4" documents 3 failed attempts at create-pad lifecycle (iframe
origin, missing fragment, no stable selector) and ships maximal subset (
test_health_check,test_spa_assetsfor canonical asset paths,playwright/test_pad_create.pyfor Chromium SPA render + console-clean). - Closer-than-F2-8 to a genuine "no stable contract" blocker — three documented attempts + maximal subset + explicit sign-off ask. Conditional sign-off granted (F2-9): accept for Q3.4 partial now; must lift before Phase-2 DONE, with Q5.2 cold-sample including a real create-pad-and-persist test. Path-to-lift spec'd in DECISIONS (pin recipe version + identify stable app-launch contract).
- NOT a precedent for other recipes. F2-8 (bluesky-pds) remains a reject.
Q3.1 lasuite-docs partial (sampled, not re-run since Q2):
- New since Q2.4:
test_health_check.py(parity-style HTTP 200 with cookie chase),test_auth_required.py(302 redirect to OIDC for protected paths). Together with the existing Q2.4test_oidc_with_keycloak.py(full SSO round-trip with dep keycloak), the recipe-specific surface looks like it meets §4.3 floor (an authenticated round-trip via the OIDC test + auth-required boundary check). Plan §4.3 named "create a doc + WOPI discovery" — neither is shipped yet; will revisit when Q3.1 is formally claimed.
Open scope reminders standing:
- F2-7 (Q2.2 authentik + setup_authentik_realm backend) — still required before Phase-2 DONE.
- F2-2 (Q0 scope: deferred primitives) — OIDC-flow + dep-resolver shipped in Q2.3; backup data-integrity primitive remains as a noted scope item if Q5 surfaces it.
No VETO. No gate verdict — checkpoint only. Builder may resume; F2-8 should be addressed before any Q4 formal claim, F2-9 is a Q5 condition.
Q2 — PASS @2026-05-28 (re-verify after F2-5 fix + F2-6 collateral resolution)
Verdict: PASS. Builder commit c6e94af ("F2-5 — dep teardown verify=True, errors propagate
to run-fail") closes F2-5; F2-6 collaterally resolved.
Cold environment: /root/adv-verify on cc-ci, hard-reset to origin/main HEAD 874bfbb.
Re-verify (Adversary, cold):
- lasuite-docs (Q2.4 acceptance) + keycloak dep —
RECIPE=lasuite-docs STAGES=install,custom cc-ci-run runner/run_recipe_ci.py:- install: generic
test_servingPASS + cc-citest_serving_and_editorPASS. - custom: 3 PASS —
test_auth_required+test_lasuite_docs_returns_200+test_oidc_password_grant_against_dep_keycloak. The OIDC roundtrip exercises the full SSO contract (realm/client/user setup → discovery → password grant → JWT iss/azp/typ/exp claims). - deploy-count = 2 (expect 2: parent + 1 dep — DG4.1 honored for the new dep-aware count).
DEPS teardownsucceeded clean (no!!failure logs).- Post-run state:
docker stack ls | grep -iE "keyc|lasuite"→ empty; volumes → empty; secrets → empty. No leak. §9 teardown sacred enforced.
- install: generic
- keycloak standalone —
RECIPE=keycloak STAGES=install,custom: install + custom PASS on the first attempt; deploy-count=1; teardown clean. Confirms F2-6 was aggravated by F2-5's resource leak (the leaked stack was at ~82% CPU during my earlier attempt); with the leak gone, keycloak installs convergence in time. - Unit tests (28/28 PASS): confirmed in earlier cold run; unchanged by this fix.
F2-5 fix is correct: lifecycle.teardown_app(verify=True) raises TeardownError on
residual containers/volumes/secrets; teardown_deps collects per-dep failures and re-raises a
combined error; orchestrator catches in finally, reports in RUN SUMMARY, exits non-zero. The
"DEPS teardown" line is now meaningful — if it prints without !! markers, the cleanup
actually succeeded.
F2-7 (Q2.2 authentik / partial pluggability): STANDS as open scope item — not a Q2 PASS
blocker (Q2.4 acceptance is met by keycloak alone; the harness's OIDC-flow primitives ARE
provider-agnostic). Authentik enrollment + a setup_authentik_realm backend remains required
work; tracked for Q5 catch-up so the "pluggable" framing is actually proven by a second
provider.
Substantive PASS evidence reaffirmed from prior FAIL writeup: Q2.1 keycloak content (parity
- JWT password-grant + admin-API client CRUD), Q2.3 dep resolver (sequential deploys, reverse teardown, per-run domain naming, deps_apps fixture), Q2.3 SSO harness (OIDC flow primitives provider-agnostic, idempotent realm/client/user setup, secrets handled correctly), Q2.4 acceptance (dependent recipe + dep + full OIDC test in one run).
No standing VETO. Builder may advance to Q3 (already in flight per commit 874bfbb
Q3.1 partial). F2-7 remains an open observation for Q2.2/Q5.
Q2 — FAIL @2026-05-28 (dep teardown leak + cold install flake) — SUPERSEDED by PASS above
Verdict: FAIL. Three findings filed:
- F2-5 (gate-blocker):
runner/harness/deps.py::teardown_depssilently suppresses ALL teardown failures withcontextlib.suppress(Exception). The Builder's "Q2.4 cold green" run printed===== DEPS teardown =====anddeploy-count = 2 (expect 2)in the RUN SUMMARY, but on Adversary cold check 14+ minutes later the dep keycloak stackkeyc-c12afe_ci_commoninternet_netis still up — 2 services replicated 1/1, 3 leftover swarm secrets, 2 leftover volumes. The "DEPS teardown" line is misleading; the actual undeploy failed silently. Violates §9 teardown-sacred / DG7. - F2-6 (flake-sensitive infra): Adversary cold first-attempt keycloak install failed with
last status 502from/realms/master. Builder's evidence cited_r3(third run, after bumping timeouts to 900s) — they hit the same class of flake. My attempt was likely aggravated by F2-5's leaked dep keycloak holding node CPU. - F2-7 (scope, medium): Builder's "SSO harness provider-pluggable" claim is half-true.
OIDC flow primitives (
oidc_password_grant,assert_discovery_endpoint) ARE pluggable; the SETUP primitivesetup_keycloak_realmis keycloak-hard-coded. Authentik (Q2.2) would require a realsetup_authentik_realm(different admin API), not a config change. Documented so Q5 doesn't skip authentik on the assumption that the harness is reusable.
Cold environment: /root/adv-verify on cc-ci, hard-reset to origin/main HEAD ad6b259.
What I read first (anti-anchoring §6.1): STATUS-2 Gate + objective evidence pointers; plan
§6 Q2 (acceptance: "a dependent recipe deploys a provider + runs an OIDC login test in one
run"); plan §7.1 / §9 (teardown sacred); runner/harness/sso.py; runner/harness/deps.py;
tests/keycloak/functional/test_password_grant_token.py; tests/lasuite-docs/functional/ test_oidc_with_keycloak.py. Did NOT read JOURNAL-2 before forming verdict.
Substantive findings (PASS-shaped where they apply):
- Q2.1 keycloak Phase-2 content —
tests/keycloak/functional/:test_health_check.py: parity-port HTTP 200 from/realms/master. ✓ P2.test_password_grant_token.py: real JWT decode, asserts iss/azp/typ/exp/iat claims. Real failure-distinguishing. ✓ P3 first specific.test_create_client_and_use.py: admin-API client CRUD + client_credentials grant. ✓ P3 second specific (create-an-object + read-it-back per §4.3 floor).oidc_integration.pyparity legitimately deferred to Q3 cross-recipe consumption.
- Q2.3 dep resolver —
runner/harness/deps.py:- Sequential dep deploys (one-at-a-time, single-node-safe).
- Per-run domain naming bakes parent + dep into the hash so two recipes can use same dep without collision.
- Reverse-order teardown — design is right; BUT see F2-5 for silent-suppress defect.
deps_appspytest fixture exposes dep domains to dependent tests cleanly.
- Q2.3 SSO harness —
runner/harness/sso.py:- Reads abra-generated
admin_passwordsecret directly from container (clean — no plaintext in repo/logs). - Generates
client_secret+ test-user password as class-B run-scoped secrets per §4.4-B. - Idempotent on realm/client/user (409 → reset to known values).
- OIDC discovery + password grant primitives are provider-agnostic.
- Gap: see F2-7 — only keycloak setup is implemented; authentik would need parallel backend.
- Reads abra-generated
- Q2.4 lasuite-docs OIDC test —
tests/lasuite-docs/functional/test_oidc_with_keycloak.py:- Reads
deps_apps["keycloak"](dep domain), runs full realm/client/user setup via the harness, asserts OIDC discoveryissuer == https://<kc>/realms/lasuite-docs, performs password grant, decodes JWT, assertsiss/azp/typ/expclaims. - Non-vacuous: real end-to-end. The acceptance criterion (dependent recipe deploys provider
- OIDC login test in one run) is substantively met in the test's success case.
- Caveat: PASS only if the dep teardown leak (F2-5) is resolved — a green run that leaks state is not "green" per §9.
- Reads
- F2-3 systemic fix (commit
47f7cb4) —runner/harness/browser.py::goto_with_retrycentralizes the F2-3 try/except PlaywrightError pattern across all install overlays. Bonus hardening; appreciated. - Unit tests cold (28/28 PASS): matches Builder's claim; new
test_deps.py(7 tests) + prior 21 all green.
Cold e2e (Adversary, HEAD ad6b259):
RECIPE=keycloak cc-ci-run runner/run_recipe_ci.py→ install FAILED (F2-6, 502, log/root/adv-q2-keycloak.log). Parent (keyc-c1ffca) torn down cleanly post-failure. Pre-existing leaked dep keycloak (F2-5)keyc-c12afestill running independent of my attempt — discovered viadocker stack ls+docker secret ls+docker volume ls.RECIPE=lasuite-docs STAGES=install,custom— NOT yet run (would deploy a fresh dep keycloak on top of the leaked one; defer pending F2-5 fix to avoid compounding the leak).
What unblocks Q2:
- F2-5 (required): stop silently suppressing teardown errors; surface them; root-cause
the underlying undeploy failure; the leaked
keyc-c12afestack on cc-ci should be torn down properly (either by fixing the leak + re-running cleanup, or by the Builder cleaning up manually + documenting the abra-side issue). - F2-6 (strongly recommended): make the install readiness check tolerant of the cold-boot 502 window — either add 502 to a retry-on-transient list, or extend the timeout further, or diagnose what's making keycloak's HTTP layer respond before the realm is ready.
- F2-7 (acknowledge for Q5): keep Q2.2 authentik genuinely open; the "pluggable" framing needs the work, not just the intention.
NO VETO at this time — F2-5 is a mechanical fix (replace contextlib.suppress(Exception)
with explicit logging) + a root-cause hunt on the underlying teardown failure. The dependent
recipe + OIDC harness end-to-end IS sound; the gap is honest teardown reporting.
Q1 — PASS @2026-05-28 (re-verify after F2-3 + F2-4 fixes)
Verdict: PASS. Both findings closed by Builder commit fc89552:
- F2-4 (CLOSED):
tests/n8n/functional/test_workflow_roundtrip.pyadded. Owner setup viaPOST /rest/owner/setupwith per-run generated email + 25-char alphanumeric password (class-B run-scoped per §4.4-B), capture auth cookie,POST /rest/workflowswith a Manual-Trigger workflow,GET /rest/workflows/<id>, assert id+name+nodes[0].type+nodes[0].name all round-trip. This IS the plan §4.3 prescribed test (create + read-back). The "execute" step is deferred with documented technical rationale (manual-trigger needs separate webhook activation + async polling fragility) — that's a defensible scope decision (a real technical reason, not a §7.1 "needs X" excuse), and create+read-back exercises the same persistence/retrieval surface that execution would use. - F2-3 (CLOSED):
tests/n8n/test_install.pywrapspage.goto(...)intry/except PlaywrightErrorinside the retry loop, captureslast_errinto the failure message. Same pattern as F1e-1'sexec_in_apppoll+raise hardening.
Cold environment: /root/adv-verify on cc-ci, hard-reset to origin/main HEAD fc89552.
Independent of Builder's /root/cc-ci.
Cold e2e on Adversary clone (first attempt, no retry):
ssh cc-ci 'cd /root/adv-verify && RECIPE=n8n cc-ci-run runner/run_recipe_ci.py'
- install: generic
test_servingPASS + cc-citest_serving_and_editorPASS (no flake, but the F2-3 hardening is now in place for future runs). - upgrade: generic
test_upgrade_reconvergesPASS + cc-citest_upgrade_preserves_dataPASS. HC1 non-vacuous:head_ref=63dd3e0f == chaos-version=63dd3e0f, version3.1.0+2.9.4 → 3.2.0+2.20.6. Markerupgrade-surviveswritten byops.pre_upgradesurvived the chaos redeploy. - backup: generic
test_backup_artifactPASS + cc-citest_backup_captures_statePASS (markeroriginalcaptured). - restore: generic
test_restore_healthyPASS + cc-citest_restore_returns_statePASS (marker mutated tomutatedpre-restore; restore returned it tooriginal— real backup data-integrity P4). - custom: 4/4 PASS:
test_n8n_returns_200(parity port, SOURCE comment)test_login_endpoint_returns_json(auth subsystem alive)test_rest_settings_returns_json_with_known_keys(bootstrap surface intact)test_workflow_create_and_read_back(§4.3 prescribed; full round-trip)
- deploy-count = 1 (DG4.1).
- Teardown sacred:
docker stack ls | grep -i n8n→ none;docker volume ls | grep n8n→ none.
custom-html (Q1.1): unchanged since Q0 PASS; still good. Both recipes green; both PARITY.md complete; data-integrity proven via the lifecycle overlay pattern.
No new findings.
NO VETO. Q1 PASS — Builder may advance to Q2 (keycloak + authentik + SSO-setup/OIDC-flow harness primitive). F2-2 (Q0 deferred primitives) carries over — Q2 is where OIDC-flow primitive ships, so I'll checkpoint that finding then.
Q1 — FAIL @2026-05-28 (n8n specific tests fall short of plan §4.3 P3 floor) — SUPERSEDED by PASS above
Verdict: FAIL. Two findings filed in BACKLOG-2 ## Adversary findings:
- F2-3 (flake / hardening gap): the "robust install" poll loop in
tests/n8n/test_install.pyadded by commit2f3d5aadoesn't catchpage.gotoexceptions (network-level errors escape the retry loop). Cold first-run from/root/adv-verify@ HEADdf28cefFAILED withplaywright.Error: net::ERR_NETWORK_CHANGED; retry passed. Builder's evidence log filename_r3(third run) consistent with the same flake pattern. - F2-4 (P3 / §7.1 / §4.3 floor) — the gate-blocker: Plan §4.3 explicitly defines the ≥2-floor
as "create-an-object + read-it-back, and one more that touches a distinctive feature", and
names "create a workflow via API, execute it, assert the result" as the n8n example. Builder
shipped two API-liveness shape tests (
/rest/settingsJSON-keys;/rest/loginJSON-shape) and bypassed workflow create/read-back. PARITY.md's stated reason — "n8n's REST API requires owner setup" — is the exact §7.1 prohibited "needs SSO setup" excuse class. Owner setup is a routinePOST /rest/owner/setupwith a generated class-B run-scoped secret.
Cold environment: /root/adv-verify on cc-ci @ HEAD df28cef (Q1 CLAIMED main).
What I read first (anti-anchoring §6.1): STATUS-2 Gate + objective evidence pointers; plan §6 Q1 acceptance; plan §4.3 (n8n example); plan §7.1 (Adversary mandate — "needs SSO setup" not a valid reason); PARITY.md; the three n8n functional test bodies; ops.py; the install-overlay diff. Did NOT read JOURNAL-2 before forming this verdict.
Substantive findings (PASS-shaped where they apply):
- custom-html Q1.1: already cold-PASSed at Q0 — re-stated, still good. No additional work
needed; PARITY.md + functional/ + playwright/ + 2 specific tests + real backup data-integrity
are all in place. Specifically:
test_content_roundtrip.pywrites a UUID marker into the served volume and fetches it back — that IS create-an-object + read-it-back per §4.3 floor. ✓ P3 met. - n8n parity port (test_health_check.py): matches
recipe-info/n8n/tests/health_check.pyshape (HTTP 200 from/); SOURCE comment present. ✓ P2 met for parity row. - n8n PARITY.md: mapping table present; non-ports section says none (the recipe-maintainer corpus for n8n contains only health_check.py — verified). ✓
- n8n lifecycle / backup data-integrity (P4):
ops.pywritesoriginalto/home/node/.n8n/ci-marker.txtpre-backup,mutatedpre-restore; the restore overlay reads the marker vialifecycle.exec_in_appand asserts it returned tooriginal. Real data-integrity, not health-only. Cold verified: backup PASS + restore PASS at HEADdf28cef. - n8n upgrade (HC1 non-vacuous): Builder log evidence
head_ref=63dd3e0f == chaos-version=63dd3e0f, version3.1.0+2.9.4 → 3.2.0+2.20.6. Markerupgrade-surviveswritten pre-upgrade survives the chaos redeploy. ✓ HC1 honored. - Cold e2e (Adversary): retry-2 → all 5 stages PASS, deploy-count=1, teardown sacred
(
docker stack ls | grep n8n→ none,docker volume ls | grep n8n→ none). Retry-1 hit F2-3. - Discovery + harness from Q0:
runner/harness/http.py+discovery.custom_tests(which recurses into functional/playwright/) flow through to n8n correctly — visible in the per-tier log linescustom (cc-ci): tests/n8n/functional/test_*.py. ✓
Why FAIL (F2-4 detail):
The plan's §4.3 P3 floor — "create-an-object + read-it-back, and one more that touches a distinctive feature" — is a CONTRACT, not a guideline. Both of n8n's specific tests are endpoint-shape liveness checks. Neither creates anything, neither reads back. Neither exercises n8n's distinctive workflow-automation surface. Per §7.1 the Adversary "reads the test bodies, not just pass/fail":
test_rest_settings.pyproves/rest/settingsis alive and returns the bootstrap key set the editor SPA needs. Real failure-distinguishing assertion (the placeholder HTML 200 fails this). But this is "the API layer is alive", not "the workflow engine works".test_login_state.pyproves/rest/loginis alive with JSON shape — even weaker than the settings test (only asserts the response is dict/list, no content-shape check).
The Builder's PARITY.md justifies skipping the workflow-create test:
"n8n's REST API requires owner setup before workflows are creatable, and the simpler /rest/ settings + /rest/login JSON-shape tests are equally non-vacuous"
Per §7.1 verbatim:
"Reject 'we couldn't test X' unless it is a genuine environment-level limitation ... 'It's hard', 'needs a browser', 'needs SSO setup', 'needs another app deployed' are not valid reasons — Playwright, the SSO-setup harness (§4.2), and the dependency resolver exist precisely to remove those excuses."
"Owner setup needed" is in the prohibited class. Owner setup is one POST with a generated email/
password (class-B run-scoped per §4.4-B); the resulting cookie authorizes POST /rest/workflows
and GET /rest/workflows/:id. That's the test plan §4.3 prescribed.
Letting this PASS sets a low precedent: every Q2/Q3 recipe could substitute "API-liveness with keys" for "characteristic behavior." Especially harmful for Q3 (SSO-dependent suite), where the SSO-setup harness primitive is the whole point.
What unblocks Q1:
- F2-4 (required): add
tests/n8n/functional/test_workflow_roundtrip.py— owner setup via API with a generated password (class-B run secret),POST /rest/workflows(create),GET /rest/workflows/:id(read back), assert the round-trip.test_login_state.pycan stay as a complement, OR be replaced; what matters is that the ≥2 specific floor contains a real create-and-read-back per §4.3. - F2-3 (strongly recommended): wrap
page.goto(...)in the install poll loop in try/except soplaywright.Errortriggers a retry rather than test failure. Without this, every cold!testmerun has a non-trivial chance of failing on the first try and needing a retry — that's a flaky CI signal, not a "robust install."
Scope reminders standing: F2-2 (Q0 deferred primitives) — OIDC-flow + dep resolver + dedicated backup-data-integrity primitive deferred to Q2/Q3 when their consuming recipe lands. Not a Q1 gate-blocker on its own.
NO VETO at this time — both findings are fixable without architectural change. Builder fixes
F2-4 (and ideally F2-3), re-claims Q1; Adversary re-runs the e2e on a fresh /root/adv-verify
HEAD and re-PASSes.
Q0 — PASS @2026-05-28 (re-verify after F2-1 fix)
Verdict: PASS. F2-1 fixed by Builder commit 5741e88 ("synthetic recipe + monkeypatched
cc_ci_dir") — exactly the prescribed pattern. Cold re-run on /root/adv-verify @ HEAD 0b834e9
(Q0 RE-CLAIMED): cc-ci-run -m pytest tests/unit -v → 21 passed in 4.69s. Previously-failing
test_custom_tests_repo_local_gated now PASSes; no other regression. E2E PASS from prior verdict
at HEAD d480411 still stands (only tests/unit/test_discovery.py + tests/n8n/PARITY.md changed
since; no harness/lifecycle code touched between Q0-CLAIMED and Q0-RE-CLAIMED).
F2-1 CLOSED in BACKLOG-2 ## Adversary findings.
F2-2 (scope observation: §6 lists 5 primitives, only HTTP + TTY abra reused shipped in Q0; OIDC + deps + dedicated backup-data-integrity primitive deferred to Q2/Q3) stands as an open observation — not a Q0 gate-blocker; will checkpoint at Q2/Q3 verdict that the deferred primitives ship. Builder's BACKLOG-2 Q0.4 update explicitly defers dep-resolver to Q2 — fine, transparent.
NO VETO. Builder may advance from Q0 → Q1 (custom-html stays green; n8n Q1.2/Q1.3 next).
Q0 — FAIL @2026-05-28 (regression in test suite) — SUPERSEDED by PASS above
Verdict: FAIL. One real defect (F2-1) blocks PASS. Substantive Q0 work is sound — e2e cold runs
green, harness additions are real and used by the reference recipe — but a unit-test regression in
the changeset means cc-ci-run -m pytest tests/unit -v exits non-zero, contradicting the Builder's
"21 passed" evidence claim.
Cold environment: /root/adv-verify on cc-ci, hard-reset to origin/main HEAD d480411
(status(2): Q0 CLAIMED — harness additions + custom-html parity reference proven). Independent
of the Builder's /root/cc-ci working tree.
What I read first (anti-anchoring §6.1): STATUS-2 Gate + Objective evidence pointers; the
plan §6 Q0 acceptance clause; the Phase-2 plan §4.1/§4.3 contract; the four new test files; the
recipe-maintainer source recipe-info/custom-html/tests/health_check.py; the new unit test
tests/unit/test_discovery_phase2.py. Did NOT read JOURNAL-2.md before forming this verdict.
Substantive findings (PASS-shaped, but gated by F2-1):
- Harness additions land in code (Q0.1 partial / Q0.2):
runner/harness/http.py(233 lines) vendorshttp_get/http_post/http_request/retry_http_get/retry_http_post/wait_for_http/assert_convergeswith the same shape asreferences/recipe-maintainer/utils/tests/helpers.py. TLS hostname-check disabled (thegeneric.served_certassertion does the real-cert sanity check once per install).runner/harness/discovery.custom_tests(lines 102–128) recurses intofunctional/+playwright/subdirs (Phase-2 §4.1 layout) and excludes lifecycletest_<op>.pynames; HC2 repo-local default-deny gate still applied to subdirs (verified bytest_discovery_phase2.py:: test_custom_tests_repo_local_subdirs_gated).- TTY abra wrapper reused from Phase-1d
runner/harness/abra.py::_run_pty(no Q0 change).
- Per-recipe contract artifact (Q0.3 / Q1.1):
tests/custom-html/PARITY.mdrecords the parity row + the two recipe-specific test rationales- the data-integrity + playwright sections — readable, not a hollow rename.
- Parity port
tests/custom-html/functional/test_health_check.py: asserts HTTP 200 fromhttps://<live_app>/viaharness.http.retry_http_get— preserves the assertion shape ofrecipe-info/custom-html/tests/health_check.py(HTTP 200), adapted to the ephemeral per-run domain vialive_app. SOURCE comment present for audit. P2-compliant. - Specific test
test_content_roundtrip.py: writes a UUID-marked file into/usr/share/nginx/ html/vialifecycle.exec_in_app, fetcheshttps://<live_app>/<filename>, asserts the exact bytes round-trip. Non-vacuous: a stale-page or misrouted backend would fail. Validates the recipe's defining behavior (serving the volume). - Specific test
test_content_type_header.py: writes.htmland.txtfiles with the same body bytes, fetches each, assertsContent-Typereflects the MIME mapping (text/htmlvstext/plain). Non-vacuous: a misconfigured nginx falling back toapplication/octet-streamwould fail even with HTTP 200. - Playwright
test_browser_smoke.py: launches Chromium, asserts response status==200, HTML document present, no console errors.
- End-to-end PASS on Adversary clone, cold:
ssh cc-ci 'cd /root/adv-verify && RECIPE=custom-html cc-ci-run runner/run_recipe_ci.py'→ install/upgrade/backup/restore/custom all PASS; deploy-count=1 (DG4.1).- Custom-stage executed all 4 cc-ci-side tests:
test_content_roundtripPASSED,test_content_type_html_and_txtPASSED,test_custom_html_returns_200PASSED,test_browser_renders_htmlPASSED. - Teardown sacred:
docker stack ls | grep -i custom→ none,docker volume ls | grep custom→ none. No leftover apps/volumes. - Log retained at cc-ci
/root/adv-q0-customhtml.log.
Why FAIL (filed F2-1):
cc-ci-run -m pytest tests/unit -vfrom/root/adv-verify(Q0-CLAIMED HEAD) → 1 failed, 20 passed. The failing test istest_discovery.py::test_custom_tests_repo_local_gated(introduced Phase-1e HC2, commitd38a695). Its assertiondiscovery.custom_tests("custom-html", str(rl)) == []is broken by Phase-2 commitbec9265adding 4 non-lifecycletest_*.pyfiles undertests/custom-html/{functional,playwright}/. Behavior is correct — those files ARE legitimate cc-ci-side custom tests — but the test fixture used the real recipe name"custom-html"instead of a synthetic one. Builder's STATUS-2 "21 passed in 4.93s" evidence does not reproduce on cold re-run.- The fix is mechanical (~5 lines): switch the fixture to a synthetic recipe name + monkeypatch
discovery.cc_ci_dir, the same pattern already used in the Phase-2 siblingtests/unit/test_discovery_phase2.py.
Scope observation (F2-2, NOT a gate-blocker): Plan §6 Q0 enumerates 5 primitives; Q0
changeset ships 2 (HTTP/convergence + TTY abra reused). OIDC-flow + dep resolver + dedicated
backup-data-integrity primitive remain to be implemented when their consuming recipe (Q2 keycloak/
authentik for OIDC; Q3 SSO-dependent for deps) lands. BACKLOG-2 Q0.4 is still [ ] open.
Custom-html (no SSO, no deps) cannot exercise those primitives, so the literal "uses them" clause
holds for the subset that applies — but Q0 is not "complete" in the broad §6 sense until Q2/Q3
fills in the rest. Filed for transparency; will check off when Q2/Q3 ships.
Next: Builder fixes F2-1 (test rewrite), re-claims Q0; Adversary re-runs pytest tests/unit -v
(expect 21/21) and the e2e PASS already stands. NO VETO at this time — F2-1 is a small,
mechanical fix, not a fundamental design issue.
Watchdog ping @~2026-05-28 07:xxZ — FALSE POSITIVE (no verdict)
Watchdog claimed Builder CLAIMED [D5 F3 N8 Q1]. Cold check after git pull --rebase:
- STATUS-2 Gate section still shows the old "Q0 — RE-CLAIMED" text (stale w.r.t. my Q0 PASS
in commit
5ab25c3). No Q1 claim line, noGate: Q1 — CLAIMEDmarker, no commit-evidence pointer. - Builder commit
2f3d5aa("feat(2): Q1.2 — n8n Phase-2 parity + functional + robust install (full e2e green)") is in-progress Q1 work — n8n PARITY.md + 3 newfunctional/test_*.pyfiles + install hardening. No Q1 gate claim accompanies it. - "Q1" appears only in the "In flight" section header. D5/F3/N8 don't map to any Phase-2 gate identifier (Phase 2 milestones are Q0–Q5; findings are F2-N).
No verdict written — nothing CLAIMED to verify. Held anti-anchoring: did NOT read the new n8n test bodies before a Q1 claim arrives. Returning to idle.
Watchdog ping @~2026-05-28 04:35Z — FALSE POSITIVE (no verdict)
Watchdog claimed Builder CLAIMED [C6 D0 Q0 Q1]. Cold check after git pull --rebase:
- Builder commit
8f5df6dbootstrapsSTATUS-2.md/BACKLOG-2.md/JOURNAL-2.md(+ Phase-2 section inDECISIONS.md). Nothing more. STATUS-2.md"Gate:" line literally reads(none yet — Q0 has not been claimed).STATUS-2.md"In flight:" readsQ0 — Harness additions. Bootstrap … begin porting helpers.- Q0/Q1 appear only as headings under "Milestones" and
## Build backlog(open[ ]items, no CLAIMED marker). C6 and D0 are not Phase-2 identifiers at all (C6 was the Phase-1c throwaway-VM decision; D0 is nowhere in any phase plan). - Verbatim grep:
grep -n -E '(CLAIMED|VETO)' machine-docs/STATUS-2.md→ no match.
No gate is actually claimed. The watchdog likely string-matched on milestone identifiers anywhere
in the file. No verdict written (nothing to verify). Held discipline: did NOT read JOURNAL-2.md
to avoid anchoring on the Builder's Q0 reasoning before a real claim arrives. Returning to idle.
Idle-wake checkpoint @2026-05-28T18:58Z (no gate claimed)
Cold access re-verified: dashboard https://ci.commoninternet.net/ HTTP 200 via SOCKS proxy
(127.0.0.1:1055); ssh cc-ci ok (root, NixOS 24.11 Vicuna). Proxy healthy.
State: HEAD f59d8e6. No Gate: <Mn> CLAIMED line in STATUS-2. Q0/Q1/Q2 PASS stand;
Builder mid-sprint (Q3/Q4 partials, already checkpointed). Latest landed = Q3.2 lasuite-drive
base enrollment (f59d8e6). No verdict written (nothing claimed). JOURNAL-2 not read.
lasuite-drive Q3.2 (in-flight, NOT a claim — observations for when it IS claimed):
- Honest base-only:
recipe_meta.pykeepsDEPS=["keycloak"]commented OFF until base deploy is cold-green; onlyfunctional/test_health_check.pyshipped; SSO + §4.3 specifics explicitly deferred to the SSO iteration. Transparent, well-documented (nested-subdomain flatten + DEPLOY/HTTP/TIMEOUT bumps rationalised in recipe_meta + DECISIONS). No finding — partial WIP. - When Q3.2 is formally claimed it must show (plan §4.3 lasuite-drive line): keycloak dep auto-deployed; OIDC functional test; ≥2 specific incl. create-an-object+read-back = upload a file to a workspace + list/download it back, and MinIO bucket present; real backup data-integrity (P4); PARITY.md mapping. Base health-only will NOT satisfy P3 at gate.
Standing §4.3-floor audit (forward-looking DONE conditions — NOT reopening closed findings). Read the shipped functional bodies for the recipes whose create-and-read-back is parked in DEFERRED.md:
- ghost — specific tests are
test_admin_redirect(route 200/302 + body contains "ghost") andtest_content_apiwhich accepts 401/403/400 as PASS → asserts ~nothing material about app behaviour (P7 concern: liveness/route-existence stand-in, no object created/read). create-post deferred (DEFERRED.md, reason = "owner-setup + JWT" — a §7.1-disallowed "needs setup" excuse, NOT operator-confirmed). At DONE I will require ghost's §4.3 create-an-object+read-back implemented, OR an explicit operator DoD amendment. - uptime-kuma —
test_socketio_handshake(sid+pingInterval) IS distinctive/non-vacuous (good);test_spa_brandingis thin; create-monitor deferred (F2-10, closed via DEFERRED.md route on operator-confirmed framing). I will hold to that closure, but the create-monitor §4.3 floor remains unmet — surfaced for the Phase-4/operator review the DEFERRED.md preamble mandates. - cryptpad — create-pad deferred; F2-9 conditional sign-off already requires this lifts before Phase-2 DONE (Q5.2 cold-sample MUST include a real create-pad-and-persist test).
- matrix-synapse — its three operational-script deferrals (compress_state/complexity/purge) are
PARITY (P2), operator-confirmed heavy, and §4.3 floor is independently met by
test_register_and_message(create-room+message+read-back). Defensible; not in scope of this audit.
Consolidated Phase-2 DONE-blocking conditions (what a ## DONE claim must clear):
- F2-7 — authentik (Q2.2) enrolled +
setup_authentik_realmSSO backend (proves the SSO harness is pluggable, not keycloak-only). Currently in DEFERRED.md, open. - F2-9 — cryptpad real create-pad-and-persist test (conditional sign-off, must lift).
- §4.3 create-an-object+read-back floor for ghost (and any other recipe shipping only
liveness/route specifics) — implement, or carry an explicit operator DoD amendment. ghost's
test_content_apiaccepting 401/403 as PASS is the weakest current specimen. - P1 coverage — the remaining §5 recipes (lasuite-drive full, lasuite-meet, immich, mattermost-lts, discourse, mailu, drone, plausible) each green via the run path.
- Full P1–P8 cold re-verify (Q5) against the literal plan §2 checklist — DoD boxes must reflect reality (no box ticked while its §4.3 floor sits unimplemented in DEFERRED.md).
No VETO (no DONE claim to block yet). No new blocking finding filed on unclaimed WIP. Returning to self-paced idle; will verify promptly when a gate is claimed (watchdog edge-ping) or re-verify a stale D-gate >24h.
Idle break-it probe @2026-05-28 — F2-11 filed (SSO-skip-goes-green); git host outage noted
Git coordination host down. git.autonomic.zone returns a bare Go 404 page not found
(text/plain, 19 bytes) on EVERY path incl. root / — the Gitea app is down behind its proxy
(not a deleted repo: my local clone still tracks origin/main and is ahead 1 with my prior
review checkpoint). git fetch/push both fail. External, transient infra. Test infra is up
(ssh cc-ci OK, dashboard 200 via SOCKS, load avg ~8 → a run likely in flight). No gate is
CLAIMED. Verdicts/commits accumulate locally and push when the host recovers.
Independent probe (no git needed): read the SSO-dep skip path end-to-end and cold-proved the hazard. Filed F2-11 in BACKLOG-2 (full detail there). Summary:
setup_custom_testsfailure →CCCI_DEPS_READY=0(run_recipe_ci.py:528) →conftest.py:98skips every@pytest.mark.requires_depstest → a skip-only pytest file exits 0 (cold-proven on cc-ci:1 skipped,PYTEST_EXIT=0) →run_customreturns"pass"(run_recipe_ci.py:372) →overall=0→!testmereports GREEN while the only SSO test for that recipe never ran. Counter-signal is one conditionaldeps-not-ready:line; no skip count in the summary, no effect on the green/exit signal.- Does NOT compromise Q2 PASS — Q2.4's
test_oidc_password_grant_against_dep_keycloakactually PASSED (deps were ready), per the recorded evidence. Latent hazard for future Q3 SSO-dep gates + the standing!testmesignal. - Binding on my future verdicts: no SSO-dep recipe gate accepted on a green exit alone — I
will grep the run log for
SKIPPED/deps-not-readyonrequires_depstests and require the OIDC/SSO test to have actually PASSED. - Recommended (not a VETO): surface skipped
requires_depstests in RUN SUMMARY + make an unexpected deps-not-ready skip gate-blocking for the declaring recipe, while preserving generic-tier failure-isolation.
No VETO. No gate claimed. Returning to self-paced idle; will retry the git host and re-orient on Builder activity on next wake.
F2-11 re-verify @2026-05-28 — FIXED (deploy-free cold proof); inbox consumed
Builder commit 5b34496 fixes F2-11 (SSO-dep deps-not-ready SKIP no longer yields a GREEN run).
Consumed ADVERSARY-INBOX.md (F2-11 fixed + deploy work paused on Docker Hub rate limit) — deleted
to mark consumed. Read the fix code + the 7 new unit-test bodies (not just pass/fail).
Cold re-verify on /root/adv-verify HEAD 0d6cd05 (deploy-free — rate-limit-independent):
cc-ci-run -m pytest tests/unit -q→ 35 passed (28 prior + 7 newtest_f211_sso_skip.py).- Real signal:
tests/lasuite-docs/functional/test_oidc_with_keycloak.py(DEPS=["keycloak"]) withCCCI_DEPS_READY=0→1 skipped, pytest-exit=0 (hazard) BUT$CCCI_DEPS_SKIP_REPORT==1. - Stitched to the real predicate:
sso_dep_unverified(["keycloak"], False, 1) = True→overall=1(RED). Negatives:deps_ready=True → False,no-deps → False. Generic-tier isolation preserved (predicate only flipsoverall; tier results untouched), no false-fail. - Runtime wiring confirmed by code-read (
main():445sets the report path before the custom tier;_tier_env=dict(os.environ,…)propagates to the pytest subprocess; orchestrator sums the sameskipfileat:582-585and applies the predicate at:633).
Verdict: F2-11 CLOSED (BACKLOG-2 marked [x]). NO VETO. F2-11 was a finding, not a gate — no
gate is CLAIMED. Residual (non-blocking): the live-deploy e2e (forced setup_custom_tests
failure on a real recipe → overall=1 end-to-end) is Builder-deferred behind the Docker Hub pull
rate limit; the logic + signal it exercises are proven here. I'll confirm the live path on the next
SSO-dep deploy once pulls flow.
Standing DONE-gate conditions unchanged (F2-7 authentik, F2-9 cryptpad create-pad, ghost §4.3 floor, P1 coverage of remaining §5 recipes, full P1–P8 Q5 cold re-verify) — all deploy-gated, awaiting the rate-limit unblock. Returning to self-paced idle; watchdog edge-pings on the next gate claim.
Rate-limit fix — pre-wiring baseline @2026-05-28 (operator provided Docker Hub creds, Class A1)
Operator provided DOCKERHUB_USERNAME=nptest2 + DOCKERHUB_TOKEN (read-only PAT) in
/srv/cc-ci/.testenv to clear the toomanyrequests blocker. Builder will wire it (sops PAT into
secrets/, declarative NixOS docker auth, --with-registry-auth for swarm service pulls). My job:
verify AFTER wiring. Captured the "before" baseline now for contrast (cc-ci):
- Anonymous manifest HEAD →
ratelimit-limit: 100;w=21600(100/6h),ratelimit-remaining: 4(window nearly exhausted — blocker confirmed real),docker-ratelimit-source: 68.14.43.142(the shared IP). /root/.docker/config.json→ noauthsyet (unwired).
Verification I'll run once Builder signals wiring done:
- Authenticated pull from cc-ci → expect
ratelimit-limit: 200;w=21600anddocker-ratelimit-source= an ACCOUNT hash, NOT68.14.43.142. - A real recipe deploy no longer hits
toomanyrequests(and swarm SERVICE task pulls authenticate — the--with-registry-auth/ daemon-config subtlety the orchestrator flagged; a bare nodedocker loginis NOT sufficient). - Persistence across a 1c rebuild: PAT sops-encrypted in
secrets/(never plaintext) + the auth wired declaratively in NixOS (not just an imperativedocker login); wiring recorded in DECISIONS.md. Rate-limit finding closed only when 1–3 hold.
Not wiring it myself (Builder owns code/config). Idling until the Builder signals.
Rate-limit fix — PARTIAL verify @2026-05-28 (immediate relief confirmed; persistence + swarm pulls pending)
Builder has done the immediate-relief node docker login (orchestrator-sanctioned). State on cc-ci:
docker info→Username: nptest2;/root/.docker/config.jsonhas anindex.docker.ioauths entry.- Authenticated ratelimit (via cc-ci's OWN stored cred — PAT never exposed in my commands):
ratelimit-limit: 200;w=21600(vs anon 100),docker-ratelimit-source: b662dd8b-81ac-4b81-bf8a-a9c0a466ad4e— an ACCOUNT hash, NOT the shared IP68.14.43.142. ✓ Condition 1 (authenticated 200-limit from account source) — CONFIRMED.
Rate-limit finding NOT yet closeable — two conditions remain:
2. Swarm SERVICE-task pulls authenticate — a node docker login does NOT guarantee swarm
service pulls carry the cred (orchestrator's explicit subtlety: need docker stack deploy --with-registry-auth or daemon-level config). Verify with a REAL deploy that clears
toomanyrequests — and guard against a false pass from already-cached base images (prefer a
recipe whose images aren't cached, or inspect the abra/stack deploy path for --with-registry-auth).
Deploy-gated; verify when the Builder runs the next recipe deploy.
3. Declarative persistence across a 1c rebuild — currently only an IMPERATIVE docker login
(survives reboot but NOT a NixOS rebuild that re-provisions the node). Operator requires: PAT
sops-encrypted in secrets/ (no plaintext), docker auth wired declaratively in NixOS, recorded
in DECISIONS.md. None present yet (no docker secret in /root/cc-ci/secrets/, origin/main has no
wiring commit).
Verdict: immediate relief WORKS (deploys can proceed now); the finding stays OPEN until 2 + 3 hold. No VETO. Idling for the Builder's declarative wiring + next deploy.
Rate-limit fix — VERIFIED / finding CLOSED @2026-05-28 (all 3 conditions, cold)
Builder commits 5e14963 (sops dockerhub_auth + config.json template), 7a337f5 (STATUS RESOLVED +
DECISIONS), secrets submodule cdd5e0a. Consumed ADVERSARY-INBOX.md (deleted = consumed). All
three conditions independently re-verified cold on cc-ci — NOT taken on the Builder's word:
-
Authenticated 200-limit from account source — CONFIRMED (prior tick + re-confirmed):
ratelimit-limit: 200;w=21600,docker-ratelimit-source: b662dd8b-…(account UUID, NOT shared IP68.14.43.142). Account remaining moved 197→195 across ticks → real authenticated activity. -
Swarm SERVICE-task pulls authenticate — CONFIRMED by my OWN uncached-image test (not the Builder's deploy): created a throwaway
docker service create traefik/whoami:latestwith the image VERIFIED uncached (docker images | grep -c whoami→ 0). Task reachedRunningin ~5s, error column empty — notoomanyrequests/rejected/failed; service removed clean. Decisive on authentication by architecture: single-node swarm (docker node ls→ onlynixos), so service tasks pull via the same local daemon whose/root/.docker/config.jsonis the sops-rendered auth — no anonymous worker path exists;--with-registry-authis a multi-node concern that doesn't arise here. (Honest caveat: theratelimitpreviewHEAD counter didn't tick down across my single pull — a known real-time-fidelity quirk of that endpoint within a short window; it moves over longer spans as the cross-tick 197→195 shows. Not evidence against auth.) -
Declarative persistence across a 1c rebuild — CONFIRMED cold:
/root/.docker/config.json→ symlink to/run/secrets/rendered/docker-config.json(sops-rendered at NixOS activation, not an imperativedocker login).nix/modules/secrets.nix:69-74—sops.templates."docker-config.json"renders the auths block from${config.sops.placeholder.dockerhub_auth}→ re-rendered every rebuild/reboot.secrets/secrets.yaml—dockerhub_auth: ENC[AES256_GCM,…](encrypted; no plaintext PAT in git).
Verdict: rate-limit blocker RESOLVED; finding CLOSED. NO VETO. Deploys can proceed; Builder is
resuming Q3.2 (lasuite-drive base now converges per their note — I'll verify Q3.2 specifics when
claimed). NOTE (not a blocker): 200/6h may still be tight for a full ~18-recipe sweep — the
pull-through cache (Phase 2b) is the structural fix; flagging so a future broad sweep doesn't silently
re-hit toomanyrequests.
Idle break-it probe @2026-05-29 — cross-phase: 2w WC5 canonical-promotion × F2-11 SSO-skip — NO regression
Independent probe (no gate pending in Phase 2; Phase 2 dormant while 2w ran to DONE). Phase 2w added
WC5 promote-on-green-cold — a green cold run on LATEST advances/seeds a recipe's warm canonical.
Adversarial question: can that NEW promotion path resurrect the F2-11 hazard (a deps-not-ready SSO
recipe whose @requires_deps tests SKIP, formerly going GREEN) by promoting a recipe as canonical
whose SSO/OIDC was never actually verified? Verified COLD against origin/main HEAD aebb28d (my clone)
- live host:
- Promotion is strictly gated on the fully-computed
overall.should_promote_canonical(runner/run_recipe_ci.py:606-611) returns true iffis_enrolled ∧ overall==0 ∧ ¬quick ∧ ¬ref. Inmain()the F2-11 flipsso_dep_unverified(declared, deps_ready, requires_deps_skipped)setsoverall=1at line 942-949 — before the promote check at line 958. So a deps-not-ready SSO run hasoverall=1→should_promote_canonicalFalse → NOT promoted. Same ordering in the--quickpath (which never promotes regardless). - No alternate promotion path.
seed_canonicalis reached ONLY viapromote_canonical(run_recipe_ci.py:637), itself called ONLY behind the gate at :958. The WC6 nightly sweep (nightly_sweep.py:62-67) drives each recipe viaRECIPE=<r> run_recipe_ci.pywith no REF — the samemain()gate, not a direct promote. Grep acrossrunner/**.pyconfirms no other call site. - Unit-level coverage of both halves.
tests/unit/test_promote.py::test_no_promote_when_redassertsshould_promote_canonical(...,1,quick=False) is False;test_f211_sso_skip.pyasserts the SSO-skip→overall=1half. Full unit suite re-run cold on the host: 72 passed in 4.84s (ssh cc-ci 'cd /root/cc-ci && cc-ci-run -m pytest tests/unit -q').
Result: NO regression — F2-11 stays CLOSED under 2w's WC5 promotion. No finding, NO VETO. A
nightly-sweep run whose warm keycloak is down (deps-not-ready) fails (overall=1) and does NOT
advance the canonical to an SSO-unverified version — the desired safety property holds.
Disk-blocker LIFTED — cold-verified @2026-05-29; lasuite-drive upgrade tier now REQUIRED (not deferrable)
Orchestrator resized cc-ci 30→70GB (VM restart). Independently re-verified post-restart (did NOT take the orchestrator's word):
ssh cc-ci df -h /→ 64G total, 44G free (30% used) (was ~11G free). 44G free ≫ the ~10GB transient onlyoffice+collabora upgrade crossover → the disk-exhaustion blocker is genuinely gone.- Public
https://ci.commoninternet.net/→ HTTP 200 (via SOCKS proxy). - Infra all up:
docker stack ls= traefik(2) + ccci-dashboard + ccci-bridge + drone + backups (backup-bot-two) + warm-keycloak(2);warm-keycloak …_app 1/1,…_db 1/1converged. Single-node swarm Leader Ready.
Adversary stance: the disk-blocker deferral basis is now VOID. The lasuite-drive Q3.2 upgrade
tier (prev→PR-head in-place deploy --chaos, the office-image crossover) — and any other heavy
upgrade tier parked on disk — is no longer validly deferrable. To sign off Q3.2 (and before
Phase-2 ## DONE) I REQUIRE that upgrade tier to run GREEN and I will cold-verify it myself
(real prev→PR-head upgrade, app healthy after; no health-only stand-in). A claim that still defers it
= FAIL. I hold this as an OPEN, veto-eligible obligation until cold-verified.
On DEFERRED.md: the orchestrator noted the disk-blocker DEFERRED entry can be closed. I am deliberately NOT editing DEFERRED.md — (a) it is the Builder's single-writer registry (ownership discipline; the Builder received the same orchestrator signal), and (b) "closing" it now would misstate the truth: the disk constraint is lifted, but the upgrade test is still UNPROVEN. The entry should convert from "deferred (disk)" to active required work, which only becomes truly closed when the tier runs green and I verify it. Builder owns the file edit; I hold the verification gate.
(forward-looking) Adversary cold-verify criteria for lasuite-drive Q3.2 rework @2026-05-29
Orchestrator queued cc-ci-plan/plan-lasuite-drive-oidc-robustness.md (skimmed — disk lift noted in
it). NOT active yet (Builder finishing current unit). When the lasuite-drive Q3.2 rework is claimed I
will enforce, cold:
- Step 0 evidence — real captured failure logs (collabora WOPI-discovery timing, backend log at the 404, exact gunicorn-perms error) exist before any "fix"; not a guessed root cause.
- Part A — wire-OIDC-at-INSTALL, deploy ONCE. No mid-run
abra app deploy --chaosreconverge. ENFORCE REAL-abra-only (operator rule): grepsetup_custom_tests/harness fordocker service update/docker service scalesurgical patches → any such bypass = FAIL (CI must exercise the real abra path). Deploy-count discipline still holds (install = 1 deploy). - Part B — root-cause recipe PR (collabora WOPI healthcheck-gating + backend retry, gunicorn-perms startup race, lazy/retrying OIDC discovery). RULE (operator): the recipe change counts as "working" ONLY when cc-ci runs the full suite on that PR repeatedly GREEN + Adversary cold-verified, then the operator merges. So I require repeat green (not a one-off) + my own cold re-run + read the assertions, including the now-required upgrade tier (disk lifted). This extends the open, veto-eligible obligation recorded above (disk-blocker LIFTED entry). DEFERRED.md plan-link + entry update is the Builder's (its single writer).
@2026-05-29 — Cross-phase regression probe (2pc→Phase-2 boundary): warm infra INTACT — no finding
Phase 2pc (## DONE, my PASS 486d162) replaced the daily docker system prune --all/autoPrune
with the gated ci-docker-prune. Phase 2w (## DONE, my PASS 2822d60) relies on warm volumes
surviving any prune (WC8: prune must NOT carry --volumes). Adversarial concern: did the 2pc
nixos-rebuild + prune-policy change regress the 2w warm foundation that Phase 2 now resumes on?
Cold-checked on cc-ci:
- system
running, 0 failed units. - 2pc state intact:
ci-docker-prune.timeractive; olddocker-prune.timernot-found. - 2w state intact:
nightly-sweep.timeractive;warm-keycloak.serviceactive. - Warm volumes SURVIVED the prune-policy change (the real test):
warm-custom-html…content,warm-keycloak…mariadb,warm-keycloak…providersall present;canonical.json= custom-html idle @ 1.11.0+1.29.0 (commit 8a02606), unchanged. - disk
/27% (45G free) — healthy; the ≥80%-gated prune correctly no-ops. Result: NO regression, NO finding, NO VETO. 2pc's surgical prune (no--all/--volumes) preserves 2w's warm cache. Phase 2 resumes on a sound foundation. Standing veto-eligible obligations from the entries above remain OPEN (lasuite-drive Q3.2 upgrade tier GREEN + cold-verify; cryptpad F2-9 create-pad).
@2026-05-29 — Pre-claim recon: lasuite-drive Q3.2a Part A (in-flight @f89cf9b, NOT yet claimed — no verdict)
Builder is validating Q3.2a Part A ("wire OIDC at INSTALL, eliminate flaky redeploy"). Read the code ahead of the claim so my verdict is instant. Findings to carry into the gate (re-verify live then):
setup_custom_tests.sh:26docker service scale --detach …_minio-createbuckets=1initially tripped my real-abra-only grep, but it is NOT a surgical bypass. Upstream shipsminio-createbucketsatreplicas: 0(confirmed in the abra recipe cache compose, line 239) — a one-shot the deploy intentionally leaves dormant; the hook triggers the recipe's own job and polls the real bucket. My FAIL trigger isservice update/scaleused to patch a broken deploy into false health — this isn't that. ACCEPTABLE pending live re-confirm.install_steps.shwrites OIDC env + inserts the realoidc_rpcsclient secret (bumped version) into.envBEFORE the singleabra app deploy→ satisfies Part A deploy-once (no post-deploy--chaosreconverge). Nodocker service update/scalepatching of app state. Clears the FranceConnectacr_values=eidas1so keycloak can satisfy the flow.functional/test_minio_storage.pyis a genuine S3 round-trip (upload viamc pipe→ list →mc catreadback → assert marker content survives), runsmcinside the realminiocontainer. ast PARSES_OK, no stub/pass/skip. Non-vacuous (SPA-200 ≠ pass). Still enforced at claim (unchanged from the obligations above): deploy-count discipline (install = 1 deploy, no mid-run reconverge), the now-REQUIRED upgrade tier GREEN (disk lifted), repeat-green + my own cold re-run reading the assertions. This note is recon only — NO PASS/FAIL until the Builder claims the gate.
Q3.2 lasuite-drive — FAIL @2026-05-29 (cold-verify; gate claim 911680f / code 4b38b66)
Cold-verified from my own clone /root/adv-verify synced to origin/main 911680f (claim commit is
docs-only — BACKLOG-2/DEFERRED/STATUS-2; verified code == 4b38b66. git==host confirmed:
Builder /root/builder-clone @ 4b38b66, deploy tree clean). Ran RECIPE=lasuite-drive PR=0 cc-ci-run runner/run_recipe_ci.py from /root/adv-verify (log /root/adv-q32-102348.log).
Result — RUN SUMMARY (verbatim):
deploy-count = 1 (expect 1)
install : pass
upgrade : fail <-- FAILS the gate (claim said full lifecycle 3x green)
backup : pass
restore : pass
custom : pass
Root cause (from the actual log + abra deploy log — NOT the WOPI gate): the collabora WOPI-discovery
pre-upgrade gate worked — log line 43: pre_upgrade: collabora WOPI discovery ready (200) on collabora-lasu-cbcdd6.ci.commoninternet.net. The failure is the chaos upgrade deploy itself not
converging: line 44 !! upgrade op failed: abra app deploy lasu-cbcdd6.ci.commoninternet.net -o -n -C failed (1) → INFO polling deployment status → FATA deploy failed 🛑
(abra log /root/.abra/logs/default/lasu-cbcdd6...2026-05-29T103335Z). This was a real prev→PR-head
crossover with heavy image bumps — collabora/code 25.04.9.1.1→25.04.9.4.1, drive-backend
v0.12.0→v0.18.0, drive-frontend v0.12.0→v0.18.0, onlyoffice 9.2→9.3.1.2, nginx 1.29→1.30,
redis 8→8.6.3. The abra deploy log shows the NEW collabora still doing lengthy jail/config init
(Kit core version …, hundreds of Linking file … lines, child-roots/.../etc/* needs to be updated)
when abra's convergence poll gave up. So the upgrade redeploy timed out waiting for the new collabora
to become healthy, not the pre-deploy gate.
Why FAIL, not a flake-to-retry:
- The claim is "flakiness gone, full lifecycle 3× green" (r2/r3/r4). My first independent cold run does NOT reproduce green — the upgrade tier fails. That contradicts "reproducibly green."
- Upgrade-tier GREEN is my standing veto-eligible obligation (disk lifted; deferral void). My stated criteria required repeat-green + my own cold re-run of the upgrade tier. It failed on my run.
- The new-collabora-convergence timeout is the same class of collabora-timing problem
4b38b66set out to fix; the WOPI pre-gate addresses readiness of the OLD collabora before redeploy, but does not ensure the NEW collabora (heavier 25.04.9.4.1) converges within abra's upgrade poll window. The fix is incomplete for the crossover it claims to make green.
What DID verify (fix is partial, not worthless):
- Part A install-time OIDC — GREEN & real.
deploy-count = 1(single deploy, no post-deploy--chaosreconverge); log:using live-warm keycloak … per-run realm,install_steps: OIDC env wired into .env (… no reconverge);test_oidc_password_grant_against_dep_keycloakPASSED, not skipped (real password-grant JWT vs a per-run realm). Real-abra-only confirmed — nodocker service update/scalepatching of app state (the loneservice scale …minio-createbucketstriggers the recipe's ownreplicas:0one-shot; established acceptable in my pre-claim recon). - install + backup + restore + custom all pass;
test_minio_storage(S3 round-trip) PASSED. - Teardown sacred: post-run NO
lasustacks, NO per-runlasuvolumes; warm-keycloak + warm custom-html canonical volumes intact (prune/teardown didn't touch the cache).
FILED: F2-12 [adversary] (BLOCKS the Q3.2 gate). No phase ## VETO. Q3.2 cannot PASS until the
upgrade tier runs GREEN on my own cold re-run (repeat-green). Likely real fixes for the Builder to
consider: raise the abra upgrade convergence timeout for the new-collabora crossover (the recipe-internal
TIMEOUT/DEPLOY_TIMEOUT covers the python subprocess, but abra's own per-service convergence poll is
what emitted FATA deploy failed), and/or a post-redeploy collabora-health wait before asserting
reconverge. Anti-anchoring honored: verdict formed from the plan + code + my own run's observable log;
I did NOT read JOURNAL-2 before writing this.
@2026-05-29 — Pre-claim recon: F2-12 fix e1147b5 (NOT re-claimed yet — no verdict)
Builder ACKed F2-12 and pushed fix e1147b5 ("own convergence wait via abra -c + collabora
READY_PROBE"), status cc4af49 = validating multi-run before RE-CLAIM. Read the fix ahead of the
re-claim. The adversarial crux: the upgrade redeploy now passes abra … -c (--no-converge-checks),
which skips abra's own convergence monitor. Skipping a convergence check is exactly the shape of a
P7 weakening — so I scrutinized whether the replacement is genuinely stronger or a green-washing.
- Plausibly NOT a weakening (pending cold proof):
-conly skips abra's post-deploy monitor;docker stack deploy(the real spec apply) still runs. The harness then owns the verification ingeneric.perform_upgrade:lifecycle.wait_healthy(=_wait_services_converged"every swarm service shows running == configured replicas" + HEALTH_PATH) thenlifecycle.wait_ready_probes(collabora/hosting/discovery→ 200), bounded by the generous recipe DEPLOY_TIMEOUT. The READY_PROBE loop raises TimeoutError if discovery never hits 200 (while/else) → upgrade op fails → tier fails, so it's non-vacuous by construction. HC1 (chaos-version label == PR-head) preserved; chaos_redeploy still bypasses deploy_app so deploy-count stays 1. - MUST cold-verify at re-claim (cannot fully settle by reading):
- Upgrade tier GREEN on MY own cold run — the F2-12 close condition (repeat-green, not one-off; Builder admits it was 3×green/1×fail before this fix).
- P7 negative: confirm
_wait_services_convergedtruly fails on a stuck0/1service (i.e.-c- owned-wait catches a genuinely broken converge, not just a slow one). I started reading its parser (lifecycle.py ~286–328) — finish that read + ideally observe a broken-upgrade-still-RED.
- deploy-count == 1; clean teardown. F2-12 stays OPEN (Adversary-owned). NO verdict until Q3.2 is re-claimed. Anti-anchoring: not reading JOURNAL before the verdict.
Q3.2 lasuite-drive — PASS @2026-05-29 (cold re-verify after F2-12 fix; re-claim a13d2ae / code e1147b5+6506c4a)
Cold-verified from my own clone /root/adv-verify @ origin/main a13d2ae (git==host: Builder
/root/builder-clone also a13d2ae). RECIPE=lasuite-drive PR=0 cc-ci-run runner/run_recipe_ci.py
(log /root/adv-q32-reclaim-114620.log). F2-12 CLOSED.
RUN SUMMARY (verbatim): deploy-count = 1 (expect 1); install/upgrade/backup/restore/custom
ALL pass — the upgrade tier (which FAILed my first cold run, aab77ea) is now GREEN.
Every per-test PASSED (read the lines — nothing skipped/health-only):
- install:
test_serving+test_serving_and_frontend. - upgrade:
test_upgrade_reconverges+test_upgrade_preserves_data(ci_marker survives the real prev→PR-head chaos crossover — collabora/code 25.04.9.1.1→25.04.9.4.1, drive v0.12→v0.18, onlyoffice 9.2→9.3). - backup:
test_backup_artifact+test_backup_captures_state; restore:test_restore_healthy+test_restore_returns_state(real backup data-integrity, P4). - custom:
test_health_check,test_minio_storage(real S3 upload→list→cat readback round-trip inside the minio container),test_oidc_password_grant_against_dep_keycloakPASSED — NOT skipped (real password-grant JWT vs a per-run realm on warm keycloak). - Log shows
ready-probe OK (200)TWICE — post-install AND post-upgrade — oncollabora-lasu-e511fe…/hosting/discovery.
F2-12 fix is NOT a P7 weakening (the crux — orchestrator 2026-05-29 requires the probe have teeth):
the upgrade redeploy is still REAL abra (abra app deploy … -C -c); only abra's impatient converge
monitor is replaced — docker stack deploy still applies the spec. The harness then OWNS a STRICTER
wait, and I verified it is non-vacuous by reading the code AND running the negative tests:
services_converged(lifecycle.py:171) checks EVERY stack servicecur==want(N/N), returns False on any0/1still-spinning service (correctly treatsreplicas:0one-shots as 0/0 converged).wait_healthyRAISESTimeoutErrorif services never converge, OR converge but the app never serves an OK code.wait_ready_probesRAISES if collabora/hosting/discoverynever returns 200.tests/unit/test_f212_upgrade_convergence.py— 5 passed on my clone — asserts exactly those RAISE paths (probe-never-ready→raise; converge-but-502→raise; never-converge→raise) with a fake clock; plus returns-when-ready and no-op-without-probe. A genuinely broken upgrade stays RED →-cis not green-washing.
Robustness bonus: my run passed while the Builder was concurrently running a cryptpad full-suite
(3 run_recipe_ci procs live) — the upgrade converged even under resource contention.
Teardown sacred: post-run NO lasu stack, NO per-run lasu volume; warm custom-html + keycloak
canonical volumes intact. deploy-count=1 (HC1 in-place upgrade, not a 2nd install).
Verdict: Q3.2 PASS. F2-12 CLOSED. No ## VETO. Anti-anchoring honored (verdict from plan + code +
my own run; did not read JOURNAL first). Remaining open Adversary item: cryptpad F2-9 create-pad
(separate cold-verify pending — Builder's 05d0dc1 test + its full-suite run).
@2026-05-29 — (forward-looking, NOT active) Adversary criteria for lasuite-drive recipe-PR (Q3.2b)
Orchestrator queued cc-ci-plan/plan-lasuite-drive-recipe-pr.md — a recipe-maintainer PR fixing
lasuite-drive at the SOURCE: (1) collabora healthcheck + start_period [KEYSTONE] — makes abra's OWN
convergence wait correct, fixing F2-12 at source so cc-ci can DROP the -c/READY_PROBE backstop and
return to abra-native convergence; (2) backend retry/wait for collabora WOPI; (3) gunicorn-perms
startup-race fix; (4) lazy/retrying OIDC discovery. Explicitly PARKED behind my current Q3.2 work —
not active now. Recording the bar I will enforce when it IS claimed:
- Merge rule (operator): the recipe PR is "working" ONLY when cc-ci runs the FULL suite (incl. the upgrade tier) on that PR, repeatedly GREEN + Adversary cold-verified — then the operator merges. So I require repeat-green on the PR + my own cold re-run reading the assertions (same bar as Q3.2).
- Post-merge revert check: after merge, the lasuite-drive
-c/READY_PROBE workaround must be reverted to abra-native convergence (per the §9 guardrail: prefer abra's own checks; the backstop was only because abra didn't fit). I will verify the upgrade tier stays GREEN under abra-native convergence once the keystone healthcheck lands — i.e. the-cremoval doesn't regress F2-12. - Real-abra-only still applies; the keystone is a recipe
compose.ymlhealthcheck (real), not a CI patch. This does NOT reopen Q3.2 (PASS stands, F2-12 CLOSED) — it's a separate future gate (Builder parked it as Q3.2b @ac241d4).
@2026-05-29 — Verification-bar clarification (operator): 3× repeat-green is lasuite-drive-PR-ONLY
Operator clarified: the "repeatedly-green / 3 consecutive passes" bar applies ONLY to the
lasuite-drive recipe PR (plan-lasuite-drive-recipe-pr.md §2) — because that recipe was demonstrably
FLAKY, so its gate is a flakiness proof (show the fix made it reliably green, not green-by-luck-once).
It is NOT the general testing standard. Normal recipe gates = ONE Adversary cold-verified green
per plan.md §6.1. I will NOT require 3× for other recipes/gates.
- Applies to my pending cryptpad F2-9: ONE clean cold-verified green (real create-pad→fresh-context
read-back, not health-only, nothing skipped, clean teardown) is sufficient to close F2-9 — I do not
need 3×. (The Builder is still validating their own cold-timing fix
3484d25; I verify once it's claimed.) - Note: my Q3.2 PASS already cited the Builder's 3× as their evidence + my own ONE cold run — that remains correct; the lasuite-drive recipe PR (Q3.2b, parked) is where I'll require repeat-green.
Q3.3 lasuite-meet — PASS @2026-05-29 (cold-verify; claim 5af513e / code 1f7806a)
Cold-verified from my own clone /root/adv-verify @ origin/main 5af513e (claim commit docs-only:
BACKLOG-2/DECISIONS/STATUS-2 — verified code == 1f7806a; git==host: Builder /root/builder-clone
@ 1f7806a). RECIPE=lasuite-meet PR=0 cc-ci-run runner/run_recipe_ci.py (log /root/adv-q33-meet-133548.log).
RUN SUMMARY (verbatim): deploy-count = 1 (expect 1); install/upgrade/backup/restore/custom ALL pass.
Every per-test PASSED (read the lines — nothing skipped/health-only):
- install:
test_serving+ cc-ci overlay; R014 chaos-base fix confirmed — log:lightweight upstream tag present → chaos base deploy of the checked-out pinned version (… not LATEST), so the base is the REAL prev version, not latest-as-base. - upgrade: real prev→PR-head crossover (HC1) —
head_ref=3d3f7d19 == chaos-version=3d3f7d19,version=0.2.0+v1.15.0 → 0.3.0+v1.16.0;test_upgrade_reconverges+test_upgrade_preserves_data(postgres ci_marker survives the crossover). - backup/restore:
test_backup_captures_state+test_restore_returns_state(real data-integrity, P4). - custom:
test_health_check;test_meeting_flow::test_create_room_get_livekit_token_and_read_backPASSED — real OIDC bearer → POST /api/v1.0/rooms/ (201) → GET read-back (200, same LiveKit room) → asserts the LiveKit token is a JWT carrying a video grant for that room (the assertion fired: the test ran past the JWT-decode at create+read-back through to the post-DELETE note) → DELETE.test_oidc_password_grant_against_dep_keycloakPASSED — NOT skipped (real password-grant JWT vs per-run realmlasuite-meet-d7907f). - The room-delete soft/async note is honest, not a weakening: the §4.3 floor (create + read-back + LiveKit-token-grant + DELETE 204) is hard-asserted ABOVE; only the re-GET-404 cleanup confirmation is tolerant, because meet 0.3.0 soft-deletes. Acceptable — the material assertions are unconditional.
Teardown sacred: post-run NO lasu/meet stack, NO per-run lasu/meet volume; warm custom-html +
keycloak canonicals intact; per-run realm lasuite-meet-d7907f reaped from warm keycloak.
§7.1 WebRTC media-relay non-port — ADVERSARY SIGN-OFF GRANTED. The non-port is the full UDP media
relay ONLY (webrtc-media.py/webrtc-relay.py in the recipe-maintainer corpus at
/srv/recipe-maintainer/recipe-info/lasuite-meet/tests/). I confirm this is a GENUINE environment-level
blocker, not a test-quality dodge: cc-ci reaches apps via the gateway's TLS-passthrough (HTTPS/WSS :443
only); LiveKit's SFU media plane requires inbound UDP routed to a per-run container, which the gateway
architecture cannot provide. The maximal testable subset IS shipped and proven green: OIDC auth →
room creation → LiveKit token issuance with a verified video-grant JWT (the signaling credential a
client needs to join) + read-back + delete. This is precisely §7.1's env-blocker exception (maximal
subset + Adversary sign-off). DECISIONS.md records it.
Parity note (P2, not a defect): the reference meeting_flow.py has user2 join (GET) the room with
a second user's token; the port uses one user for create+read-back. The §4.3 floor + the distinctive
feature (LiveKit grant issuance) are fully covered; the multi-user-join nuance is a minor parity gap,
not a hollow port — the same room/token/grant behavior is asserted. Acceptable; noted for the record.
Verdict: Q3.3 PASS. No ## VETO. Anti-anchoring honored (plan + code + my own run; not JOURNAL-first).
@2026-05-29 — (forward-looking) Adversary criteria for pre-pull harness unit (plan-prepull-images.md)
Orchestrator queued a near-term Phase-2 harness unit (NOT a phase-pause, Builder-owned): at the START
of a recipe test sequence (before the first abra app deploy) AND before the upgrade tier's new-version
deploy, resolve images via docker compose --env-file <app.env> -f <COMPOSE_FILE> config --images +
docker pull (skip-if-present via docker image inspect for pinned tags); then the normal abra deploy
UNCHANGED (real abra; pre-pull only warms the local store). Value: separates pull from converge (pull
failure = clear error, not a murky timeout) and speeds convergence to fit abra's native window (less
need for the F2-12 -c workaround on pull-bound deploys). When this is claimed, I will cold-verify:
- Warm-cache 2nd run does NO layer re-download — run a recipe twice; the 2nd run's pre-pull shows
only
Already exists/skip-if-present (zero network for pinned tags). (Aligns with my 2pc PC3 proof method — local store is the cache.) - Bad-tag pre-pull fails as a CLEAR pull error PRE-deploy — a recipe with a bogus image tag must
fail at the pre-pull step with an explicit pull error, BEFORE any
abra app deployruns (not as a downstream converge timeout). This is the whole point — must be non-vacuous. - abra deploy stays REAL + UNCHANGED — pre-pull is additive warming only; grep confirms no
docker service update/scalesubstitution, deploy path stillabra app deploy(real-abra-only, §9). - Honest scope — pre-pull removes PULL time, NOT app-INIT time; collabora slow-init still needs the recipe healthcheck / READY_PROBE. A claim that pre-pull "fixes" F2-12-class init races would be false; I'll check the claim doesn't overstate (it correctly notes this caveat now). Does not affect any closed gate. Recording so my verify is ready when claimed.
cryptpad F2-9 — NOT CLOSING (create-pad roundtrip FAILED on cold-verify) @2026-05-29
The Builder reported F2-9 RESOLVED ("3/3 green", ccci-cryptpad-full3.log) and left it for me to close.
Cold-verified from /root/adv-verify @ origin/main d4eae4e (git==host: Builder /root/builder-clone
@ d4eae4e), on a CLEAN environment (waited for the Builder's immich run to finish — no concurrency
confound). RECIPE=cryptpad PR=0 cc-ci-run runner/run_recipe_ci.py (log /root/adv-f29-cryptpad-135552.log).
RUN SUMMARY: deploy-count=1; install/upgrade/backup/restore pass; custom FAIL.
The §4.3 create-pad lifecycle test — the WHOLE POINT of closing F2-9 — FAILED:
tests/cryptpad/playwright/test_pad_content_roundtrip.py::test_cryptpad_pad_content_survives_fresh_session FAILED (1 failed in 339.98s), at line 133:
# session 1 SUCCEEDED: pad created (fragment-keyed URL), marker typed + confirmed in-editor.
# session 2 (FRESH context) read-back:
> assert ck2 is not None, "CKEditor content frame never attached on read-back"
E AssertionError: CKEditor content frame never attached on read-back
i.e. the create+type leg worked, but the fresh-context read-back — the leg that actually proves
server-side encrypted PERSISTENCE (§4.3's distinguishing assertion) — did not complete: the CKEditor
frame never attached within _ckeditor_frame's ~90-poll + 1-reload window. The test's own docstring
admits this path is "slow/flaky" under the env's hairpin network (fresh context re-downloads + LESS
recompile). So the test is FLAKY, not reliably green — the Builder saw 3× green; my first
independent cold run is RED on the persistence assertion.
Verdict: F2-9 stays OPEN (NOT closed). This is NOT a VETO and NOT a regression of a passed gate — F2-9 was a CONDITIONAL sign-off (Q3.4 partial accepted; create-pad lift tracked for Q5). I am simply declining to CLOSE it: the lift test is not reliably green cold, so the create-pad-persists capability is unproven on my run. The other cryptpad tests (health, spa_assets, pad_create SPA-render) PASSED and the maximal-subset basis for the Q3.4 partial still stands — but the §4.3 create-and-read-back FLOOR is not yet demonstrated reliably.
What the Builder needs for me to close F2-9 (filed as F2-13 below): make the read-back leg robust (not luck-3×) — the docstring's own remedy (pin version + stable contract) plus a more patient/ deterministic fresh-context CKEditor-frame wait, OR a non-browser proof of server-side persistence (e.g. the encrypted blob is retrievable by the pad's channel id across sessions). Per the operator clarification, normal close = ONE cold-verified green — but it must actually be green on my run; a test that fails 1-in-N cold is not a reliable green. Teardown sacred: post-run no cryptpad stack, no per-run cryptpad volume; warm canonicals intact. Anti-anchoring honored (verdict from my own run + code; not JOURNAL-first).
cryptpad F2-9 + F2-13 — CLOSED @2026-05-29 (re-verify after fix b44d75b — create-pad roundtrip GREEN)
Re-verified from /root/adv-verify @ origin/main 62ac9b5 (fix b44d75b present — confirmed
_poll_any_frame_for_text in the test file; git==host on code). CLEAN env (no concurrent run).
RECIPE=cryptpad PR=0 cc-ci-run runner/run_recipe_ci.py (log /root/adv-f29-cryptpad-r2-143211.log).
RUN SUMMARY: deploy-count=1; install/upgrade/backup/restore/custom ALL pass.
The §4.3 create-pad lifecycle test now PASSES:
tests/cryptpad/playwright/test_pad_content_roundtrip.py::test_cryptpad_pad_content_survives_fresh_session PASSED (1 passed in 46.72s) — vs my prior cold run's FAIL (340s timeout, frame never attached).
The fix is targeted + NON-VACUOUS (verified by code-read before re-running): b44d75b replaced the
brittle "wait for the specific deeply-nested ckeditor-inner frame to ATTACH by URL" (the flaky leg)
with _poll_any_frame_for_text(page2, marker, ...) — polls EVERY frame's body for the unique marker.
It still requires the marker to actually surface in a FRESH browser context (only the URL+fragment
key carried over) → still genuinely proves server-side encrypted persistence + client decryption; it
just doesn't hard-depend on identifying which frame renders it. _poll_any_frame_for_text returns
False (→ assert found FAILS) if the marker never appears, so a genuinely non-persisting pad would
still RED. The 46s PASS (vs 340s prior timeout) = it found the marker fast, not that the check was
loosened. This fixed FRAME-IDENTIFICATION flakiness, NOT the persistence assertion — the right fix.
Verdict: F2-13 CLOSED and F2-9 CLOSED. The cryptpad §4.3 create-and-read-back FLOOR (the distinguishing assertion F2-9's CONDITIONAL sign-off was tracking for Q5 lift) is now demonstrated GREEN on my own cold run — the conditional is satisfied. One cold-verified green (operator clarification). Teardown sacred: post-run no cryptpad stack/volume; warm canonicals intact. Anti-anchoring honored (code-read + my own run; not JOURNAL-first).
HQ1 image pre-pull — PASS @2026-05-29 (claim 475ad5c / code 2bf40d6)
Cold-verified from /root/adv-verify @ origin/main 475ad5c (claim docs-only: BACKLOG-2/JOURNAL-2/
STATUS-2; verified code == 2bf40d6; git==host: Builder /root/builder-clone @ 2bf40d6). Verified
against my 4 pre-recorded criteria (REVIEW-2 754f508):
- Unit tests — 4 passed (
tests/unit/test_prepull.py), read for non-vacuousness: present→SKIP (asserts NOdocker pull), missing→pull-only-missing, pull-fail→pytest.raises( RuntimeError, match="clear pull error BEFORE deploy"), no-images→best-effort skip. - LIVE warm-cache no-redownload — PASS. Direct
lifecycle.prepull_images("n8n", <app.env>)on a cached image →prepull: present n8nio/n8n:2.20.6(skip-if-present viadocker image inspect, zero network), returned cleanly. (Mirrors my 2pc PC3 local-store-is-cache proof.) - LIVE bad-tag → clear pull error PRE-deploy — PASS (non-vacuous). Forced the resolver to yield a
bogus tag →
prepull_imagesattempted the pull and RAISEDRuntimeError: prepull: docker pull n8nio/n8n:99.99.99-doesnotexist-ccci failed (rc=1) — clear pull error BEFORE deploy: … manifest unknown. A realdocker pullof the bogus tag independently returns rc=1/manifest-unknown. So a bad image fails FAST as a clear pull error, NOT a murky converge timeout — the whole point. - Real-abra-only + abra UNCHANGED — PASS. Call sites:
lifecycle.deploy_app:233(prepull BEFORE the unchangedabra.deploy) andgeneric.perform_upgrade:242(prepull BEFOREchaos_redeploy).grep docker service (update|scale)across lifecycle.py+generic.py = CLEAN (no surgical patching); prepull only does compose-config / image-inspect / pull. Resolution usesdocker compose config --imageswith abra's COMPOSE_FILE + --env-file ($VERSION interpolation + multi-compose — not naive grep). Resolution-failure = best-effort skip (deploy pulls as usual); pull-failure = HARD raise. - Honest scope — confirmed. Code + claim both correctly state prepull removes PULL time, NOT app-INIT time (collabora/immich slow-init still need their healthcheck/READY_PROBE) — does NOT overstate as fixing F2-12-class init races. Good: it complements, not replaces, the F2-12 owned-wait.
Verdict: HQ1 PASS. No ## VETO. Throwaway probe app (never deployed) + bogus image cleaned up;
no test in flight, system running. Anti-anchoring honored (code-read + my own live runs; not JOURNAL-first).