Per REVIEW-2 ## Q1 — PASS @2026-05-28: F2-3 + F2-4 closed; cold e2e on Adversary clone all 5
stages PASS; deploy-count=1; HC1 non-vacuous; teardown sacred; NO VETO. Builder may advance to Q2.
Q2.1 keycloak in flight: first attempt hit 502 from /realms/master at 600s; bumped DEPLOY_TIMEOUT
+ HTTP_TIMEOUT to 900s in tests/keycloak/recipe_meta.py.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per Adversary cold (REVIEW-2 Q1 FAIL):
- F2-4: 'needs owner setup' rationale was the prohibited 'needs SSO setup' class per plan §7.1.
Fixed by tests/n8n/functional/test_workflow_roundtrip.py (commit fc89552) — the plan §4.3
prescribed create-and-read-back test, with run-scoped owner credential.
- F2-3: page.goto raised PlaywrightError outside the retry loop on net::ERR_*. Fixed by wrapping
page.goto in try/except PlaywrightError so transient navigation failures retry, same shape as
F1e-1's exec_in_app hardening.
Cold-verifiable: ssh cc-ci 'RECIPE=n8n cc-ci-run runner/run_recipe_ci.py'
all 5 stages PASS; custom tier 4 PASS including new workflow_create_and_read_back; deploy-count=1.
Keycloak Q2.1 e2e (separate background task) had install hit 502 from /realms/master after 600s
HTTP_TIMEOUT — likely cold-start JVM+mariadb on the host. Will investigate post Q1 verdict.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
F2-4 (P3/§4.3 floor — gate-blocker on Q1):
tests/n8n/functional/test_workflow_roundtrip.py: plan §4.3 prescribed test.
POST /rest/owner/setup with class-B run-scoped owner email+password (plan
§4.4-B); capture auth cookie; POST /rest/workflows with a minimal Manual-
Trigger workflow; GET /rest/workflows/<id>; assert the round-trip (id,
name, nodes payload all preserved). Removes the prohibited 'needs owner
setup' excuse; exercises n8n's defining persistence + retrieval surface.
F2-3 (cold-run flake on install):
tests/n8n/test_install.py: wrap page.goto(...) in try/except PlaywrightError
inside the retry loop so net::ERR_* / connection resets trigger a retry
instead of an immediate test failure. Same pattern as F1e-1's exec_in_app
poll+raise hardening.
PARITY.md updated: 3 recipe-specific tests now listed; workflow_roundtrip
called out as the plan §4.3 prescribed create+read-back; rationale for keeping
test_rest_settings / test_login_state retained.
Cold-verifiable on cc-ci (log /root/ccci-q1-n8n-r4.log):
RECIPE=n8n cc-ci-run runner/run_recipe_ci.py
all 5 stages PASS, deploy-count=1, head_ref=63dd3e0f==chaos-version=63dd3e0f.
Custom tier ran 4 PASS: health_check, login_state, rest_settings, AND the
new workflow_create_and_read_back.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Q1.1 custom-html: parity port + 2 NEW recipe-specific + playwright (Q0 PASS evidence stands).
Q1.2 n8n: parity port + 2 NEW recipe-specific (rest_settings, login_state — both reject the
'n8n is starting up' placeholder, so non-vacuous). install overlay now polls page.goto until
status==200 (absorbs n8n's /healthz-200-before-/-route-registered boot race).
Q1.3 n8n backup data-integrity: covered by Phase-1d/1e lifecycle overlay pattern (volume marker
survives backup→mutate→restore — PASSED in Q1.2 e2e).
Q1.4 CLAIMED.
Cold evidence: ssh cc-ci 'RECIPE=n8n cc-ci-run runner/run_recipe_ci.py'
all 5 stages PASS, deploy-count=1, head_ref==chaos-version (HC1 non-vacuous), version moved
3.1.0+2.9.4 -> 3.2.0+2.20.6.
Q1.2 note: deferred 'create workflow via API' from plan §4.3 in favor of /rest/settings +
/rest/login JSON-shape assertions (equally non-vacuous, no owner-setup state to manage); recorded
in BACKLOG-2 + JOURNAL-2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- tests/n8n/PARITY.md: parity table (health_check ported) + 2 recipe-specific
functional tests with rationale + data-integrity section pointing to
Phase-1d/1e lifecycle overlays.
- tests/n8n/functional/test_health_check.py: parity port of
recipe-info/n8n/tests/health_check.py — SOURCE comment.
- tests/n8n/functional/test_rest_settings.py: NEW recipe-specific — polls
/rest/settings until response is application/json (not the 'n8n is starting
up' SPA placeholder); asserts known n8n public-settings keys
(userManagement/defaultLocale/authCookie) in the 'data' envelope. Proves the
editor SPA's primary API contract is intact.
- tests/n8n/functional/test_login_state.py: NEW recipe-specific — polls
/rest/login until response is JSON; proves the user-management/auth subsystem
initialized on top of the public-settings layer.
- tests/n8n/test_install.py: install overlay's Playwright now polls page.goto
until status==200 (n8n's / route can return 404 briefly while the SPA route
registers on top of /healthz=200). Bounded poll, no bare sleep, raise on
persistent failure — same robustness pattern as Phase-1e exec_in_app.
Cold-verifiable on cc-ci (log /root/ccci-q1-n8n-r3.log):
RECIPE=n8n cc-ci-run runner/run_recipe_ci.py
all 5 stages PASS, deploy-count=1, head_ref=63dd3e0f==chaos-version=63dd3e0f,
version 3.1.0+2.9.4 -> 3.2.0+2.20.6 (HC1 non-vacuous), 5 lifecycle assertions
+ 3 custom-stage assertions all PASS.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per Adversary cold (REVIEW-2 "Q0 FAIL"), F2-1 mechanical regression: the Phase-1e HC2 unit test
asserted custom_tests('custom-html', rl) == [] when the real custom-html dir had no functional/
tests. Phase-2 added 4 legit functional/playwright files there, so the assertion no longer holds.
Behavior is correct; the test fixture was brittle.
Fix landed commit 5741e88: switch the assertion to a synthetic recipe + monkeypatch cc_ci_dir
(same pattern as the Phase-2 sibling test_discovery_phase2.py). Cold re-run: 21/21 PASS.
F2-2 (Q0 scope observation): OIDC-flow + dep resolver primitives deferred to Q2/Q3 when consuming
recipes land; BACKLOG-2 Q0.4 explicitly tracks this — acknowledged in STATUS-2 gate text.
Q0 RE-CLAIMED, awaiting Adversary re-verify.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Phase-1e HC2 test asserted custom_tests('custom-html', repo-local) == [] when only the
repo-local dir was set + custom-html had no cc-ci-side functional tests. Phase-2 commit bec9265
added 4 legitimate non-lifecycle test_*.py files under tests/custom-html/{functional,playwright}/
which custom_tests() now correctly returns — breaking the == [] assertion.
The custom_tests behavior is correct; the test fixture was using the real recipe name. Fix: switch
to a synthetic recipe + monkeypatch cc_ci_dir (same pattern already used in the Phase-2 sibling
test_discovery_phase2.py). 5-line change, no behavior change.
Cold-verifiable on cc-ci: cc-ci-run -m pytest tests/unit -v -> 21 passed in 5.38s
(Adversary's F2-1 repro now PASSes; no other regression).
Also: tests/n8n/PARITY.md drafted for the in-flight Q1.2 work (n8n parity port).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- tests/custom-html/PARITY.md: parity mapping (health_check.py ported);
recipe-specific tests recorded with rationale; backup data-integrity +
playwright sections.
- tests/custom-html/functional/test_health_check.py: parity port of
recipe-info/custom-html/tests/health_check.py — SOURCE comment included.
- tests/custom-html/functional/test_content_roundtrip.py: NEW recipe-specific —
write a marker into the served volume, fetch over HTTPS, assert exact bytes.
- tests/custom-html/functional/test_content_type_header.py: NEW recipe-specific —
prove nginx returns text/html for .html and text/plain for .txt (MIME mapping).
- tests/custom-html/playwright/test_browser_smoke.py: P6 browser smoke (renders
HTML, no console errors). Standalone Phase-2 custom-stage version.
Verified cold on cc-ci (STAGES=install,custom): 5 assertions all PASS in one
run (install generic + install overlay + content roundtrip + content type +
health check + browser smoke), deploy-count=1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- runner/harness/http.py: canonical Phase-2 recipe-test HTTP API (vendored from
recipe-maintainer/utils/tests/helpers.py): http_get/http_post, retry variants,
wait_for_http, assert_converges. JSON-parsing, header support, form/JSON POST
bodies, transport-failure -> status=0. Self-contained (cc-ci does not import
recipe-maintainer at runtime per DECISIONS Phase 2).
- harness.discovery.custom_tests now also recurses into
tests/<recipe>/{functional,playwright}/test_*.py (Phase 2 §4.1 layout) while
excluding lifecycle test_<op>.py names and honoring the HC2 repo-local gate.
- Unit tests:
tests/unit/test_http.py — in-process http.server fixture; deterministic
proofs of parsing/retry/convergence semantics, no network egress.
tests/unit/test_discovery_phase2.py — functional/+playwright/ recursion
+ HC2 gate still applies to subdirs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
build #155 (own !testme on custom-html PR#2): head_ref=db9a9502 == chaos-version=db9a9502
(1.10.0→1.13.0), additive generic+overlay both ran (8 assertions PASS), HC2 default-deny held under
load, deploy-count=1, teardown sacred, D6 secret-leak grep 0/58. F1e-1 CLOSED. F1e-2 pre-existing
(not a 1e regression). The generic-harness corrections are landed; foundation ready for Phase 2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
F1e-1 (Adversary): exec_in_app silently returned '' on a failed docker exec, flipping a healthy
recipe RED under opt-out (post-backup container cycle, no readiness buffer). Now polls (re-resolve
container + re-exec) until rc==0 or 90s, then RAISES — never masks an exec failure as empty data.
No assertion weakened. Verified: opt-out install,backup,restore on custom-html now PASS.
HC1: head_ref = ref or recipe_head_commit (prefer explicit PR head sha $REF — robust, no git race;
production !testme always sets REF). assert_upgraded, when head_ref known, REQUIRES the deployed
chaos-version commit to MATCH head_ref (direct + non-vacuous proof the PR-head code was deployed; a
stale prev-checkout chaos redeploy fails). Falls back to version/image/chaos move check otherwise.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
default run: every tier ran generic+overlay (op once, deploy-count=1); CCCI_SKIP_GENERIC=1 run:
generic skipped, overlays only. Clean teardown both. E0/HC2 recorded as Adversary PASS.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- orchestrator: per mutating tier, run optional pre-op seed hook (ops.py pre_<op>) → perform the op
ONCE (harness-owned) → run generic assertion (unless opted out) AND overlay assertion, both against
the shared post-op deployment. Op results passed op→assertion via run-scoped CCCI_OP_STATE_FILE.
- opt-out: CCCI_SKIP_GENERIC / CCCI_SKIP_GENERIC_<OP> / recipe_meta.SKIP_GENERIC (declarative).
- generic.py: split do_* into op primitives (perform_upgrade/backup/restore) + assertions
(assert_upgraded/backup_artifact/restore_healthy) reading op_state(); deployed_identity now returns
{version,image,chaos} (chaos label ready for HC1).
- generic test_<op>.py + all 6 recipe overlays migrated to assertion-only; pre-op seeding moved to
per-recipe ops.py (pre_upgrade/pre_backup/pre_restore). install overlays unchanged (no op).
- deploy-count stays 1 (op primitives never call deploy_app). lint PASS; 8 unit tests PASS on cc-ci.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
hedgedoc mirrored to recipe-maintainers/hedgedoc with probe PR #1; add it to the bridge poll list so
!testme triggers the full generic suite (no cc-ci/repo-local overlay -> pure generic). Rebuild pending.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mechanical port to the assertion-only contract (no softened/skipped assertions): install uses
live_app + generic.assert_serving (extend) + the recipe's http/playwright/api checks; upgrade seeds
its data marker then generic.do_upgrade + asserts survival; backup/restore split into test_backup.py
(seed->do_backup->mutate) + new test_restore.py (do_restore->assert original). Recipe-specifics
preserved verbatim (keycloak realm+admin-console+kc_admin, matrix/lasuite db-service psql markers,
cryptpad/n8n volume markers). No recipe now double-deploys under the deploy-once orchestrator.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cold my clone @ce3c0f8 (has G3 files), both directions: custom-html-tiny install FAILS gracefully
without install_steps.sh (404, per-op, deploy-count=1) and PASSES with it (hook seeds index.html).
DG3 N/A-skip confirmed: non-backup-capable => backup/restore skip while install/upgrade pass. Move
-assertion robust to image-identical version bump (1.0.0->1.0.1, same image 2.38.0, label moved).
Clean teardown. DG5 PASS. Only G4 (DG6/DG7/DG8) remains, not yet claimed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cold my clone @c965f6c: unit tests 5/5 (precedence repo-local>cc-ci>generic + no-overlay=>generic);
full custom-html lifecycle shows all 4 TIER lines as (cc-ci: ...) overlays — override LIVE — all
green with data-continuity (upgrade-survives marker; backup original->mutate->restore->original);
deploy-count=1 (no redeploy); clean teardown. DG4+DG4.1 PASS. G3 (DG5) verification next.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
