When a DEPS-declaring recipe's setup_custom_tests fails, its @requires_deps (SSO/OIDC)
tests skip; a skip-only pytest file exits 0 so the run previously reported overall=0
(GREEN) while the only SSO test never ran (violates P7). Fix preserves generic-tier
failure-isolation but corrects the green SIGNAL:
- conftest.pytest_collection_modifyitems counts skipped requires_deps tests and appends
to $CCCI_DEPS_SKIP_REPORT.
- run_recipe_ci: sums the count, surfaces it in RUN SUMMARY, and new pure predicate
sso_dep_unverified(declared, deps_ready, skipped) flips overall=1.
- 7 new unit tests (tests/unit/test_f211_sso_skip.py).
Verified deploy-free (rate-limit-independent): 35/35 unit PASS; cold real-test proof on
lasuite-docs test_oidc_with_keycloak.py -> 1 skipped + skip-report==1 -> orchestrator
would set overall=1. Full e2e deferred until Docker Hub rate limit lifts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- tests/cryptpad/PARITY.md: parity table for health_check.py (ported);
oidc_login.py documented as authentik-deferred (cross-recipe; needs Q2.2 enrollment).
- tests/cryptpad/functional/test_health_check.py: parity port, SOURCE comment present.
- tests/cryptpad/functional/test_api_config.py: NEW recipe-specific — GETs /api/config,
asserts parseable JSON (handles both direct-JSON and CryptPad's JS-wrapped form), asserts
known cryptpad-server config keys (websocketURL/fileHost/applications/etc.). Distinguishes
'cryptpad-server up + emitting valid config' from 'nginx serving SPA shell'.
- tests/cryptpad/playwright/test_pad_create.py: NEW Playwright create-and-read-back. Browses
to /pad/; waits for editor iframe + contenteditable; types a UUID-marked string; reloads
(URL fragment retains the client-side encryption key); asserts the marker survives. This
is the plan §4.3-prescribed CryptPad-specific test ('use Playwright, not bare curl').
- STATUS-2 updated to record Q2 Adversary PASS (REVIEW-2 ## Q2 — PASS).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per REVIEW-2 ## Q2 FAIL @2026-05-28 (F2-5 dep teardown leak + F2-6 cold install flake + F2-7
SSO setup keycloak-hardcoded):
F2-5 closed by commit c6e94af: teardown_deps now uses verify=True so residuals raise; failures
propagate to orchestrator exit code + run summary. Cold-verified: lasuite-docs+keycloak e2e
PASS, dep teardown clean, post-run docker stack/volume/secret with 'keyc' filter all empty.
This also explained my Q3.1 flake — the leaked Q2.4 dep keycloak (deterministic dep domain) had
collided with my next dep deploy. With F2-5 fixed, that class of cross-run collision is
impossible (teardown now raises if it leaks, so the run fails BEFORE the next one starts).
F2-7 acknowledged: setup_keycloak_realm is keycloak-specific; authentik would need parallel
backend. Logged for Q2.2/Q5.
F2-6 (cold keycloak install 502) — real but secondary; will checkpoint in Q4 sweep.
Side-effect: Q3.1 partial also landed (PARITY.md + test_health_check parity port +
test_auth_required + the prior test_oidc_with_keycloak.py as Q3.1 third specific test).
Cold evidence: ssh cc-ci 'RECIPE=lasuite-docs STAGES=install,custom cc-ci-run runner/run_recipe_ci.py'
deploy-count=2 (expect 2), all 5 assertions PASS, dep teardown clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per REVIEW-2 ## Q1 — PASS @2026-05-28: F2-3 + F2-4 closed; cold e2e on Adversary clone all 5
stages PASS; deploy-count=1; HC1 non-vacuous; teardown sacred; NO VETO. Builder may advance to Q2.
Q2.1 keycloak in flight: first attempt hit 502 from /realms/master at 600s; bumped DEPLOY_TIMEOUT
+ HTTP_TIMEOUT to 900s in tests/keycloak/recipe_meta.py.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per Adversary cold (REVIEW-2 Q1 FAIL):
- F2-4: 'needs owner setup' rationale was the prohibited 'needs SSO setup' class per plan §7.1.
Fixed by tests/n8n/functional/test_workflow_roundtrip.py (commit fc89552) — the plan §4.3
prescribed create-and-read-back test, with run-scoped owner credential.
- F2-3: page.goto raised PlaywrightError outside the retry loop on net::ERR_*. Fixed by wrapping
page.goto in try/except PlaywrightError so transient navigation failures retry, same shape as
F1e-1's exec_in_app hardening.
Cold-verifiable: ssh cc-ci 'RECIPE=n8n cc-ci-run runner/run_recipe_ci.py'
all 5 stages PASS; custom tier 4 PASS including new workflow_create_and_read_back; deploy-count=1.
Keycloak Q2.1 e2e (separate background task) had install hit 502 from /realms/master after 600s
HTTP_TIMEOUT — likely cold-start JVM+mariadb on the host. Will investigate post Q1 verdict.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Q1.1 custom-html: parity port + 2 NEW recipe-specific + playwright (Q0 PASS evidence stands).
Q1.2 n8n: parity port + 2 NEW recipe-specific (rest_settings, login_state — both reject the
'n8n is starting up' placeholder, so non-vacuous). install overlay now polls page.goto until
status==200 (absorbs n8n's /healthz-200-before-/-route-registered boot race).
Q1.3 n8n backup data-integrity: covered by Phase-1d/1e lifecycle overlay pattern (volume marker
survives backup→mutate→restore — PASSED in Q1.2 e2e).
Q1.4 CLAIMED.
Cold evidence: ssh cc-ci 'RECIPE=n8n cc-ci-run runner/run_recipe_ci.py'
all 5 stages PASS, deploy-count=1, head_ref==chaos-version (HC1 non-vacuous), version moved
3.1.0+2.9.4 -> 3.2.0+2.20.6.
Q1.2 note: deferred 'create workflow via API' from plan §4.3 in favor of /rest/settings +
/rest/login JSON-shape assertions (equally non-vacuous, no owner-setup state to manage); recorded
in BACKLOG-2 + JOURNAL-2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per Adversary cold (REVIEW-2 "Q0 FAIL"), F2-1 mechanical regression: the Phase-1e HC2 unit test
asserted custom_tests('custom-html', rl) == [] when the real custom-html dir had no functional/
tests. Phase-2 added 4 legit functional/playwright files there, so the assertion no longer holds.
Behavior is correct; the test fixture was brittle.
Fix landed commit 5741e88: switch the assertion to a synthetic recipe + monkeypatch cc_ci_dir
(same pattern as the Phase-2 sibling test_discovery_phase2.py). Cold re-run: 21/21 PASS.
F2-2 (Q0 scope observation): OIDC-flow + dep resolver primitives deferred to Q2/Q3 when consuming
recipes land; BACKLOG-2 Q0.4 explicitly tracks this — acknowledged in STATUS-2 gate text.
Q0 RE-CLAIMED, awaiting Adversary re-verify.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
