The M2 keycloak enrollment made the canonical collision-free at the DOMAIN layer
(warm-canon-keycloak vs warm-keycloak) but warm STATE stayed keyed by bare recipe:
warmsnap.app_dir("keycloak") resolved both the live-warm reconciler's last_good and
the data-warm canonical's canonical.json + snapshot/ into /var/lib/ci-warm/keycloak/.
snapshot() atomically REPLACES that slot, so the two deployments destroyed each
other's known-good; restore() then raised SnapshotError (fails closed, no cross-stack
data write). Worst case: a sweep promote landing inside the reconciler's
snapshot->wait_healthy window makes its rollback restore() raise after
abra.undeploy(live), leaving the shared OIDC provider undeployed.
Fix: canonical.canonical_ns() is now the single namespace from which BOTH the
canonical's domain and its warm-state slot derive, so they cannot drift apart. A
live-warm provider gets ns "canon-<recipe>": domain warm-canon-keycloak (unchanged)
and slot /var/lib/ci-warm/canon-keycloak/. Every other recipe keeps ns "<recipe>" —
zero on-disk change for the 15 existing canonicals, and no migration on cc-ci
(keycloak's canonical was never seeded: its dir holds only last_good).
- warmsnap: functions take a SLOT, not a recipe; add live_slot(); meta records "slot".
- warmsnap: _assert_slot_not_foreign() refuses to snapshot/restore a slot recorded
against a different domain -- defence in depth, naming-scheme-independent, fails
before the destructive swap rather than at the next restore.
- canonical: registry_path/seed_canonical/prune_stale go through canonical_slot().
- prune_stale: the "reconciler dirs are never pruned" invariant is now STRUCTURAL --
<recipe>/ never gains a canonical.json, so de-enrolling keycloak can no longer
rmtree the reconciler's last_good (consequence 4).
- warm_reconcile: last_good + snapshot/restore go through warmsnap.live_slot().
- run_recipe_ci: canonical rollback restores from canonical_slot(recipe).
- Correct the two comments that claimed the deployments "can never touch each other".
Tests: 10 new (slot disjointness for every WARM_DOMAINS recipe, slot<->stack 1:1,
registry not in the reconciler dir, prune spares last_good, foreign-slot refusal in
both snapshot and restore). Unit suite 315 -> 325, no regressions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FS8p1esg57UAC69riNvuBX
canonical_domain() routes any recipe in warm.WARM_DOMAINS (keycloak) to a distinct warm-canon-<recipe>
domain so the data-warm canonical promote can never collide with the live-warm OIDC provider at
warm-keycloak. keycloak WARM_CANONICAL=True (full canonical coverage without risking live SSO).
keycloak is the always-on shared OIDC dep provider at warm-keycloak.ci..., the SAME stable domain a
data-warm canonical would use → the sweep's promote would collide with the live provider that
lasuite-*/drone depend on. keycloak is kept current by roll_warm_infra (WC1.1) instead.
WARM_CANONICAL=False; exception recorded in DECISIONS. Enrolled set now 20.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
WARM_CANONICAL=True added to every recipe in cc-ci-plan/used-recipes.md (20 weekly +
uptime-kuma external). enrolled_recipes() now returns all 21. Test fixtures
(custom-html-*-bad, concurrency, regression) intentionally left unenrolled.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
harness.meta.HookCtx (frozen): .domain, .base_url, .meta (RecipeMeta), .deps
(provisioned dep creds from $CCCI_DEPS_FILE or None), .op (current lifecycle op
or None); built via meta.hook_ctx() at each hook call site.
All recipe callables now take ctx: EXTRA_ENV(ctx), UPGRADE_EXTRA_ENV(ctx),
READY_PROBE(ctx), BACKUP_VERIFY(ctx), SCREENSHOT(page, ctx), ops.py pre_<op>(ctx).
Dict-valued EXTRA_ENV/UPGRADE_EXTRA_ENV unchanged (only the callable signature
moved). Call sites converted: deploy_app env shaping, perform_upgrade,
wait_ready_probes (gains op=), _perform_op BACKUP_VERIFY, screenshot.capture,
_run_pre_hook.
Legacy signatures fail FAST with a clear migration message: the registry carries
hook_params per hook key, enforced at meta.load() (MetaError names the old vs new
signature); ops.py pre-op hooks get the same check at the orchestrator call site
(meta.check_hook_signature) — no silent TypeError mid-run.
Migrated every in-repo user mechanically (17 ops.py files; cryptpad/lasuite-*/
mailu EXTRA_ENV; mumble+lasuite-drive READY_PROBE; ghost/discourse BACKUP_VERIFY)
— seeded values, probes and assertions byte-identical (domain -> ctx.domain;
keycloak pre_restore's meta arg -> ctx.meta).
Unit tests: hook_ctx field contract, ctx.deps from the run deps file, legacy-
signature MetaError (READY_PROBE/EXTRA_ENV/SCREENSHOT + pre-op checker), ctx
signatures accepted. Docs table regenerated (signature docs in key docs).
Verified on cc-ci: cc-ci-run -m pytest tests/unit -q -> 180 passed; scripts/lint.sh -> PASS.
Push builds have been RED on the lint step since ~build 209 from accumulated
formatting drift. This is the mechanical cleanup: ruff format + ruff --fix
(UP038 isinstance unions, SIM105 contextlib.suppress, UP031 f-strings, SIM115
tempfile context manager), shfmt -i 2 -ci, nixpkgs-fmt/statix/deadnix (merged
attrsets, dropped unused lib args), yamllint, and shell quoting fixes in
tests/lasuite-docs/setup_custom_tests.sh. No behaviour changes intended;
lint: PASS, unit tests: 138 passed.
Phase 2 lesson from F2-3 (n8n install Playwright flake on net::ERR_NETWORK_CHANGED): every
install overlay that does page.goto needs the same try/except PlaywrightError + status retry.
Centralize in runner/harness/browser.py::goto_with_retry; apply to ALL install overlays.
- runner/harness/browser.py: shared helper. Polls page.goto until status in accept_statuses;
catches PlaywrightError (net::ERR_*) as a retryable signal, not a failure. Raises AssertionError
with last_status + last_err diagnostic only on deadline expiry.
- tests/custom-html/test_install.py: now uses goto_with_retry (200 only, wait_until=load).
- tests/custom-html/playwright/test_browser_smoke.py: same.
- tests/n8n/test_install.py: replaced inline retry loop with goto_with_retry (200, 304).
- tests/keycloak/test_install.py: goto_with_retry for admin console (200, 302, 303; 45s goto).
- tests/cryptpad/test_install.py: goto_with_retry (200, 304; 60s goto, wait_until=load).
- tests/lasuite-docs/test_install.py: goto_with_retry (200, 301, 302; 60s goto).
Cold-verifiable: ssh cc-ci 'RECIPE=custom-html cc-ci-run runner/run_recipe_ci.py'
all 5 stages PASS (including the install overlay that flaked in the deps_smoke run),
deploy-count=1, head_ref=8a026066==chaos-version=8a026066 (HC1 non-vacuous).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- orchestrator: per mutating tier, run optional pre-op seed hook (ops.py pre_<op>) → perform the op
ONCE (harness-owned) → run generic assertion (unless opted out) AND overlay assertion, both against
the shared post-op deployment. Op results passed op→assertion via run-scoped CCCI_OP_STATE_FILE.
- opt-out: CCCI_SKIP_GENERIC / CCCI_SKIP_GENERIC_<OP> / recipe_meta.SKIP_GENERIC (declarative).
- generic.py: split do_* into op primitives (perform_upgrade/backup/restore) + assertions
(assert_upgraded/backup_artifact/restore_healthy) reading op_state(); deployed_identity now returns
{version,image,chaos} (chaos label ready for HC1).
- generic test_<op>.py + all 6 recipe overlays migrated to assertion-only; pre-op seeding moved to
per-recipe ops.py (pre_upgrade/pre_backup/pre_restore). install overlays unchanged (no op).
- deploy-count stays 1 (op primitives never call deploy_app). lint PASS; 8 unit tests PASS on cc-ci.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
hedgedoc mirrored to recipe-maintainers/hedgedoc with probe PR #1; add it to the bridge poll list so
!testme triggers the full generic suite (no cc-ci/repo-local overlay -> pure generic). Rebuild pending.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mechanical port to the assertion-only contract (no softened/skipped assertions): install uses
live_app + generic.assert_serving (extend) + the recipe's http/playwright/api checks; upgrade seeds
its data marker then generic.do_upgrade + asserts survival; backup/restore split into test_backup.py
(seed->do_backup->mutate) + new test_restore.py (do_restore->assert original). Recipe-specifics
preserved verbatim (keycloak realm+admin-console+kc_admin, matrix/lasuite db-service psql markers,
cryptpad/n8n volume markers). No recipe now double-deploys under the deploy-once orchestrator.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
keycloak+mariadb deployed via only tests/keycloak/recipe_meta.py + test_install.py
(realm health + Playwright admin login). Proves recipe-agnostic enrollment (D5).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
