Commit Graph

10 Commits

Author SHA1 Message Date
1b8d26b504 feat(2w): W0.2 live-warm keycloak dep mode in orchestrator (WC1)
- runner/harness/warm.py: stable-domain scheme (warm-<recipe>), is_warm_up
  probe, live_app_hexes scan, per-run realm_for naming, reap_orphan_realms.
- run_recipe_ci.py: split declared deps into live-warm (shared provider +
  per-run realm, no deploy, realm deleted at teardown) vs cold (co-deploy).
  Warm path used only when provider is up; cold fallback otherwise. Reap
  orphan realms at run start (concurrency-safe). deploy-count excludes warm
  deps. Realm naming now per-run namespaced (<parent>-<6hex>).
- dependent tests assert the namespaced realm pattern (stronger than ==parent).

Live proof on warm keycloak: realm create -> password-grant JWT -> discovery
issuer -> delete(idempotent) -> reap(keeps live hex, deletes orphan): PASS.
43 unit pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-28 23:26:02 +01:00
cd25f52eae feat(2): close DEFERRED #5 — lasuite-docs OIDC parity + create-a-doc (§4.3) cold green
Per orchestrator's SSO-dep plan + the refactor in 41ede13, DEFERRED.md entry #5 (lasuite-docs
OIDC parity ports + create-a-doc) closes by execution.

- tests/lasuite-docs/functional/test_oidc_login.py: parity port of recipe-maintainer
  oidc_login.py. Anonymous GET /api/v1.0/users/me/ → 302 to keycloak realm OR 401/403;
  password-grant token → 200 with user.email matching the provisioned test user.
- tests/lasuite-docs/functional/test_create_doc.py: plan §4.3 prescribed create-an-object +
  read-it-back. POST /api/v1.0/documents/ with OIDC Bearer → captured id; GET
  /api/v1.0/documents/<id>/ → asserts id+title round-trip.

Both marked \@pytest.mark.requires_deps; skipped with 'deps-not-ready' if setup_custom_tests
fails (failure isolation per plan-sso-dep-testing.md §4).

Cold-verifiable: ssh cc-ci 'RECIPE=lasuite-docs STAGES=install,custom cc-ci-run runner/run_recipe_ci.py'
  install: 2 PASS; custom: 5 PASS incl. test_oidc_login_via_keycloak +
  test_create_doc_and_read_back; deploy-count=2 (recipe + keycloak dep).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 19:26:54 +01:00
41ede13042 feat(2): refactor — SSO-dep plan refinement (deps AFTER generic + setup_custom_tests + failure isolation)
Per operator-2026-05-28 SSO-dep plan (plan-sso-dep-testing.md). Substantial orchestrator
restructuring:

NEW LIFECYCLE ORDER:
  1. Recipe deploy ALONE (no deps).
  2. install / upgrade / backup / restore — recipe-only generic tiers.
  3. setup_custom_tests step (NEW):
     a. Deploy each declared dep + provision realm/client/test-user via harness.sso.
     b. Write $CCCI_DEPS_FILE in dict shape {dep_recipe: {domain, realm, client_id, client_secret,
        admin_user, admin_password, discovery_url, token_url, ...}}.
     c. Run tests/<recipe>/setup_custom_tests.sh hook (jq-readable; wires OIDC env via abra
        secret insert + .env edits + in-place 'abra app deploy --force --chaos').
  4. CUSTOM tier with deps-ready flag; @pytest.mark.requires_deps tests skip with
     'deps-not-ready: <reason>' when setup_custom_tests fails. NON-deps custom tests still run
     normally — FAILURE ISOLATION (a DoD item per plan).
  5. Teardown: recipe first, deps in reverse declaration order.

Harness changes:
- runner/run_recipe_ci.py: deps deploy moves from BEFORE recipe deploy to AFTER restore tier.
  Adds _enrich_deps_with_sso() + _run_setup_custom_tests_hook(). DG4.1 generalised to
  'one abra app new per app' (recipe + each dep); in-place redeploys (\--force) don't count.
- runner/harness/deps.py: write_run_state + load_run_state accept dict OR list shape;
  deps_as_dict() coerces either to a recipe→entry map.
- runner/harness/sso.py: admin_password_inside() public re-export.
- tests/conftest.py: deps_creds fixture (full creds dict); deps_apps fixture flattens to
  recipe→domain string. pytest_collection_modifyitems hook skips
  \@pytest.mark.requires_deps tests when CCCI_DEPS_READY=0.
  pytest_configure registers the marker.

Recipe content:
- tests/lasuite-docs/setup_custom_tests.sh: NEW hook reads $CCCI_DEPS_FILE via jq;
  inserts oidc_rpcs secret at BUMPED version (v1→v2) since abra app new -S generates v1 first
  and Swarm forbids overwriting; updates SECRET_OIDC_RPCS_VERSION in .env; writes 9 OIDC env
  vars (REALM/DISCOVERY/AUTH/TOKEN/USERINFO/LOGOUT/JWKS/CLIENT_ID/SCOPES); ensures trailing
  newline on .env so writes don't concatenate (caught a 'TIMEOUT=900OIDC_REALM=...' bug);
  triggers in-place 'abra app deploy --force --chaos --no-input'.
- tests/lasuite-docs/functional/test_oidc_with_keycloak.py: refactored to consume deps_creds
  fixture (no longer calls setup_keycloak_realm itself — the orchestrator does it in
  setup_custom_tests). Marked \@pytest.mark.requires_deps.

Cold-verifiable on cc-ci (log /root/ccci-refactor-lasuite-r5.log):
  RECIPE=lasuite-docs STAGES=install,custom cc-ci-run runner/run_recipe_ci.py
  install: PASS, custom: 3 PASS incl. test_oidc_password_grant_against_dep_keycloak.
  deploy-count = 2 (expect 2) — DG4.1 generalised holds.
  Smoke regression: RECIPE=custom-html STAGES=install,custom → 5 PASS, deploy-count=1.

Closes DEFERRED.md #5 (lasuite-docs OIDC parity ports via this plan).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 19:11:42 +01:00
874bfbb915 feat(2): Q3.1 partial — lasuite-docs PARITY + health_check + auth_required (Q2.4 still passes)
- tests/lasuite-docs/PARITY.md: parity table for health_check.py (ported);
  oidc_login.py + upload_conversion.py documented as Q3.1 follow-up needing OIDC env wiring;
  ≥2 recipe-specific tests rationale (test_oidc_with_keycloak + test_auth_required).
- tests/lasuite-docs/functional/test_health_check.py: parity port of
  recipe-info/lasuite-docs/tests/health_check.py — HTTP 200/301/302 from root.
- tests/lasuite-docs/functional/test_auth_required.py: NEW recipe-specific —
  GET /api/v1.0/users/me/ asserts 401/403 (auth required). Non-vacuous: distinguishes
  correctly-wired OIDC gate from anonymous access (200), missing route (404), broken (5xx).

The Q2.4 acceptance test (test_oidc_with_keycloak.py) continues to verify the dep resolver +
SSO harness against the per-run keycloak dep (F2-5 fix verified cold; see ccci-f25-verify.log).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 09:21:00 +01:00
9e88741864 feat(2): Q2.4 acceptance — lasuite-docs + keycloak dep + OIDC password grant (cold green)
- tests/lasuite-docs/recipe_meta.py: DEPS = ['keycloak'] declares the SSO provider dep.
  Orchestrator deploys a per-run keycloak BEFORE lasuite-docs (Q2.3 dep resolver) and tears it
  down AFTER in finally.
- tests/lasuite-docs/functional/test_oidc_with_keycloak.py: Q2 gate acceptance test.
  - Asserts deps_apps['keycloak'] is the per-run dep domain.
  - Calls harness.sso.setup_keycloak_realm to create realm/client/test-user idempotently.
  - GET /.well-known/openid-configuration; asserts issuer = https://<kc>/realms/lasuite-docs.
  - harness.sso.oidc_password_grant: password-grant flow; asserts the JWT iss/azp/typ/exp.
  - Non-vacuous: each step uses real per-run-generated creds (class-B per §4.4-B), would fail
    on broken admin API / token endpoint / wrong claims.

Cold-verifiable on cc-ci (log /root/ccci-q24-lasuite-keycloak.log):
  RECIPE=lasuite-docs STAGES=install,custom cc-ci-run runner/run_recipe_ci.py
  ===== DEPS: ['keycloak'] =====
    dep: deploying keycloak -> keyc-c12afe.ci.commoninternet.net
    dep: keycloak ready @ keyc-c12afe.ci.commoninternet.net
  ===== TIER: install =====   2 PASS (generic + cc-ci overlay)
  ===== TIER: custom =====    1 PASS (test_oidc_password_grant_against_dep_keycloak)
  ===== DEPS teardown =====
  ===== RUN SUMMARY =====
  deploy-count = 2 (expect 2)   # 1 parent + 1 dep

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 08:08:11 +01:00
47f7cb47c2 fix(2): F2-3 systemic — harness.browser.goto_with_retry; applied to all install overlays
Phase 2 lesson from F2-3 (n8n install Playwright flake on net::ERR_NETWORK_CHANGED): every
install overlay that does page.goto needs the same try/except PlaywrightError + status retry.
Centralize in runner/harness/browser.py::goto_with_retry; apply to ALL install overlays.

- runner/harness/browser.py: shared helper. Polls page.goto until status in accept_statuses;
  catches PlaywrightError (net::ERR_*) as a retryable signal, not a failure. Raises AssertionError
  with last_status + last_err diagnostic only on deadline expiry.
- tests/custom-html/test_install.py: now uses goto_with_retry (200 only, wait_until=load).
- tests/custom-html/playwright/test_browser_smoke.py: same.
- tests/n8n/test_install.py: replaced inline retry loop with goto_with_retry (200, 304).
- tests/keycloak/test_install.py: goto_with_retry for admin console (200, 302, 303; 45s goto).
- tests/cryptpad/test_install.py: goto_with_retry (200, 304; 60s goto, wait_until=load).
- tests/lasuite-docs/test_install.py: goto_with_retry (200, 301, 302; 60s goto).

Cold-verifiable: ssh cc-ci 'RECIPE=custom-html cc-ci-run runner/run_recipe_ci.py'
  all 5 stages PASS (including the install overlay that flaked in the deps_smoke run),
  deploy-count=1, head_ref=8a026066==chaos-version=8a026066 (HC1 non-vacuous).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 07:46:34 +01:00
b7e6cbd7be feat(1e): HC3 additive generic + op/assertion split (orchestrator owns the op)
- orchestrator: per mutating tier, run optional pre-op seed hook (ops.py pre_<op>) → perform the op
  ONCE (harness-owned) → run generic assertion (unless opted out) AND overlay assertion, both against
  the shared post-op deployment. Op results passed op→assertion via run-scoped CCCI_OP_STATE_FILE.
- opt-out: CCCI_SKIP_GENERIC / CCCI_SKIP_GENERIC_<OP> / recipe_meta.SKIP_GENERIC (declarative).
- generic.py: split do_* into op primitives (perform_upgrade/backup/restore) + assertions
  (assert_upgraded/backup_artifact/restore_healthy) reading op_state(); deployed_identity now returns
  {version,image,chaos} (chaos label ready for HC1).
- generic test_<op>.py + all 6 recipe overlays migrated to assertion-only; pre-op seeding moved to
  per-recipe ops.py (pre_upgrade/pre_backup/pre_restore). install overlays unchanged (no op).
- deploy-count stays 1 (op primitives never call deploy_app). lint PASS; 8 unit tests PASS on cc-ci.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 03:12:04 +01:00
afd75a48db feat(1d): migrate keycloak/cryptpad/matrix-synapse/n8n/lasuite-docs overlays to deploy-once contract (DG7)
Mechanical port to the assertion-only contract (no softened/skipped assertions): install uses
live_app + generic.assert_serving (extend) + the recipe's http/playwright/api checks; upgrade seeds
its data marker then generic.do_upgrade + asserts survival; backup/restore split into test_backup.py
(seed->do_backup->mutate) + new test_restore.py (do_restore->assert original). Recipe-specifics
preserved verbatim (keycloak realm+admin-console+kc_admin, matrix/lasuite db-service psql markers,
cryptpad/n8n volume markers). No recipe now double-deploys under the deploy-once orchestrator.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 01:32:53 +01:00
2cede01ed7 style(1b): auto-format + lint-clean the whole codebase (RL1)
Mechanical, semantics-preserving cleanup so the codebase passes the new lint stage:
- ruff format: all 32 Python files (wraps long signatures, normalizes quotes/blank lines).
- nixpkgs-fmt: modules/drone-runner.nix.
- shfmt (-i 2 -ci): scripts/*.sh.

Lint fixes (reviewed, behavior-preserving — no test weakened):
- ruff SIM105: try/except-pass -> contextlib.suppress (abra.py app_config rm; lifecycle.py janitor).
- ruff SIM115: open().read() -> with open() (run_recipe_ci.py redaction-values + gitea-token).
- statix: merge repeated sops `secrets.*` keys into one `secrets = { ... }` (comments kept);
  empty fn pattern `{ ... }:` -> `_:` (packages.nix).
- deadnix: drop unused lambda args (flake `self`; configuration.nix `lib`; overlay `final` -> `_`).

Verified on cc-ci: `scripts/lint.sh` -> lint: PASS; nixosConfigurations.cc-ci evaluates;
all Python byte-compiles. The deployed bridge/dashboard/runner source changes hash (reformat),
so cc-ci will be rebuilt to the new closure in W2 before the cold D1-D10 re-verification.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 20:52:05 +01:00
daf67e53b9 M6.5: enroll lasuite-docs (recipe #5, multi-service + S3/MinIO) — install verified green
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone Build is passing
9-service stack (frontend/backend/celery/y-provider/docspec/postgres/redis/minio/nginx) converges
9/9 and serves the SPA; install 2 passed on host. Root-caused a deploy timeout: cold-pulling ~9
large images exceeds abra's default 300s convergence TIMEOUT -> bumped to 900 via EXTRA_ENV (the
generic per-recipe mechanism, no harness surgery). upgrade/backup use a postgres marker (docs/docs)
exercising the pg_backup.sh DB-dump hook; verifying next.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 06:32:23 +01:00