feat(2): refactor — SSO-dep plan refinement (deps AFTER generic + setup_custom_tests + failure isolation)

Per operator-2026-05-28 SSO-dep plan (plan-sso-dep-testing.md). Substantial orchestrator
restructuring:

NEW LIFECYCLE ORDER:
  1. Recipe deploy ALONE (no deps).
  2. install / upgrade / backup / restore — recipe-only generic tiers.
  3. setup_custom_tests step (NEW):
     a. Deploy each declared dep + provision realm/client/test-user via harness.sso.
     b. Write $CCCI_DEPS_FILE in dict shape {dep_recipe: {domain, realm, client_id, client_secret,
        admin_user, admin_password, discovery_url, token_url, ...}}.
     c. Run tests/<recipe>/setup_custom_tests.sh hook (jq-readable; wires OIDC env via abra
        secret insert + .env edits + in-place 'abra app deploy --force --chaos').
  4. CUSTOM tier with deps-ready flag; @pytest.mark.requires_deps tests skip with
     'deps-not-ready: <reason>' when setup_custom_tests fails. NON-deps custom tests still run
     normally — FAILURE ISOLATION (a DoD item per plan).
  5. Teardown: recipe first, deps in reverse declaration order.

Harness changes:
- runner/run_recipe_ci.py: deps deploy moves from BEFORE recipe deploy to AFTER restore tier.
  Adds _enrich_deps_with_sso() + _run_setup_custom_tests_hook(). DG4.1 generalised to
  'one abra app new per app' (recipe + each dep); in-place redeploys (\--force) don't count.
- runner/harness/deps.py: write_run_state + load_run_state accept dict OR list shape;
  deps_as_dict() coerces either to a recipe→entry map.
- runner/harness/sso.py: admin_password_inside() public re-export.
- tests/conftest.py: deps_creds fixture (full creds dict); deps_apps fixture flattens to
  recipe→domain string. pytest_collection_modifyitems hook skips
  \@pytest.mark.requires_deps tests when CCCI_DEPS_READY=0.
  pytest_configure registers the marker.

Recipe content:
- tests/lasuite-docs/setup_custom_tests.sh: NEW hook reads $CCCI_DEPS_FILE via jq;
  inserts oidc_rpcs secret at BUMPED version (v1→v2) since abra app new -S generates v1 first
  and Swarm forbids overwriting; updates SECRET_OIDC_RPCS_VERSION in .env; writes 9 OIDC env
  vars (REALM/DISCOVERY/AUTH/TOKEN/USERINFO/LOGOUT/JWKS/CLIENT_ID/SCOPES); ensures trailing
  newline on .env so writes don't concatenate (caught a 'TIMEOUT=900OIDC_REALM=...' bug);
  triggers in-place 'abra app deploy --force --chaos --no-input'.
- tests/lasuite-docs/functional/test_oidc_with_keycloak.py: refactored to consume deps_creds
  fixture (no longer calls setup_keycloak_realm itself — the orchestrator does it in
  setup_custom_tests). Marked \@pytest.mark.requires_deps.

Cold-verifiable on cc-ci (log /root/ccci-refactor-lasuite-r5.log):
  RECIPE=lasuite-docs STAGES=install,custom cc-ci-run runner/run_recipe_ci.py
  install: PASS, custom: 3 PASS incl. test_oidc_password_grant_against_dep_keycloak.
  deploy-count = 2 (expect 2) — DG4.1 generalised holds.
  Smoke regression: RECIPE=custom-html STAGES=install,custom → 5 PASS, deploy-count=1.

Closes DEFERRED.md #5 (lasuite-docs OIDC parity ports via this plan).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-28 19:11:42 +01:00
parent 5832da4fd1
commit 41ede13042
7 changed files with 386 additions and 104 deletions

View File

@ -6,6 +6,38 @@ op, the **generic still runs alongside it** (the floor is never silently lost).
meaningful on **any** recipe immediately (zero config), and adding recipe-specific coverage is a thin
overlay that adds, it doesn't subtract.
## Architectural invariant — generic-first, custom-additive (read this first)
This is the load-bearing principle of the whole test architecture. If you're maintaining cc-ci a
year from now, this is the one rule that should still hold.
- **Generic tests are simple and easily runnable.** They are recipe-agnostic, depend only on the
recipe being deployable (install / upgrade / backup / restore against the recipe alone), and
ship as the floor for every recipe. No SSO provider, no external deps, no per-recipe state
scaffolding — just "does this recipe deploy and lifecycle work?"
- **Generic must not depend on custom.** A custom test or a custom-tests setup (e.g. SSO/OIDC dep
provisioning) **can never be a precondition for the generic tier to pass.** Concretely: the
orchestrator runs all generic tiers (install → upgrade → backup → restore) against the recipe
**alone, with no deps deployed**, then runs the `setup_custom_tests` step (deps + post-deps
wiring) only after — and a failure there is **isolated** to the custom tier (tests tagged
`@pytest.mark.requires_deps` skip with reason `"deps-not-ready"`; generic tier reports
normally). See `cc-ci-plan/plan-sso-dep-testing.md` for the SSO-dep specifics.
- **Custom tests are the thoroughness layer — and they cost more to maintain.** They're more
thorough (authenticated APIs, multi-app flows, version-specific browser selectors, helper
scripts, state-management) and *therefore* take more maintenance: an SSO provider's admin API
changes, a recipe's app-launch URL contract shifts between versions, a Socket.IO primitive
needs to track upstream — these are real ongoing costs that the generic tier deliberately
doesn't carry.
- **A future maintainer can choose to focus on the generic tier alone** and still get meaningful
signal: every enrolled recipe gets *some* CI coverage from the generic floor, and the
custom-additive layer can be scaled down or paused without breaking that floor. The choice of
*how much* per-recipe depth to maintain is open to whoever owns cc-ci later — generic-only is
a valid permanent operating mode.
If anything in this codebase ever asks you to make generic depend on custom (or to put a custom
precondition before a generic tier), that's the signal it's drifted off the invariant — push back
and restore the separation.
## The model: tiers against one shared deployment
A run is a sequence of **tiers**. The orchestrator (`runner/run_recipe_ci.py`) deploys the app