feat(2): refactor — SSO-dep plan refinement (deps AFTER generic + setup_custom_tests + failure isolation)
Per operator-2026-05-28 SSO-dep plan (plan-sso-dep-testing.md). Substantial orchestrator
restructuring:
NEW LIFECYCLE ORDER:
1. Recipe deploy ALONE (no deps).
2. install / upgrade / backup / restore — recipe-only generic tiers.
3. setup_custom_tests step (NEW):
a. Deploy each declared dep + provision realm/client/test-user via harness.sso.
b. Write $CCCI_DEPS_FILE in dict shape {dep_recipe: {domain, realm, client_id, client_secret,
admin_user, admin_password, discovery_url, token_url, ...}}.
c. Run tests/<recipe>/setup_custom_tests.sh hook (jq-readable; wires OIDC env via abra
secret insert + .env edits + in-place 'abra app deploy --force --chaos').
4. CUSTOM tier with deps-ready flag; @pytest.mark.requires_deps tests skip with
'deps-not-ready: <reason>' when setup_custom_tests fails. NON-deps custom tests still run
normally — FAILURE ISOLATION (a DoD item per plan).
5. Teardown: recipe first, deps in reverse declaration order.
Harness changes:
- runner/run_recipe_ci.py: deps deploy moves from BEFORE recipe deploy to AFTER restore tier.
Adds _enrich_deps_with_sso() + _run_setup_custom_tests_hook(). DG4.1 generalised to
'one abra app new per app' (recipe + each dep); in-place redeploys (\--force) don't count.
- runner/harness/deps.py: write_run_state + load_run_state accept dict OR list shape;
deps_as_dict() coerces either to a recipe→entry map.
- runner/harness/sso.py: admin_password_inside() public re-export.
- tests/conftest.py: deps_creds fixture (full creds dict); deps_apps fixture flattens to
recipe→domain string. pytest_collection_modifyitems hook skips
\@pytest.mark.requires_deps tests when CCCI_DEPS_READY=0.
pytest_configure registers the marker.
Recipe content:
- tests/lasuite-docs/setup_custom_tests.sh: NEW hook reads $CCCI_DEPS_FILE via jq;
inserts oidc_rpcs secret at BUMPED version (v1→v2) since abra app new -S generates v1 first
and Swarm forbids overwriting; updates SECRET_OIDC_RPCS_VERSION in .env; writes 9 OIDC env
vars (REALM/DISCOVERY/AUTH/TOKEN/USERINFO/LOGOUT/JWKS/CLIENT_ID/SCOPES); ensures trailing
newline on .env so writes don't concatenate (caught a 'TIMEOUT=900OIDC_REALM=...' bug);
triggers in-place 'abra app deploy --force --chaos --no-input'.
- tests/lasuite-docs/functional/test_oidc_with_keycloak.py: refactored to consume deps_creds
fixture (no longer calls setup_keycloak_realm itself — the orchestrator does it in
setup_custom_tests). Marked \@pytest.mark.requires_deps.
Cold-verifiable on cc-ci (log /root/ccci-refactor-lasuite-r5.log):
RECIPE=lasuite-docs STAGES=install,custom cc-ci-run runner/run_recipe_ci.py
install: PASS, custom: 3 PASS incl. test_oidc_password_grant_against_dep_keycloak.
deploy-count = 2 (expect 2) — DG4.1 generalised holds.
Smoke regression: RECIPE=custom-html STAGES=install,custom → 5 PASS, deploy-count=1.
Closes DEFERRED.md #5 (lasuite-docs OIDC parity ports via this plan).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@ -6,6 +6,38 @@ op, the **generic still runs alongside it** (the floor is never silently lost).
|
||||
meaningful on **any** recipe immediately (zero config), and adding recipe-specific coverage is a thin
|
||||
overlay that adds, it doesn't subtract.
|
||||
|
||||
## Architectural invariant — generic-first, custom-additive (read this first)
|
||||
|
||||
This is the load-bearing principle of the whole test architecture. If you're maintaining cc-ci a
|
||||
year from now, this is the one rule that should still hold.
|
||||
|
||||
- **Generic tests are simple and easily runnable.** They are recipe-agnostic, depend only on the
|
||||
recipe being deployable (install / upgrade / backup / restore against the recipe alone), and
|
||||
ship as the floor for every recipe. No SSO provider, no external deps, no per-recipe state
|
||||
scaffolding — just "does this recipe deploy and lifecycle work?"
|
||||
- **Generic must not depend on custom.** A custom test or a custom-tests setup (e.g. SSO/OIDC dep
|
||||
provisioning) **can never be a precondition for the generic tier to pass.** Concretely: the
|
||||
orchestrator runs all generic tiers (install → upgrade → backup → restore) against the recipe
|
||||
**alone, with no deps deployed**, then runs the `setup_custom_tests` step (deps + post-deps
|
||||
wiring) only after — and a failure there is **isolated** to the custom tier (tests tagged
|
||||
`@pytest.mark.requires_deps` skip with reason `"deps-not-ready"`; generic tier reports
|
||||
normally). See `cc-ci-plan/plan-sso-dep-testing.md` for the SSO-dep specifics.
|
||||
- **Custom tests are the thoroughness layer — and they cost more to maintain.** They're more
|
||||
thorough (authenticated APIs, multi-app flows, version-specific browser selectors, helper
|
||||
scripts, state-management) and *therefore* take more maintenance: an SSO provider's admin API
|
||||
changes, a recipe's app-launch URL contract shifts between versions, a Socket.IO primitive
|
||||
needs to track upstream — these are real ongoing costs that the generic tier deliberately
|
||||
doesn't carry.
|
||||
- **A future maintainer can choose to focus on the generic tier alone** and still get meaningful
|
||||
signal: every enrolled recipe gets *some* CI coverage from the generic floor, and the
|
||||
custom-additive layer can be scaled down or paused without breaking that floor. The choice of
|
||||
*how much* per-recipe depth to maintain is open to whoever owns cc-ci later — generic-only is
|
||||
a valid permanent operating mode.
|
||||
|
||||
If anything in this codebase ever asks you to make generic depend on custom (or to put a custom
|
||||
precondition before a generic tier), that's the signal it's drifted off the invariant — push back
|
||||
and restore the separation.
|
||||
|
||||
## The model: tiers against one shared deployment
|
||||
|
||||
A run is a sequence of **tiers**. The orchestrator (`runner/run_recipe_ci.py`) deploys the app
|
||||
|
||||
@ -60,9 +60,17 @@ def dep_domain(parent_recipe: str, pr: str, ref: str | None, dep_recipe: str) ->
|
||||
return naming.app_domain(dep_recipe, pr, synthetic_ref)
|
||||
|
||||
|
||||
def write_run_state(deps_state: list[dict]) -> None:
|
||||
"""Write the deps state file ($CCCI_DEPS_FILE) so dependent tests can find their dep apps via
|
||||
the `deps_apps` fixture. No-op if the env var isn't set."""
|
||||
def write_run_state(deps_state) -> None:
|
||||
"""Write the deps state file ($CCCI_DEPS_FILE). Two shapes supported (canonical=keyed dict):
|
||||
|
||||
1. **Legacy list-of-entries:** `[{"recipe": "<dep>", "domain": "<d>"}, ...]` (Q2.3 original).
|
||||
Still accepted by `load_run_state` for backwards compat — `deps_apps` fixture flattens.
|
||||
2. **NEW per-spec dict (operator-2026-05-28 SSO-dep plan §3.2):**
|
||||
`{"<dep_recipe>": {"recipe": "<dep>", "domain": "<d>", "realm": "...",
|
||||
"client_id": "...", "client_secret": "...", "admin_user": "...", "admin_password": "..."}}`.
|
||||
The `setup_custom_tests.sh` per-recipe hook reads this via `jq` to wire OIDC env.
|
||||
|
||||
No-op if `$CCCI_DEPS_FILE` isn't set."""
|
||||
path = os.environ.get("CCCI_DEPS_FILE")
|
||||
if not path:
|
||||
return
|
||||
@ -143,8 +151,9 @@ def teardown_deps(state: list[dict]) -> None:
|
||||
raise lifecycle.TeardownError("dep teardown failures: " + " ; ".join(errors))
|
||||
|
||||
|
||||
def load_run_state() -> list[dict]:
|
||||
"""Read the current run's deps state (used by the `deps_apps` fixture). Returns [] if unset."""
|
||||
def load_run_state():
|
||||
"""Read the current run's deps state. Returns the JSON content (list OR dict — both shapes
|
||||
supported, see write_run_state). Returns [] if file is empty/unset."""
|
||||
path = os.environ.get("CCCI_DEPS_FILE")
|
||||
if not path or not os.path.exists(path):
|
||||
return []
|
||||
@ -153,3 +162,15 @@ def load_run_state() -> list[dict]:
|
||||
return json.load(f) or []
|
||||
except (OSError, ValueError):
|
||||
return []
|
||||
|
||||
|
||||
def deps_as_dict(state) -> dict[str, dict]:
|
||||
"""Coerce either shape (legacy list or new dict) into a recipe→entry dict for the deps_apps
|
||||
fixture + dependent-tests consumption."""
|
||||
if isinstance(state, dict):
|
||||
return state
|
||||
out: dict[str, dict] = {}
|
||||
for entry in state or []:
|
||||
if isinstance(entry, dict) and entry.get("recipe"):
|
||||
out[entry["recipe"]] = entry
|
||||
return out
|
||||
|
||||
@ -264,6 +264,12 @@ def assert_discovery_endpoint(creds: dict) -> dict:
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def admin_password_inside(provider_domain: str) -> str:
|
||||
"""Read the abra-generated admin_password from inside the provider container.
|
||||
Public re-export of the previously-private _kc_admin_password for the orchestrator wiring."""
|
||||
return _kc_admin_password(provider_domain)
|
||||
|
||||
|
||||
def write_sso_creds(creds: dict) -> None:
|
||||
"""Persist creds to $CCCI_SSO_CREDS_FILE for the dependent recipe's tests to read. The file is
|
||||
in /tmp (the runner's per-process tempdir) and deleted at run end alongside the deps file."""
|
||||
|
||||
@ -279,6 +279,80 @@ def run_lifecycle_tier(
|
||||
return "pass" if rc_all == 0 else "fail"
|
||||
|
||||
|
||||
def _enrich_deps_with_sso(parent_recipe: str, parent_domain: str, deps_list) -> dict[str, dict]:
|
||||
"""For each dep, set up a fresh realm/client + test user via the harness's provider-specific
|
||||
setup function, then return a recipe→entry dict carrying domain + admin + realm/client/user
|
||||
info — the shape the `setup_custom_tests.sh` hook (and dependent tests) read.
|
||||
|
||||
Provider routing: today only `keycloak` is supported. authentik will need a parallel
|
||||
`setup_authentik_realm` when an authentik-dep recipe enrolls (DEFERRED.md #9).
|
||||
"""
|
||||
from harness import sso # local import — sso may not be needed for dep-less runs
|
||||
|
||||
out: dict[str, dict] = {}
|
||||
for entry in deps_list or []:
|
||||
dep_recipe = entry.get("recipe")
|
||||
dep_domain = entry.get("domain")
|
||||
if not dep_recipe or not dep_domain:
|
||||
continue
|
||||
if dep_recipe != "keycloak":
|
||||
# Provider not yet supported — record bare entry; setup_custom_tests.sh / tests will
|
||||
# raise if they need realm/client info they don't see.
|
||||
out[dep_recipe] = entry
|
||||
continue
|
||||
# The realm/client name uses the parent recipe name so collisions across parents are
|
||||
# impossible on a shared keycloak (and the values are predictable for debugging).
|
||||
realm = parent_recipe
|
||||
client_id = parent_recipe
|
||||
creds = sso.setup_keycloak_realm(
|
||||
dep_domain,
|
||||
realm=realm,
|
||||
client_id=client_id,
|
||||
redirect_uris=[f"https://{parent_domain}/*"],
|
||||
web_origins=[f"https://{parent_domain}"],
|
||||
)
|
||||
out[dep_recipe] = {
|
||||
"recipe": dep_recipe,
|
||||
"domain": dep_domain,
|
||||
"realm": creds["realm"],
|
||||
"client_id": creds["client_id"],
|
||||
"client_secret": creds["client_secret"],
|
||||
"user": creds["user"],
|
||||
"password": creds["password"],
|
||||
"email": creds["email"],
|
||||
"discovery_url": creds["discovery_url"],
|
||||
"token_url": creds["token_url"],
|
||||
"auth_url": creds["auth_url"],
|
||||
"userinfo_url": creds["userinfo_url"],
|
||||
"admin_user": "admin",
|
||||
"admin_password": sso.admin_password_inside(dep_domain),
|
||||
}
|
||||
return out
|
||||
|
||||
|
||||
def _run_setup_custom_tests_hook(recipe: str, domain: str, deps_file: str) -> None:
|
||||
"""Run `tests/<recipe>/setup_custom_tests.sh` if present (operator-2026-05-28 SSO-dep plan
|
||||
§3.2). The hook reads `$CCCI_DEPS_FILE`, sets OIDC env via `abra app config set` + secret
|
||||
insert, and triggers an in-place `abra app deploy --force --chaos`. Failure here propagates
|
||||
to mark deps-not-ready (caught in main())."""
|
||||
path = os.path.join(ROOT, "tests", recipe, "setup_custom_tests.sh")
|
||||
if not os.path.isfile(path):
|
||||
# No hook = recipe doesn't need post-deps wiring; deps are deployed + creds available
|
||||
# via deps_apps fixture as-is.
|
||||
print(f" setup_custom_tests: no hook at {os.path.relpath(path, ROOT)} (deps creds ready in $CCCI_DEPS_FILE)", flush=True)
|
||||
return
|
||||
print(f" setup_custom_tests hook: {os.path.relpath(path, ROOT)}", flush=True)
|
||||
rc = subprocess.run(
|
||||
["bash", path],
|
||||
check=False,
|
||||
env=dict(os.environ, CCCI_APP_DOMAIN=domain, CCCI_RECIPE=recipe, CCCI_DEPS_FILE=deps_file),
|
||||
)
|
||||
if rc.returncode != 0:
|
||||
raise RuntimeError(
|
||||
f"setup_custom_tests.sh exited {rc.returncode} (deps env not wired into parent)"
|
||||
)
|
||||
|
||||
|
||||
def run_custom(recipe: str, repo_local: str | None, domain: str) -> str:
|
||||
"""Run all discovered non-lifecycle custom test_*.py (both locations, additive). Returns
|
||||
'skip' if none defined, else 'pass'/'fail'."""
|
||||
@ -344,59 +418,47 @@ def main() -> int:
|
||||
os.environ["CCCI_OP_STATE_FILE"] = statefile
|
||||
op_state: dict = {}
|
||||
|
||||
# Run-scoped dep state (Phase 2 Q2.3): if this recipe declares DEPS in recipe_meta, the
|
||||
# orchestrator deploys each dep BEFORE the recipe under test, persists their per-run identity
|
||||
# here for dependent tests to read via the `deps_apps` fixture, and tears them down LAST in
|
||||
# finally (reverse order). Empty list when no deps declared.
|
||||
# Run-scoped dep state (Phase 2 Q2.3, refined per operator-2026-05-28 SSO-dep plan §1):
|
||||
# deps now deploy AFTER generic tiers (between RESTORE and CUSTOM) so a failed dep deploy
|
||||
# cannot break the generic-tier signal. The `setup_custom_tests` step deploys each dep + runs
|
||||
# `tests/<recipe>/setup_custom_tests.sh` to wire OIDC env via in-place redeploy.
|
||||
# `$CCCI_DEPS_FILE` is written with the full creds dict the hook script needs (jq-readable).
|
||||
depsfile = os.path.join(tempfile.gettempdir(), f"ccci-deps-{domain}.json")
|
||||
with open(depsfile, "w") as f:
|
||||
json.dump([], f)
|
||||
json.dump({}, f)
|
||||
os.environ["CCCI_DEPS_FILE"] = depsfile
|
||||
declared = deps_mod.declared_deps(recipe)
|
||||
if declared:
|
||||
print(f"\n===== DEPS: {declared} =====", flush=True)
|
||||
deps_state: list[dict] = []
|
||||
print(f"\n===== DEPS declared (deploy AFTER generic tiers): {declared} =====", flush=True)
|
||||
deps_state: dict[str, dict] = {} # new shape: recipe→entry dict (sso-dep plan §1)
|
||||
deps_ready = True
|
||||
deps_not_ready_reason: str = ""
|
||||
|
||||
results: dict[str, str] = {}
|
||||
lifecycle.janitor()
|
||||
dep_deploy_failed = False
|
||||
dep_teardown_error: str | None = None
|
||||
try:
|
||||
# ---- deps deploy FIRST (sequentially), if declared (Q2.3) ----
|
||||
if declared:
|
||||
try:
|
||||
# Build a per-dep meta map for readiness waits (timeouts/health-path/codes)
|
||||
dep_metas = {d: _load_meta(d) for d in declared}
|
||||
deps_state = deps_mod.deploy_deps(
|
||||
recipe, os.environ.get("PR", "0"), ref, declared, meta_for=dep_metas
|
||||
)
|
||||
except Exception as e: # noqa: BLE001 — failed dep deploy is a recipe install failure
|
||||
print(f"!! dep deploy failed: {_scrub(str(e))}", flush=True)
|
||||
dep_deploy_failed = True
|
||||
# ---- deploy ONCE + wait ready (the single deployment all tiers share) ----
|
||||
if dep_deploy_failed:
|
||||
# ---- deploy RECIPE FIRST, alone (no deps yet — generic tiers run recipe-only) ----
|
||||
try:
|
||||
lifecycle.deploy_app(
|
||||
recipe,
|
||||
domain,
|
||||
version=base,
|
||||
secrets=True,
|
||||
install_steps_hook=hook,
|
||||
deploy_timeout=int(meta.get("DEPLOY_TIMEOUT", 900)),
|
||||
)
|
||||
lifecycle.wait_healthy(
|
||||
domain,
|
||||
ok_codes=tuple(meta["HEALTH_OK"]),
|
||||
path=meta["HEALTH_PATH"],
|
||||
deploy_timeout=meta["DEPLOY_TIMEOUT"],
|
||||
http_timeout=meta["HTTP_TIMEOUT"],
|
||||
)
|
||||
deploy_ok = True
|
||||
except Exception as e: # noqa: BLE001 — a failed deploy is a reported INSTALL failure
|
||||
print(f"!! deploy/readiness failed: {e}", flush=True)
|
||||
deploy_ok = False
|
||||
else:
|
||||
try:
|
||||
lifecycle.deploy_app(
|
||||
recipe,
|
||||
domain,
|
||||
version=base,
|
||||
secrets=True,
|
||||
install_steps_hook=hook,
|
||||
deploy_timeout=int(meta.get("DEPLOY_TIMEOUT", 900)),
|
||||
)
|
||||
lifecycle.wait_healthy(
|
||||
domain,
|
||||
ok_codes=tuple(meta["HEALTH_OK"]),
|
||||
path=meta["HEALTH_PATH"],
|
||||
deploy_timeout=meta["DEPLOY_TIMEOUT"],
|
||||
http_timeout=meta["HTTP_TIMEOUT"],
|
||||
)
|
||||
deploy_ok = True
|
||||
except Exception as e: # noqa: BLE001 — a failed deploy is a reported INSTALL failure, not a crash
|
||||
print(f"!! deploy/readiness failed: {e}", flush=True)
|
||||
deploy_ok = False
|
||||
|
||||
# ---- INSTALL tier (always; additive generic + overlay, no op) ----
|
||||
if "install" in stages:
|
||||
@ -433,8 +495,38 @@ def main() -> int:
|
||||
if backup_cap
|
||||
else "skip"
|
||||
)
|
||||
# ---- setup_custom_tests step (NEW, operator-2026-05-28 SSO-dep plan §3.2) ----
|
||||
# Deploy each declared dep + wire OIDC env into the parent app via the per-recipe
|
||||
# setup_custom_tests.sh hook + in-place redeploy. Failure here marks deps-not-ready
|
||||
# but does NOT abort the run — @pytest.mark.requires_deps tests skip with reason;
|
||||
# non-deps custom tests still run normally.
|
||||
if declared:
|
||||
print("\n===== setup_custom_tests: deps + OIDC wiring =====", flush=True)
|
||||
try:
|
||||
dep_metas = {d: _load_meta(d) for d in declared}
|
||||
deps_list = deps_mod.deploy_deps(
|
||||
recipe, os.environ.get("PR", "0"), ref, declared, meta_for=dep_metas
|
||||
)
|
||||
# Enrich each dep entry with SSO creds (realm/client/secret) by setting up a
|
||||
# keycloak realm per dep. The dict form is what setup_custom_tests.sh reads.
|
||||
deps_state = _enrich_deps_with_sso(recipe, domain, deps_list)
|
||||
deps_mod.write_run_state(deps_state)
|
||||
# Run the per-recipe post-deps hook (jq-driven OIDC wiring + in-place redeploy)
|
||||
_run_setup_custom_tests_hook(recipe, domain, depsfile)
|
||||
except Exception as e: # noqa: BLE001 — setup failure is ISOLATED to dep-marked tests
|
||||
deps_ready = False
|
||||
deps_not_ready_reason = _scrub(str(e))[:300]
|
||||
print(
|
||||
f"!! setup_custom_tests failed (deps-not-ready): {deps_not_ready_reason}",
|
||||
flush=True,
|
||||
)
|
||||
|
||||
# ---- CUSTOM tier ----
|
||||
if "custom" in stages:
|
||||
# Pass deps-ready state via env; conftest.py skips @pytest.mark.requires_deps
|
||||
# tests when CCCI_DEPS_READY=0.
|
||||
os.environ["CCCI_DEPS_READY"] = "1" if deps_ready else "0"
|
||||
os.environ["CCCI_DEPS_NOT_READY_REASON"] = deps_not_ready_reason
|
||||
results["custom"] = run_custom(recipe, repo_local, domain)
|
||||
else:
|
||||
# install failed → the shared deployment is dead; remaining tiers cannot run on it.
|
||||
@ -451,7 +543,13 @@ def main() -> int:
|
||||
if deps_state:
|
||||
print("\n===== DEPS teardown =====", flush=True)
|
||||
try:
|
||||
deps_mod.teardown_deps(deps_state)
|
||||
# teardown_deps accepts a list of entries; flatten the dict-shape state in
|
||||
# declaration-reverse order so teardown sequencing matches §1's contract.
|
||||
if isinstance(deps_state, dict):
|
||||
list_for_teardown = [deps_state[d] for d in declared if d in deps_state]
|
||||
else:
|
||||
list_for_teardown = deps_state
|
||||
deps_mod.teardown_deps(list_for_teardown)
|
||||
except lifecycle.TeardownError as e:
|
||||
dep_teardown_error = str(e)
|
||||
print(f"!! {dep_teardown_error}", flush=True)
|
||||
@ -466,13 +564,22 @@ def main() -> int:
|
||||
os.remove(depsfile)
|
||||
|
||||
# ---- per-op summary (DG6 feed) ----
|
||||
# Phase 2 Q2.3: deps each `deploy_app` once, so the expected count = 1 (recipe under test) +
|
||||
# len(deps). DG4.1 still holds — no extra deploys per recipe — just accommodates declared deps.
|
||||
expected_deploy_count = 1 + len(deps_state)
|
||||
# SSO-dep plan §1: DG4.1 generalised — one `abra app new` per app in the run (recipe + each
|
||||
# dep). In-place reconfigure-and-redeploy (the setup_custom_tests step's
|
||||
# `abra app deploy --force --chaos`) is NOT a fresh `app_new` and does NOT increment the
|
||||
# count. So expected = 1 + (number of deps that actually got deployed).
|
||||
deps_deployed_count = len(deps_state) if isinstance(deps_state, dict) else len(deps_state or [])
|
||||
expected_deploy_count = 1 + deps_deployed_count
|
||||
print("\n===== RUN SUMMARY =====", flush=True)
|
||||
print(f"deploy-count = {deploy_count} (expect {expected_deploy_count})")
|
||||
if deps_state:
|
||||
print(f" deps deployed: {[d['recipe'] for d in deps_state]}")
|
||||
deps_list_for_summary = (
|
||||
list(deps_state.keys()) if isinstance(deps_state, dict)
|
||||
else [d.get("recipe", "?") for d in deps_state]
|
||||
)
|
||||
print(f" deps deployed: {deps_list_for_summary}")
|
||||
if not deps_ready:
|
||||
print(f" deps-not-ready: {deps_not_ready_reason}")
|
||||
order = [s for s in ALL_STAGES if s in results]
|
||||
for op in order:
|
||||
print(f" {op:8s}: {results[op]}")
|
||||
|
||||
@ -74,13 +74,50 @@ def live_app() -> str:
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def deps_apps() -> dict[str, str]:
|
||||
"""Phase 2 Q2.3 dependency-resolver contract: when a recipe declares `DEPS = [...]` in its
|
||||
`recipe_meta.py`, the orchestrator deploys each dep BEFORE the recipe under test, persists
|
||||
their per-run identity to `$CCCI_DEPS_FILE`, and tears them down LAST in finally. Tests access
|
||||
the dep's per-run domain via this fixture: `deps_apps["keycloak"]` returns the dep's domain
|
||||
or raises KeyError if the dep wasn't declared. Returns {} when the recipe declared no deps."""
|
||||
state = deps_mod.load_run_state()
|
||||
return {entry["recipe"]: entry["domain"] for entry in state if entry.get("domain")}
|
||||
"""Phase 2 Q2.3 dependency-resolver contract (refined operator-2026-05-28 SSO-dep plan §1):
|
||||
when a recipe declares `DEPS = [...]` in its `recipe_meta.py`, the orchestrator deploys each
|
||||
dep AFTER the generic tiers (between RESTORE and CUSTOM) and persists their per-run identity
|
||||
+ SSO creds to `$CCCI_DEPS_FILE`. Tests access the dep's per-run domain via this fixture.
|
||||
For full SSO creds (realm/client/secret/admin) use the `deps_creds` fixture instead.
|
||||
|
||||
Returns `{dep_recipe: domain}` (str→str). Empty when no deps declared OR deps-not-ready."""
|
||||
state = deps_mod.deps_as_dict(deps_mod.load_run_state())
|
||||
return {r: e["domain"] for r, e in state.items() if e.get("domain")}
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def deps_creds() -> dict[str, dict]:
|
||||
"""Full SSO-creds dict for each declared dep (operator-2026-05-28 SSO-dep plan §1).
|
||||
`deps_creds["keycloak"]` returns the entry written by setup_custom_tests with keys
|
||||
domain/realm/client_id/client_secret/user/password/email/admin_user/admin_password/
|
||||
discovery_url/token_url/.... Use this in `@pytest.mark.requires_deps` tests that need to
|
||||
authenticate via OIDC."""
|
||||
return deps_mod.deps_as_dict(deps_mod.load_run_state())
|
||||
|
||||
|
||||
def pytest_collection_modifyitems(config, items):
|
||||
"""SSO-dep plan §4: tests marked `@pytest.mark.requires_deps` are skipped with reason
|
||||
`deps-not-ready: <captured-err>` when the orchestrator's setup_custom_tests step failed
|
||||
(orchestrator sets CCCI_DEPS_READY=0 in env). Non-deps custom tests are unaffected.
|
||||
|
||||
This is failure-isolation per plan §1 — generic tiers cannot break the SSO-marked tests'
|
||||
skip status, and an SSO-setup failure cannot break the generic tiers (they run first)."""
|
||||
deps_ready_env = os.environ.get("CCCI_DEPS_READY", "1")
|
||||
if deps_ready_env == "1":
|
||||
return
|
||||
reason = os.environ.get("CCCI_DEPS_NOT_READY_REASON", "(no reason given)")
|
||||
skip_mark = pytest.mark.skip(reason=f"deps-not-ready: {reason}")
|
||||
for item in items:
|
||||
if "requires_deps" in item.keywords:
|
||||
item.add_marker(skip_mark)
|
||||
|
||||
|
||||
def pytest_configure(config):
|
||||
"""Register the `requires_deps` marker so pytest doesn't warn about it."""
|
||||
config.addinivalue_line(
|
||||
"markers",
|
||||
"requires_deps: test requires DEPS-declared services + setup_custom_tests success.",
|
||||
)
|
||||
|
||||
|
||||
def _wait_healthy(domain, meta):
|
||||
|
||||
@ -1,27 +1,13 @@
|
||||
"""lasuite-docs — Q2 SSO-flow acceptance test (Phase 2 §6 Q2 gate).
|
||||
"""lasuite-docs — Q2 SSO-flow acceptance test (operator-2026-05-28 SSO-dep plan).
|
||||
|
||||
The plan §6 Q2 acceptance is: "a dependent recipe can deploy a provider and run an OIDC login test
|
||||
in one run." This test exercises that contract end-to-end:
|
||||
|
||||
1. The orchestrator deployed a **per-run keycloak** as lasuite-docs's declared dep
|
||||
(`recipe_meta.DEPS = ["keycloak"]`). Its domain is in `deps_apps["keycloak"]`.
|
||||
2. Use the shared SSO-setup harness primitive (`runner/harness/sso.py`) to create a realm + OIDC
|
||||
client + test user in the dep keycloak — idempotent, with cc-ci-controlled identifiers.
|
||||
3. Exercise the OIDC discovery endpoint (`.well-known/openid-configuration`); assert issuer is
|
||||
the per-run keycloak's `/realms/<realm>`.
|
||||
4. Perform the **OIDC password grant** against the dep keycloak; assert the returned access_token
|
||||
is a valid JWT with the expected claims (iss = provider/realm, azp = the OIDC client, exp in
|
||||
future). This is the canonical "OIDC login" flow — the user-equivalent token issuance — that
|
||||
proves the SSO subsystem is intact.
|
||||
|
||||
Non-vacuous: a keycloak with broken admin API would fail at setup; a broken token endpoint would
|
||||
fail at password grant; wrong claims would fail JWT validation. Each step uses real credentials
|
||||
generated for THIS run (class-B per §4.4-B); destroyed when the dep is torn down at run end.
|
||||
|
||||
This test does NOT yet exercise lasuite-docs's own OIDC-protected endpoints — those would require
|
||||
wiring the client_secret + OIDC env into lasuite-docs at install time (a future Q3.1 task; see
|
||||
DECISIONS.md). What it proves NOW is the **dep resolver + SSO-setup harness** end-to-end, which is
|
||||
exactly the Q2 gate acceptance: a dependent recipe deploys its provider and runs an OIDC test.
|
||||
Refactored to the refined SSO-dep model:
|
||||
- The orchestrator deploys a per-run keycloak dep AFTER generic tiers and provisions a fresh
|
||||
realm/client/user via `harness.sso.setup_keycloak_realm`. The creds are written to
|
||||
`$CCCI_DEPS_FILE` (read here via the `deps_creds` fixture).
|
||||
- This test no longer calls `setup_keycloak_realm` itself — that's the orchestrator's job in
|
||||
the setup_custom_tests step. We just consume the credentials and exercise the OIDC flow.
|
||||
- Marked `@pytest.mark.requires_deps` so if setup_custom_tests failed, this test SKIPs with a
|
||||
clear `deps-not-ready` reason rather than red-flagging a non-recipe failure.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@ -32,6 +18,8 @@ import os
|
||||
import sys
|
||||
import time
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "..", "runner"))
|
||||
from harness import sso # noqa: E402
|
||||
|
||||
@ -41,43 +29,55 @@ def _b64url_decode(seg: str) -> bytes:
|
||||
return base64.urlsafe_b64decode(seg + pad)
|
||||
|
||||
|
||||
def test_oidc_password_grant_against_dep_keycloak(live_app, deps_apps):
|
||||
"""End-to-end: keycloak dep is up → set up realm/client/user → OIDC password grant → JWT."""
|
||||
assert "keycloak" in deps_apps, (
|
||||
f"keycloak dep not deployed; deps_apps={deps_apps}. Q2.3 dep resolver did not run."
|
||||
@pytest.mark.requires_deps
|
||||
def test_oidc_password_grant_against_dep_keycloak(live_app, deps_creds):
|
||||
"""The dep keycloak issues a JWT for the pre-provisioned test user via OIDC password grant."""
|
||||
assert "keycloak" in deps_creds, (
|
||||
f"keycloak creds not in deps_creds; got {list(deps_creds.keys())}. "
|
||||
"setup_custom_tests should have populated this."
|
||||
)
|
||||
kc_domain = deps_apps["keycloak"]
|
||||
kc = deps_creds["keycloak"]
|
||||
|
||||
creds = sso.setup_keycloak_realm(
|
||||
kc_domain,
|
||||
realm="lasuite-docs",
|
||||
client_id="docs",
|
||||
redirect_uris=[f"https://{live_app}/*"],
|
||||
web_origins=[f"https://{live_app}"],
|
||||
)
|
||||
# Sanity-check the creds shape
|
||||
assert creds["provider"] == "keycloak"
|
||||
assert creds["provider_domain"] == kc_domain
|
||||
assert creds["realm"] == "lasuite-docs"
|
||||
assert creds["client_id"] == "docs"
|
||||
assert isinstance(creds["client_secret"], str) and len(creds["client_secret"]) >= 16
|
||||
assert isinstance(creds["password"], str) and len(creds["password"]) >= 16
|
||||
# Sanity-check the creds shape — orchestrator-written
|
||||
assert kc["domain"]
|
||||
assert kc["realm"] == "lasuite-docs" # orchestrator names the realm after the parent recipe
|
||||
assert kc["client_id"] == "lasuite-docs"
|
||||
assert isinstance(kc["client_secret"], str) and len(kc["client_secret"]) >= 16
|
||||
assert isinstance(kc["password"], str) and len(kc["password"]) >= 16
|
||||
|
||||
# Build a creds dict in the shape sso.* primitives expect
|
||||
creds = {
|
||||
"provider": "keycloak",
|
||||
"provider_domain": kc["domain"],
|
||||
"realm": kc["realm"],
|
||||
"client_id": kc["client_id"],
|
||||
"client_secret": kc["client_secret"],
|
||||
"user": kc["user"],
|
||||
"password": kc["password"],
|
||||
"email": kc["email"],
|
||||
"discovery_url": kc["discovery_url"],
|
||||
"token_url": kc["token_url"],
|
||||
"auth_url": kc["auth_url"],
|
||||
"userinfo_url": kc["userinfo_url"],
|
||||
}
|
||||
|
||||
# OIDC discovery endpoint advertises the realm
|
||||
discovery = sso.assert_discovery_endpoint(creds)
|
||||
expected_iss = f"https://{kc_domain}/realms/lasuite-docs"
|
||||
expected_iss = f"https://{kc['domain']}/realms/{kc['realm']}"
|
||||
assert discovery.get("issuer") == expected_iss
|
||||
assert discovery.get("token_endpoint", "").startswith(expected_iss + "/")
|
||||
assert discovery.get("authorization_endpoint", "").startswith(expected_iss + "/")
|
||||
|
||||
# Password grant flow → real JWT
|
||||
# Password grant → real JWT
|
||||
token = sso.oidc_password_grant(creds)
|
||||
assert isinstance(token, str) and token.count(".") == 2, (
|
||||
f"access_token is not a JWT: {token!r}"
|
||||
)
|
||||
payload = json.loads(_b64url_decode(token.split(".")[1]))
|
||||
assert payload.get("iss") == expected_iss, f"JWT iss={payload.get('iss')!r} != {expected_iss!r}"
|
||||
assert payload.get("azp") == "docs", f"JWT azp={payload.get('azp')!r} != 'docs'"
|
||||
assert payload.get("azp") == kc["client_id"], (
|
||||
f"JWT azp={payload.get('azp')!r} != {kc['client_id']!r}"
|
||||
)
|
||||
assert payload.get("typ") == "Bearer", f"JWT typ={payload.get('typ')!r} != 'Bearer'"
|
||||
exp = payload.get("exp")
|
||||
assert isinstance(exp, int) and exp > time.time(), (
|
||||
|
||||
79
tests/lasuite-docs/setup_custom_tests.sh
Executable file
79
tests/lasuite-docs/setup_custom_tests.sh
Executable file
@ -0,0 +1,79 @@
|
||||
#!/usr/bin/env bash
|
||||
# lasuite-docs — post-deps setup hook (operator-2026-05-28 SSO-dep plan §3.2).
|
||||
#
|
||||
# Runs AFTER the generic tiers (install/upgrade/backup/restore) and AFTER each declared dep is
|
||||
# deployed + provisioned with realm/client via the harness. The orchestrator has written
|
||||
# $CCCI_DEPS_FILE with the keycloak dep's domain + realm + client_secret + admin creds.
|
||||
#
|
||||
# This hook:
|
||||
# 1. Reads the dep's connection info from $CCCI_DEPS_FILE.
|
||||
# 2. Inserts the OIDC client secret as an abra app secret (recipe-conventional name oidc_rpcs).
|
||||
# 3. Writes the OIDC env vars to the running app's .env via `abra app config set`.
|
||||
# 4. Triggers an in-place `abra app deploy --force --chaos` so the new env takes effect.
|
||||
# THIS IS NOT a fresh `abra app new` — the deploy-count guard (DG4.1, generalised) still
|
||||
# sees one app_new per app.
|
||||
#
|
||||
# Env supplied by the orchestrator:
|
||||
# CCCI_APP_DOMAIN — the running per-run lasuite-docs app domain
|
||||
# CCCI_RECIPE — "lasuite-docs"
|
||||
# CCCI_DEPS_FILE — JSON file (dict shape: {dep_recipe: {domain, realm, client_id, ...}, ...})
|
||||
set -euo pipefail
|
||||
|
||||
: "${CCCI_APP_DOMAIN:?missing}"
|
||||
: "${CCCI_DEPS_FILE:?missing}"
|
||||
test -s "$CCCI_DEPS_FILE" || { echo " setup_custom_tests: deps file empty"; exit 1; }
|
||||
|
||||
# Read keycloak dep info via jq
|
||||
KC_DOMAIN=$(jq -r '.keycloak.domain' "$CCCI_DEPS_FILE")
|
||||
KC_REALM=$( jq -r '.keycloak.realm' "$CCCI_DEPS_FILE")
|
||||
KC_CLIENT=$(jq -r '.keycloak.client_id' "$CCCI_DEPS_FILE")
|
||||
KC_SECRET=$(jq -r '.keycloak.client_secret' "$CCCI_DEPS_FILE")
|
||||
[ -n "$KC_DOMAIN" ] && [ "$KC_DOMAIN" != "null" ] || { echo " setup_custom_tests: no keycloak.domain in deps"; exit 1; }
|
||||
[ -n "$KC_SECRET" ] && [ "$KC_SECRET" != "null" ] || { echo " setup_custom_tests: no keycloak.client_secret"; exit 1; }
|
||||
|
||||
echo " lasuite-docs setup_custom_tests: wiring OIDC against keycloak dep ${KC_DOMAIN}"
|
||||
|
||||
# 1) Insert the OIDC client secret AT A BUMPED VERSION (the recipe-maintainer pattern).
|
||||
# `abra app new -S` already generated `oidc_rpcs:v1` (random) — Docker Swarm forbids overwriting
|
||||
# a secret at the same version, so we bump the version (v2), insert our value there, then
|
||||
# update SECRET_OIDC_RPCS_VERSION in the .env to point at the new one.
|
||||
ENV_PATH="$HOME/.abra/servers/default/${CCCI_APP_DOMAIN}.env"
|
||||
CUR_VER=$(grep -E '^\s*SECRET_OIDC_RPCS_VERSION=' "$ENV_PATH" | tail -1 | cut -d= -f2 | tr -d '"\r' || echo "v1")
|
||||
NEW_NUM=$(( ${CUR_VER#v} + 1 ))
|
||||
NEW_VER="v${NEW_NUM}"
|
||||
|
||||
INSERT_LOG=$(abra app secret insert $CCCI_APP_DOMAIN oidc_rpcs $NEW_VER $KC_SECRET --no-input 2>&1) \
|
||||
|| INSERT_LOG=$(script -qec "abra app secret insert $CCCI_APP_DOMAIN oidc_rpcs $NEW_VER $KC_SECRET --no-input" /dev/null 2>&1) \
|
||||
|| { echo " setup_custom_tests: abra app secret insert oidc_rpcs@$NEW_VER failed: $INSERT_LOG"; exit 1; }
|
||||
# Repoint the env var to the new version
|
||||
sed -i "s|^\s*SECRET_OIDC_RPCS_VERSION=.*|SECRET_OIDC_RPCS_VERSION=$NEW_VER|" "$ENV_PATH"
|
||||
echo " setup_custom_tests: oidc_rpcs secret inserted at $NEW_VER (was $CUR_VER)"
|
||||
|
||||
# 2) Write OIDC env vars to the app's .env (names per lasuite-docs's .env.sample).
|
||||
# Ensure the file ends with a newline FIRST so our appends don't concatenate onto the last line
|
||||
# (we saw `TIMEOUT=900OIDC_REALM=...` malformed by a missing-trailing-newline file).
|
||||
[ -z "$(tail -c1 "$ENV_PATH" 2>/dev/null)" ] || printf '\n' >> "$ENV_PATH"
|
||||
write_env () {
|
||||
local key="$1" val="$2"
|
||||
# remove any existing key (commented or live) then append the live key=val
|
||||
sed -i "/^\s*#\?\s*${key}=/d" "$ENV_PATH"
|
||||
# Re-ensure trailing newline after each delete (sed may leave the file without one)
|
||||
[ -z "$(tail -c1 "$ENV_PATH" 2>/dev/null)" ] || printf '\n' >> "$ENV_PATH"
|
||||
printf '%s=%s\n' "$key" "$val" >> "$ENV_PATH"
|
||||
}
|
||||
write_env OIDC_REALM "$KC_REALM"
|
||||
write_env OIDC_OP_DISCOVERY_ENDPOINT "https://${KC_DOMAIN}/realms/${KC_REALM}/.well-known/openid-configuration"
|
||||
write_env OIDC_OP_AUTHORIZATION_ENDPOINT "https://${KC_DOMAIN}/realms/${KC_REALM}/protocol/openid-connect/auth"
|
||||
write_env OIDC_OP_TOKEN_ENDPOINT "https://${KC_DOMAIN}/realms/${KC_REALM}/protocol/openid-connect/token"
|
||||
write_env OIDC_OP_USER_ENDPOINT "https://${KC_DOMAIN}/realms/${KC_REALM}/protocol/openid-connect/userinfo"
|
||||
write_env OIDC_OP_LOGOUT_ENDPOINT "https://${KC_DOMAIN}/realms/${KC_REALM}/protocol/openid-connect/logout"
|
||||
write_env OIDC_OP_JWKS_ENDPOINT "https://${KC_DOMAIN}/realms/${KC_REALM}/protocol/openid-connect/certs"
|
||||
write_env OIDC_RP_CLIENT_ID "$KC_CLIENT"
|
||||
write_env OIDC_RP_SIGN_ALGO "RS256"
|
||||
write_env OIDC_RP_SCOPES "openid email profile"
|
||||
|
||||
# 3) Trigger an in-place redeploy so the env update takes effect. --force re-deploys even when
|
||||
# the recipe hasn't changed; --chaos avoids the chaos prompt; --no-input non-interactive.
|
||||
abra app deploy "$CCCI_APP_DOMAIN" --force --chaos --no-input 2>&1 | tail -10
|
||||
|
||||
echo " lasuite-docs setup_custom_tests: OIDC wired + redeployed"
|
||||
Reference in New Issue
Block a user