211 lines
13 KiB
Markdown
211 lines
13 KiB
Markdown
# SSO-dep testing pattern (OIDC + co-deployed provider) — reference plan
|
|
|
|
**Status:** active reference for Phase 2 (does not need its own phase; it's a pattern Phase 2
|
|
overlays apply per recipe).
|
|
**Operator clarification (2026-05-28):** integrating a recipe with an OIDC/SSO dep is **loop work,
|
|
not operator work**. Anything that was deferred citing "operator input needed for OIDC" should be
|
|
re-opened and done autonomously per this plan.
|
|
**This file's path:** `/srv/cc-ci/cc-ci-plan/plan-sso-dep-testing.md`
|
|
**Companion:** the running harness in `runner/harness/sso.py` (existing primitives:
|
|
`setup_keycloak_realm`, `oidc_password_grant`, `assert_discovery_endpoint`).
|
|
|
|
---
|
|
|
|
## 0. Why this plan
|
|
|
|
Several recipes test their authenticated functionality through an OIDC/SSO provider (keycloak,
|
|
authentik). The cc-ci pattern is to **co-deploy the provider with the recipe under test in the same
|
|
ephemeral run** — one shared deployment per dep, configured at install time, used by the
|
|
recipe-under-test's authenticated tests, torn down with it. This file is the canonical pattern for
|
|
how to wire that up so any recipe that declares `DEPS = ["keycloak"]` (or `["authentik"]`) Just Works
|
|
without per-recipe ad-hoc plumbing. Recipes that need OIDC are not blocked on the operator — they
|
|
follow this plan.
|
|
|
|
## 1. The DEPS model — deps deploy AFTER generic tiers (operator-2026-05-28)
|
|
|
|
**Critical ordering rule:** generic tiers (install / upgrade / backup / restore) run against the
|
|
**recipe alone, with no dep available**, so a failure in dep-deploy or OIDC setup **cannot break
|
|
generic-tier signal**. Deps + OIDC wiring move to a **`setup_custom_tests` step** that runs *after*
|
|
generic tiers and *before* the custom tier — its failure is isolated to the SSO-marked custom tests.
|
|
|
|
A recipe's `tests/<recipe>/recipe_meta.py` declares its SSO dep:
|
|
```python
|
|
DEPS = ["keycloak"] # or ["authentik"] when that backend lands
|
|
```
|
|
|
|
### Lifecycle order (single run, per recipe)
|
|
|
|
```
|
|
1. Deploy recipe-under-test ALONE (no deps, OIDC env unset or stubbed).
|
|
- app_new #1 for the recipe; generic install_steps.sh runs RECIPE-ONLY setup (no deps).
|
|
2. INSTALL tier: generic [+ overlay] assertions against the recipe alone.
|
|
3. UPGRADE tier: abra app upgrade in place, assertions against the recipe alone.
|
|
4. BACKUP tier: in place (if backup-capable), recipe-alone marker.
|
|
5. RESTORE tier: in place, recipe-alone marker.
|
|
6. setup_custom_tests step ← NEW (operator-2026-05-28)
|
|
a. For each dep in DEPS, deploy + provision realm/client via harness.sso.setup_<provider>_realm.
|
|
b. Write $CCCI_DEPS_FILE with each dep's {domain, realm, client_id, client_secret, admin_*}.
|
|
c. Run the per-recipe post-deps hook `tests/<recipe>/setup_custom_tests.sh` to wire the OIDC
|
|
env into the running recipe (abra app config set + abra app secret insert) and trigger an
|
|
in-place redeploy of the affected services so the env takes effect.
|
|
d. Mark deps-ready = True on success; on ANY failure mark deps-ready = False and CONTINUE
|
|
(log the error; do NOT abort the run).
|
|
7. CUSTOM tier:
|
|
- If deps-ready: run all custom tests, including those tagged @pytest.mark.requires_deps.
|
|
- If NOT deps-ready: still run custom tests, but tests tagged @pytest.mark.requires_deps are
|
|
reported as ERROR/SKIP (with the captured setup_custom_tests error attached). Non-deps
|
|
custom tests still run normally.
|
|
8. Teardown (in finally): recipe first; then each dep in reverse declaration order.
|
|
```
|
|
|
|
### DG4.1 deploy-count guard, generalised
|
|
The "one deploy per run" guard becomes **one `abra app new` per app in the run** (recipe + each
|
|
dep). In-place reconfigure-and-redeploy (the step 6c env update) is **NOT** a fresh `app_new` and
|
|
does NOT increment the per-recipe count. So a run with `DEPS = ["keycloak"]` has exactly 2
|
|
`app_new` calls (recipe + keycloak), no matter how many tiers ran. The per-run summary reports
|
|
deploy-count per app for verification.
|
|
|
|
### Why this ordering
|
|
- **Generic-tier signal is preserved** when SSO/dep setup is broken — the recipe's own deploy/
|
|
upgrade/backup/restore behaviour is still tested honestly.
|
|
- **Failure isolation**: a recipe whose generic tier passes but whose SSO setup is broken yields
|
|
per-op `pass/pass/pass/pass/skip(deps-not-ready)` — far more useful than the previous
|
|
all-or-nothing.
|
|
- A recipe that genuinely can't boot without OIDC fails its generic install honestly (the recipe
|
|
should accept a stubbed/empty OIDC env at install time and only require the env when an
|
|
authenticated endpoint is hit). That's a real recipe finding, not a CI artifact.
|
|
|
|
## 2. Provider pluggability
|
|
|
|
- **Provider-agnostic primitives** (today, in `harness/sso.py`) — these stay pluggable:
|
|
- `oidc_password_grant(discovery_url, client_id, client_secret, username, password) -> token` —
|
|
pure OIDC; works against any compliant provider.
|
|
- `assert_discovery_endpoint(discovery_url, expected_issuer)` — pure OIDC.
|
|
- **Provider-specific setup** (admin API calls) — one function per provider:
|
|
- `setup_keycloak_realm(domain, admin_user, admin_password, realm, client_id, redirect_uris) ->
|
|
{client_secret, discovery_url}` — exists today.
|
|
- `setup_authentik_realm(...)` — same shape, authentik admin API; **deferred** to a future Q4
|
|
enrollment that actually wants authentik (see `machine-docs/DEFERRED.md`). Pluggable: a recipe
|
|
declaring `DEPS = ["authentik"]` would just call this instead. No change to the per-recipe
|
|
`install_steps.sh` shape beyond which provider it asks for from `$CCCI_DEPS_FILE`.
|
|
- **Don't write per-recipe SSO logic.** All recipes use the same DEPS+install_steps shape.
|
|
|
|
## 3. Per-recipe hooks — two distinct scripts (recipe-only vs post-deps)
|
|
|
|
A recipe with `DEPS = ["keycloak"]` ships **two** optional hook scripts (either may be absent if
|
|
not needed):
|
|
|
|
### 3.1 `tests/<recipe>/install_steps.sh` — RECIPE-ONLY setup, runs at install time
|
|
This is the Phase-1d custom-install-steps hook. It runs **before** the recipe deploys, **with no
|
|
dep available** (the dep hasn't been deployed yet at this point). Use it only for recipe-only
|
|
setup that the recipe needs to boot at all (e.g. seed a fixture, set a non-OIDC env). **Do NOT
|
|
read `$CCCI_DEPS_FILE` here** — it doesn't exist yet. If the recipe requires OIDC to *boot at
|
|
all*, set a safe stub here (e.g. disable auth) so the recipe can come up for generic tiers; the
|
|
real OIDC wiring happens in §3.2.
|
|
|
|
### 3.2 `tests/<recipe>/setup_custom_tests.sh` — POST-DEPS wiring, runs after generic tiers
|
|
This is the new (operator-2026-05-28) hook that wires the recipe to its already-deployed dep,
|
|
*after* the generic tiers have run. The orchestrator has already deployed each dep and written
|
|
`$CCCI_DEPS_FILE` by the time this runs. Roughly:
|
|
|
|
```sh
|
|
#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
# Read the dep's connection info from $CCCI_DEPS_FILE (orchestrator-written).
|
|
KC_DOMAIN=$(jq -r '.keycloak.domain' "$CCCI_DEPS_FILE")
|
|
KC_CLIENT=$(jq -r '.keycloak.client_id' "$CCCI_DEPS_FILE")
|
|
KC_SECRET=$(jq -r '.keycloak.client_secret' "$CCCI_DEPS_FILE")
|
|
KC_REALM=$( jq -r '.keycloak.realm' "$CCCI_DEPS_FILE")
|
|
|
|
# Inject the OIDC client secret as an abra app secret (recipe-conventional name varies — match
|
|
# the recipe's .env.sample SECRET_*).
|
|
echo "$KC_SECRET" | abra app secret insert -n "$CCCI_APP_DOMAIN" oidc_rpcs v1 -
|
|
|
|
# Write the OIDC env vars to the parent .env (names per the recipe's .env.sample).
|
|
abra app config set "$CCCI_APP_DOMAIN" \
|
|
OIDC_REALM="$KC_REALM" \
|
|
OIDC_OP_DISCOVERY_ENDPOINT="https://${KC_DOMAIN}/realms/${KC_REALM}/.well-known/openid-configuration" \
|
|
OIDC_OP_AUTHORIZATION_URL="https://${KC_DOMAIN}/realms/${KC_REALM}/protocol/openid-connect/auth" \
|
|
OIDC_OP_TOKEN_URL="https://${KC_DOMAIN}/realms/${KC_REALM}/protocol/openid-connect/token" \
|
|
OIDC_OP_USER_URL="https://${KC_DOMAIN}/realms/${KC_REALM}/protocol/openid-connect/userinfo" \
|
|
OIDC_RP_CLIENT_ID="$KC_CLIENT" \
|
|
OIDC_RP_REDIRECT_URI="https://${CCCI_APP_DOMAIN}/auth/oidc/callback"
|
|
|
|
# Force an in-place redeploy of the affected services to pick up the new env. This is NOT a fresh
|
|
# app_new (deploy-count guard still 1 for this recipe).
|
|
abra app deploy --force --chaos --no-input "$CCCI_APP_DOMAIN"
|
|
```
|
|
|
|
The OIDC env-var **names are recipe-specific** (`OIDC_OP_*` for lasuite-docs, different prefixes
|
|
elsewhere). Read the recipe's `.env.sample` to see which keys the recipe expects; the *values* follow
|
|
this template. If a recipe needs more than this (extra group/claim mappings, etc.), extend its
|
|
`setup_custom_tests.sh` only — never the shared harness.
|
|
|
|
## 4. Test pattern: authenticated endpoints (mark + isolate)
|
|
|
|
- **Mark dep-requiring tests:** every custom test that needs the dep up + OIDC wired must use
|
|
`@pytest.mark.requires_deps`. The orchestrator skips these with reason `"deps-not-ready: <err>"`
|
|
if `setup_custom_tests` failed. Non-deps custom tests are unaffected by SSO setup failures.
|
|
- **Headless API tests** — use `harness.sso.oidc_password_grant` to mint an access token, then call
|
|
the recipe's authenticated endpoint with `Authorization: Bearer <token>`. Asserts on the response.
|
|
- **Browser flows (Playwright)** — navigate to the recipe, follow the redirect to keycloak, fill the
|
|
pre-provisioned test user's credentials, return to the recipe, exercise the UI. (Use the
|
|
pre-provisioned `ci-user@example.com` / known password the realm setup creates.)
|
|
- **The realm/client is fresh per run** — no cross-run state, no shared accounts. The realm setup
|
|
creates one or more test users with known passwords (pass-through from a per-run secret) so the
|
|
tests can authenticate without prompts.
|
|
|
|
## 5. Concrete recipes that use this pattern (Phase-2 scope)
|
|
|
|
These are **loop work** under this plan, not deferred:
|
|
|
|
- **lasuite-docs** — `DEPS = ["keycloak"]`; ports the upstream `oidc_login.py` +
|
|
`upload_conversion.py` parity tests + the §4.3-prescribed `create-a-doc + read-back via
|
|
authenticated /api/v1.0/documents/`. (Re-enters `DEFERRED.md` entry #5 — this plan IS the
|
|
re-entry, not operator input.)
|
|
- **cryptpad** — `DEPS = ["keycloak"]` (cryptpad upstream tests use authentik, but a keycloak-backed
|
|
cryptpad OIDC test is equally valid and uses the same primitives). The cryptpad create-a-pad
|
|
Playwright test (DEFERRED #6) is a separate concern — that one really does need a stable
|
|
CryptPad app-launch contract; it stays deferred.
|
|
- **lasuite-drive, lasuite-meet** — same pattern when mirrored (`recipe-create-pr` skill — loop work).
|
|
- Any future recipe that requires OIDC follows this plan; no operator handoff.
|
|
|
|
## 6. What stays deferred (genuinely operator-input)
|
|
|
|
- **authentik enrollment + `setup_authentik_realm` backend** (DEFERRED #9) — **RESOLVED (operator,
|
|
2026-05-29): keycloak is our default SSO provider; default ALL recipe OIDC tests to keycloak.** Do
|
|
NOT test authentik↔keycloak integration, and do NOT enroll authentik just to "prove pluggability"
|
|
— **Phase-2 DONE is NOT gated on authentik.** Enroll authentik + add `setup_authentik_realm` ONLY
|
|
if/when a recipe genuinely **requires** authentik (won't work under keycloak). If a recipe works
|
|
with keycloak, use keycloak. So DEFERRED #9's re-entry trigger narrows to "a recipe requires
|
|
authentik" — the cross-provider-coverage trigger is dropped. (E.g. cryptpad: its upstream test
|
|
uses authentik, but test it under **keycloak** — equally valid.)
|
|
- The `--extra` flag IDEA is **not** a precondition for this plan; OIDC-dep tests are part
|
|
of the default suite for the recipes that need them.
|
|
|
|
## 7. Definition of done for this pattern
|
|
- [ ] `DEPS = [...]` honored by `runner/run_recipe_ci.py`, with the **deps-AFTER-generic** ordering
|
|
(§1): deps deploy + `setup_custom_tests` step runs between RESTORE and CUSTOM tiers;
|
|
`$CCCI_DEPS_FILE` written; deps torn down LAST in reverse order.
|
|
- [ ] **Failure isolation proven:** a forced `setup_custom_tests` failure (e.g. simulate keycloak
|
|
realm-setup error) yields a run where generic tiers report **pass** and CUSTOM
|
|
`requires_deps` tests report **skip(deps-not-ready)** — no false fail of the generic tier,
|
|
no aborted run.
|
|
- [ ] **lasuite-docs** ships `tests/lasuite-docs/setup_custom_tests.sh` per §3.2 + authenticated
|
|
tests per §4 marked `@pytest.mark.requires_deps` (closes DEFERRED #5 — keep the entry there
|
|
with the closing commit, do not re-defer).
|
|
- [ ] At least one other OIDC-dep recipe (cryptpad oidc_login or a lasuite-* once mirrored) lands
|
|
cold-green using the same pattern, demonstrating reuse.
|
|
- [ ] `docs/sso-dep-testing.md` (in the cc-ci repo) explains the pattern for future recipe
|
|
enrollments — link to this plan.
|
|
- [ ] Adversary cold-verifies the full run for one such recipe + the forced-failure isolation
|
|
case, posts PASS in `REVIEW-2.md`.
|
|
|
|
## 8. Mirror+enroll reminder (also loop work)
|
|
|
|
If a recipe in scope (e.g. `lasuite-drive`, `lasuite-meet`, `immich`) **isn't mirrored to
|
|
`git.autonomic.zone/recipe-maintainers/`**, mirror it autonomously via the `recipe-create-pr` skill
|
|
at `/srv/recipe-maintainer/.opencode/skills/recipe-create-pr/SKILL.md` (see also
|
|
`plan-phase2-recipe-tests.md §0b`). Mirror+enroll is **not** operator-pending; the bot is admin on
|
|
the org.
|