docs: P6 — rewrite customization docs to the restructured end state (rcust)
All checks were successful
continuous-integration/drone/push Build is passing

recipe-customization.md: review spec -> reference. Single registry-backed loader + validation
rules + HookCtx convention (§4); generated key table kept byte-identical (sync test); §5 end-state
shape (op_state/deps fixtures, ctx ops.py, placement rule, first-class compose.ccci.yml, no
setup_custom_tests.sh); §7 manifest block + dev-only CCCI_SKIP_GENERIC*; §8 rewritten as
restructure outcomes (R1/R2/R3/R5/R6/R7/R8 resolved + how, R4 mitigated by manifest, R9
rejected-by-decision); §9 index updated to the new symbols.

testing.md: install-time deps isolation replaces the setup_custom_tests step in the invariant
(generic still never depends on custom — failure isolation via requires_deps/F2-11); ops.py
example to pre_<op>(ctx); placement rule; generic opt-out now documented LOCAL-DEV-ONLY env with
CI !! warning (declarative SKIP_GENERIC gone); partial key list points at the generated table.

enroll-recipe.md: tree + worked examples updated (lasuite-docs install-time OIDC wiring +
install_steps.sh; mumble post-F2-14c shape — UPGRADE_EXTRA_ENV native overlay, private _
constants, no CHAOS_BASE_DEPLOY); deps fixture (entry.domain) replaces deps_apps; ctx hook
signatures; compose.ccci.yml first-class bullet; key list points at the generated table.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
autonomic-bot
2026-06-10 19:07:41 +00:00
parent 68954be53e
commit da558ca946
3 changed files with 268 additions and 272 deletions

View File

@ -16,12 +16,13 @@ year from now, this is the one rule that should still hold.
ship as the floor for every recipe. No SSO provider, no external deps, no per-recipe state
scaffolding — just "does this recipe deploy and lifecycle work?"
- **Generic must not depend on custom.** A custom test or a custom-tests setup (e.g. SSO/OIDC dep
provisioning) **can never be a precondition for the generic tier to pass.** Concretely: the
orchestrator runs all generic tiers (install → upgrade → backup → restore) against the recipe
**alone, with no deps deployed**, then runs the `setup_custom_tests` step (deps + post-deps
wiring) only after — and a failure there is **isolated** to the custom tier (tests tagged
`@pytest.mark.requires_deps` skip with reason `"deps-not-ready"`; generic tier reports
normally). See `cc-ci-plan/plan-sso-dep-testing.md` for the SSO-dep specifics.
provisioning) **can never be a precondition for the generic tier to pass.** Concretely: deps are
provisioned BEFORE the single deploy (so `install_steps.sh` can wire OIDC env into that one
deploy), but a dep-provisioning failure is **isolated** to the custom tier — the recipe still
deploys alone, every generic tier (install → upgrade → backup → restore) runs normally, and
tests tagged `@pytest.mark.requires_deps` skip with reason `"deps-not-ready"` (a counted,
reported skip — F2-11). A deps failure can never fail or block a generic tier. See
`cc-ci-plan/plan-sso-dep-testing.md` for the SSO-dep specifics.
- **Custom tests are the thoroughness layer — and they cost more to maintain.** They're more
thorough (authenticated APIs, multi-app flows, version-specific browser selectors, helper
scripts, state-management) and *therefore* take more maintenance: an SSO provider's admin API
@ -113,9 +114,11 @@ repo-local <recipe-repo>/tests/test_<op>.py (upstream-authoritative; gated
Only ONE overlay source wins for a given op (repo-local > cc-ci); the generic floor runs **in
addition** unless explicitly opted out.
**Custom (non-lifecycle) `test_*.py`** — any other `test_*.py` (e.g. `test_sso.py`) is **opt-in and
additive**: it has no generic equivalent and runs only when present, discovered from both locations
(repo-local gated by the HC2 allowlist).
**Custom (non-lifecycle) tests** — e.g. `functional/test_sso.py` — are **opt-in and additive**:
they have no generic equivalent and run only when present, discovered from both locations
(repo-local gated by the HC2 allowlist). Placement rule: custom tests live ONLY under
`functional/` or `playwright/`; a top-level `test_*.py` is a lifecycle overlay and nothing else
(top-level non-lifecycle files are not discovered).
### Pre-op seed hooks (per-recipe `ops.py`)
@ -127,35 +130,38 @@ etc.). Since the orchestrator owns the op, overlays place their seed in an optio
# tests/<recipe>/ops.py
from harness import lifecycle
def pre_upgrade(domain, meta):
def pre_upgrade(ctx):
# seed a marker before the harness performs the upgrade
lifecycle.exec_in_app(domain, ["sh", "-c", "echo upgrade-survives > /path/marker"])
lifecycle.exec_in_app(ctx.domain, ["sh", "-c", "echo upgrade-survives > /path/marker"])
def pre_backup(domain, meta):
def pre_backup(ctx):
# establish a known "original" state before the backup op captures it
lifecycle.exec_in_app(domain, ["sh", "-c", "echo original > /path/marker"])
lifecycle.exec_in_app(ctx.domain, ["sh", "-c", "echo original > /path/marker"])
def pre_restore(domain, meta):
def pre_restore(ctx):
# diverge from the backed-up state so a successful restore is observable
lifecycle.exec_in_app(domain, ["sh", "-c", "echo mutated > /path/marker"])
lifecycle.exec_in_app(ctx.domain, ["sh", "-c", "echo mutated > /path/marker"])
```
The orchestrator imports `ops.py` in-process (with the recipe dir on `sys.path`, so it can import
sibling helpers like `kc_admin.py`) and calls `pre_<op>(domain, meta)` immediately before performing
the op. Then `test_<op>.py` asserts the post-op state. See `tests/custom-html/` (volume marker),
sibling helpers like `kc_admin.py`) and calls `pre_<op>(ctx)` immediately before performing the
op — `ctx` is the uniform `HookCtx` every recipe hook receives (`.domain`, `.base_url`, `.meta`,
`.deps`, `.op``docs/recipe-customization.md` §4.1). Then `test_<op>.py` asserts the post-op
state. See `tests/custom-html/` (volume marker),
`tests/keycloak/` (admin-API/realm), `tests/matrix-synapse/`, `tests/lasuite-docs/` (psql in the `db`
service) for worked examples.
### Opting out of the generic floor
### Opting out of the generic floor (LOCAL-DEV-ONLY)
The generic runs additively by default. To skip it (e.g. when an overlay's recipe-specific check
fully replaces the generic's mechanism check) set, in increasing specificity:
The generic runs additively by default and there is **no declarative opt-out** — no recipe can
ship without the floor. For local iteration only (e.g. re-running one tier while developing an
overlay), two env escape hatches exist:
- **env `CCCI_SKIP_GENERIC=1`** — skip generic for ALL ops (run-wide).
- **env `CCCI_SKIP_GENERIC_<OP>=1`** — e.g. `CCCI_SKIP_GENERIC_UPGRADE=1` — skip generic for that one op.
- **declarative in `recipe_meta.py`** — `SKIP_GENERIC = ["upgrade"]` (per-op) or `SKIP_GENERIC = ["all"]`.
Opting out is per-recipe and visible in git — not a hidden global. Truthy = `1`/`true`/`yes`/`on`.
Truthy = `1`/`true`/`yes`/`on`. If either is active in a CI (drone) run, the run prints a loud
`!!` warning and the customization manifest records it (`docs/recipe-customization.md` §7).
## Repo-local trust gate (HC2) — default-deny
@ -215,12 +221,14 @@ installs and stays 1.
`tests/custom-html/test_upgrade.py`). Assert the POST-op state — reading app state through
`lifecycle.exec_in_app` (volume/DB) for data checks, not HTTP. Generic + your overlay both run.
3. If the overlay needs to seed PRE-op state (data-continuity markers, the backup→restore
divergence), drop `tests/<recipe>/ops.py` with `pre_upgrade/pre_backup/pre_restore(domain, meta)`.
divergence), drop `tests/<recipe>/ops.py` with `pre_upgrade/pre_backup/pre_restore(ctx)`.
4. If the recipe needs install-time setup, add `tests/<recipe>/install_steps.sh`.
5. Set per-recipe knobs (health path, timeouts, opt-out) in `recipe_meta.py`.
5. Set per-recipe knobs (health path, timeouts) in `recipe_meta.py`.
6. **Never weaken or skip an assertion to make a run pass** — a red tier is information.
Per-recipe config (`tests/<recipe>/recipe_meta.py`, all optional):
Per-recipe config (`tests/<recipe>/recipe_meta.py`, all optional — the COMPLETE key reference is
the generated table in `docs/recipe-customization.md` §4; unknown keys are hard errors, private
constants are underscore-prefixed):
```python
HEALTH_PATH = "/realms/master" # path that returns a healthy status (default "/")
@ -228,8 +236,7 @@ HEALTH_OK = (200,) # acceptable status codes (default 200/301/302)
DEPLOY_TIMEOUT = 600 # seconds for services to converge (default 600)
HTTP_TIMEOUT = 600 # seconds for the app to answer (default 300)
BACKUP_CAPABLE = True # override backup-capability auto-detection (default: scan compose)
EXTRA_ENV = {"KEY": "value"} # or EXTRA_ENV(domain) -> dict; extra .env keys set at deploy
SKIP_GENERIC = ["upgrade"] # per-recipe declarative opt-out from generic ops ("all" = every op)
EXTRA_ENV = {"KEY": "value"} # or EXTRA_ENV(ctx) -> dict; extra .env keys set at deploy
```
The harness self-tests for discovery / precedence / the HC2 allowlist live in `tests/unit/` (run: