docs: P6 — rewrite customization docs to the restructured end state (rcust)
All checks were successful
continuous-integration/drone/push Build is passing
All checks were successful
continuous-integration/drone/push Build is passing
recipe-customization.md: review spec -> reference. Single registry-backed loader + validation rules + HookCtx convention (§4); generated key table kept byte-identical (sync test); §5 end-state shape (op_state/deps fixtures, ctx ops.py, placement rule, first-class compose.ccci.yml, no setup_custom_tests.sh); §7 manifest block + dev-only CCCI_SKIP_GENERIC*; §8 rewritten as restructure outcomes (R1/R2/R3/R5/R6/R7/R8 resolved + how, R4 mitigated by manifest, R9 rejected-by-decision); §9 index updated to the new symbols. testing.md: install-time deps isolation replaces the setup_custom_tests step in the invariant (generic still never depends on custom — failure isolation via requires_deps/F2-11); ops.py example to pre_<op>(ctx); placement rule; generic opt-out now documented LOCAL-DEV-ONLY env with CI !! warning (declarative SKIP_GENERIC gone); partial key list points at the generated table. enroll-recipe.md: tree + worked examples updated (lasuite-docs install-time OIDC wiring + install_steps.sh; mumble post-F2-14c shape — UPGRADE_EXTRA_ENV native overlay, private _ constants, no CHAOS_BASE_DEPLOY); deps fixture (entry.domain) replaces deps_apps; ctx hook signatures; compose.ccci.yml first-class bullet; key list points at the generated table. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@ -16,12 +16,13 @@ year from now, this is the one rule that should still hold.
|
||||
ship as the floor for every recipe. No SSO provider, no external deps, no per-recipe state
|
||||
scaffolding — just "does this recipe deploy and lifecycle work?"
|
||||
- **Generic must not depend on custom.** A custom test or a custom-tests setup (e.g. SSO/OIDC dep
|
||||
provisioning) **can never be a precondition for the generic tier to pass.** Concretely: the
|
||||
orchestrator runs all generic tiers (install → upgrade → backup → restore) against the recipe
|
||||
**alone, with no deps deployed**, then runs the `setup_custom_tests` step (deps + post-deps
|
||||
wiring) only after — and a failure there is **isolated** to the custom tier (tests tagged
|
||||
`@pytest.mark.requires_deps` skip with reason `"deps-not-ready"`; generic tier reports
|
||||
normally). See `cc-ci-plan/plan-sso-dep-testing.md` for the SSO-dep specifics.
|
||||
provisioning) **can never be a precondition for the generic tier to pass.** Concretely: deps are
|
||||
provisioned BEFORE the single deploy (so `install_steps.sh` can wire OIDC env into that one
|
||||
deploy), but a dep-provisioning failure is **isolated** to the custom tier — the recipe still
|
||||
deploys alone, every generic tier (install → upgrade → backup → restore) runs normally, and
|
||||
tests tagged `@pytest.mark.requires_deps` skip with reason `"deps-not-ready"` (a counted,
|
||||
reported skip — F2-11). A deps failure can never fail or block a generic tier. See
|
||||
`cc-ci-plan/plan-sso-dep-testing.md` for the SSO-dep specifics.
|
||||
- **Custom tests are the thoroughness layer — and they cost more to maintain.** They're more
|
||||
thorough (authenticated APIs, multi-app flows, version-specific browser selectors, helper
|
||||
scripts, state-management) and *therefore* take more maintenance: an SSO provider's admin API
|
||||
@ -113,9 +114,11 @@ repo-local <recipe-repo>/tests/test_<op>.py (upstream-authoritative; gated
|
||||
Only ONE overlay source wins for a given op (repo-local > cc-ci); the generic floor runs **in
|
||||
addition** unless explicitly opted out.
|
||||
|
||||
**Custom (non-lifecycle) `test_*.py`** — any other `test_*.py` (e.g. `test_sso.py`) is **opt-in and
|
||||
additive**: it has no generic equivalent and runs only when present, discovered from both locations
|
||||
(repo-local gated by the HC2 allowlist).
|
||||
**Custom (non-lifecycle) tests** — e.g. `functional/test_sso.py` — are **opt-in and additive**:
|
||||
they have no generic equivalent and run only when present, discovered from both locations
|
||||
(repo-local gated by the HC2 allowlist). Placement rule: custom tests live ONLY under
|
||||
`functional/` or `playwright/`; a top-level `test_*.py` is a lifecycle overlay and nothing else
|
||||
(top-level non-lifecycle files are not discovered).
|
||||
|
||||
### Pre-op seed hooks (per-recipe `ops.py`)
|
||||
|
||||
@ -127,35 +130,38 @@ etc.). Since the orchestrator owns the op, overlays place their seed in an optio
|
||||
# tests/<recipe>/ops.py
|
||||
from harness import lifecycle
|
||||
|
||||
def pre_upgrade(domain, meta):
|
||||
def pre_upgrade(ctx):
|
||||
# seed a marker before the harness performs the upgrade
|
||||
lifecycle.exec_in_app(domain, ["sh", "-c", "echo upgrade-survives > /path/marker"])
|
||||
lifecycle.exec_in_app(ctx.domain, ["sh", "-c", "echo upgrade-survives > /path/marker"])
|
||||
|
||||
def pre_backup(domain, meta):
|
||||
def pre_backup(ctx):
|
||||
# establish a known "original" state before the backup op captures it
|
||||
lifecycle.exec_in_app(domain, ["sh", "-c", "echo original > /path/marker"])
|
||||
lifecycle.exec_in_app(ctx.domain, ["sh", "-c", "echo original > /path/marker"])
|
||||
|
||||
def pre_restore(domain, meta):
|
||||
def pre_restore(ctx):
|
||||
# diverge from the backed-up state so a successful restore is observable
|
||||
lifecycle.exec_in_app(domain, ["sh", "-c", "echo mutated > /path/marker"])
|
||||
lifecycle.exec_in_app(ctx.domain, ["sh", "-c", "echo mutated > /path/marker"])
|
||||
```
|
||||
|
||||
The orchestrator imports `ops.py` in-process (with the recipe dir on `sys.path`, so it can import
|
||||
sibling helpers like `kc_admin.py`) and calls `pre_<op>(domain, meta)` immediately before performing
|
||||
the op. Then `test_<op>.py` asserts the post-op state. See `tests/custom-html/` (volume marker),
|
||||
sibling helpers like `kc_admin.py`) and calls `pre_<op>(ctx)` immediately before performing the
|
||||
op — `ctx` is the uniform `HookCtx` every recipe hook receives (`.domain`, `.base_url`, `.meta`,
|
||||
`.deps`, `.op` — `docs/recipe-customization.md` §4.1). Then `test_<op>.py` asserts the post-op
|
||||
state. See `tests/custom-html/` (volume marker),
|
||||
`tests/keycloak/` (admin-API/realm), `tests/matrix-synapse/`, `tests/lasuite-docs/` (psql in the `db`
|
||||
service) for worked examples.
|
||||
|
||||
### Opting out of the generic floor
|
||||
### Opting out of the generic floor (LOCAL-DEV-ONLY)
|
||||
|
||||
The generic runs additively by default. To skip it (e.g. when an overlay's recipe-specific check
|
||||
fully replaces the generic's mechanism check) set, in increasing specificity:
|
||||
The generic runs additively by default and there is **no declarative opt-out** — no recipe can
|
||||
ship without the floor. For local iteration only (e.g. re-running one tier while developing an
|
||||
overlay), two env escape hatches exist:
|
||||
|
||||
- **env `CCCI_SKIP_GENERIC=1`** — skip generic for ALL ops (run-wide).
|
||||
- **env `CCCI_SKIP_GENERIC_<OP>=1`** — e.g. `CCCI_SKIP_GENERIC_UPGRADE=1` — skip generic for that one op.
|
||||
- **declarative in `recipe_meta.py`** — `SKIP_GENERIC = ["upgrade"]` (per-op) or `SKIP_GENERIC = ["all"]`.
|
||||
|
||||
Opting out is per-recipe and visible in git — not a hidden global. Truthy = `1`/`true`/`yes`/`on`.
|
||||
Truthy = `1`/`true`/`yes`/`on`. If either is active in a CI (drone) run, the run prints a loud
|
||||
`!!` warning and the customization manifest records it (`docs/recipe-customization.md` §7).
|
||||
|
||||
## Repo-local trust gate (HC2) — default-deny
|
||||
|
||||
@ -215,12 +221,14 @@ installs and stays 1.
|
||||
`tests/custom-html/test_upgrade.py`). Assert the POST-op state — reading app state through
|
||||
`lifecycle.exec_in_app` (volume/DB) for data checks, not HTTP. Generic + your overlay both run.
|
||||
3. If the overlay needs to seed PRE-op state (data-continuity markers, the backup→restore
|
||||
divergence), drop `tests/<recipe>/ops.py` with `pre_upgrade/pre_backup/pre_restore(domain, meta)`.
|
||||
divergence), drop `tests/<recipe>/ops.py` with `pre_upgrade/pre_backup/pre_restore(ctx)`.
|
||||
4. If the recipe needs install-time setup, add `tests/<recipe>/install_steps.sh`.
|
||||
5. Set per-recipe knobs (health path, timeouts, opt-out) in `recipe_meta.py`.
|
||||
5. Set per-recipe knobs (health path, timeouts) in `recipe_meta.py`.
|
||||
6. **Never weaken or skip an assertion to make a run pass** — a red tier is information.
|
||||
|
||||
Per-recipe config (`tests/<recipe>/recipe_meta.py`, all optional):
|
||||
Per-recipe config (`tests/<recipe>/recipe_meta.py`, all optional — the COMPLETE key reference is
|
||||
the generated table in `docs/recipe-customization.md` §4; unknown keys are hard errors, private
|
||||
constants are underscore-prefixed):
|
||||
|
||||
```python
|
||||
HEALTH_PATH = "/realms/master" # path that returns a healthy status (default "/")
|
||||
@ -228,8 +236,7 @@ HEALTH_OK = (200,) # acceptable status codes (default 200/301/302)
|
||||
DEPLOY_TIMEOUT = 600 # seconds for services to converge (default 600)
|
||||
HTTP_TIMEOUT = 600 # seconds for the app to answer (default 300)
|
||||
BACKUP_CAPABLE = True # override backup-capability auto-detection (default: scan compose)
|
||||
EXTRA_ENV = {"KEY": "value"} # or EXTRA_ENV(domain) -> dict; extra .env keys set at deploy
|
||||
SKIP_GENERIC = ["upgrade"] # per-recipe declarative opt-out from generic ops ("all" = every op)
|
||||
EXTRA_ENV = {"KEY": "value"} # or EXTRA_ENV(ctx) -> dict; extra .env keys set at deploy
|
||||
```
|
||||
|
||||
The harness self-tests for discovery / precedence / the HC2 allowlist live in `tests/unit/` (run:
|
||||
|
||||
Reference in New Issue
Block a user