docs: P6 — rewrite customization docs to the restructured end state (rcust)
All checks were successful
continuous-integration/drone/push Build is passing
All checks were successful
continuous-integration/drone/push Build is passing
recipe-customization.md: review spec -> reference. Single registry-backed loader + validation rules + HookCtx convention (§4); generated key table kept byte-identical (sync test); §5 end-state shape (op_state/deps fixtures, ctx ops.py, placement rule, first-class compose.ccci.yml, no setup_custom_tests.sh); §7 manifest block + dev-only CCCI_SKIP_GENERIC*; §8 rewritten as restructure outcomes (R1/R2/R3/R5/R6/R7/R8 resolved + how, R4 mitigated by manifest, R9 rejected-by-decision); §9 index updated to the new symbols. testing.md: install-time deps isolation replaces the setup_custom_tests step in the invariant (generic still never depends on custom — failure isolation via requires_deps/F2-11); ops.py example to pre_<op>(ctx); placement rule; generic opt-out now documented LOCAL-DEV-ONLY env with CI !! warning (declarative SKIP_GENERIC gone); partial key list points at the generated table. enroll-recipe.md: tree + worked examples updated (lasuite-docs install-time OIDC wiring + install_steps.sh; mumble post-F2-14c shape — UPGRADE_EXTRA_ENV native overlay, private _ constants, no CHAOS_BASE_DEPLOY); deps fixture (entry.domain) replaces deps_apps; ctx hook signatures; compose.ccci.yml first-class bullet; key list points at the generated table. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@ -14,8 +14,9 @@ those are discovered and run against the live app (D4 — see below).
|
||||
```
|
||||
tests/<recipe>/
|
||||
├── recipe_meta.py # optional per-recipe harness config (see below)
|
||||
├── install_steps.sh # optional custom install-steps hook (pre-deploy setup)
|
||||
├── ops.py # optional pre-op seed hooks (pre_install/pre_upgrade/pre_backup/pre_restore)
|
||||
├── install_steps.sh # optional custom install-steps hook (pre-deploy setup + deps env wiring)
|
||||
├── compose.ccci.yml # optional CI-only compose overlay (harness-copied, auto-chaos base deploy)
|
||||
├── ops.py # optional pre_<op>(ctx) seed hooks (install/upgrade/backup/restore)
|
||||
├── test_install.py # optional install overlay (runs ADDITIVELY alongside generic)
|
||||
├── test_upgrade.py # optional upgrade overlay (runs ADDITIVELY alongside generic)
|
||||
├── test_backup.py # optional backup overlay (runs ADDITIVELY alongside generic)
|
||||
@ -39,11 +40,14 @@ To add recipe-specific coverage, drop a `tests/<recipe>/test_<op>.py` **overlay*
|
||||
**ALONGSIDE** the generic for that op (HC3 additive, Phase 1e); the generic floor is never silently
|
||||
dropped. Overlays are **assertion-only** against the shared live deployment (the `live_app` fixture;
|
||||
they never perform the op or deploy/teardown — the orchestrator owns those). If the overlay needs to
|
||||
SEED pre-op state (data-continuity markers, the backup→restore divergence), put `pre_<op>(domain,
|
||||
meta)` callables in `tests/<recipe>/ops.py` — the orchestrator runs them BEFORE the op. Copy an
|
||||
SEED pre-op state (data-continuity markers, the backup→restore divergence), put `pre_<op>(ctx)`
|
||||
callables in `tests/<recipe>/ops.py` — the orchestrator runs them BEFORE the op (`ctx` is the
|
||||
uniform `HookCtx` every hook receives — `docs/recipe-customization.md` §4.1). Copy an
|
||||
existing recipe (`tests/custom-html/` simple/volume marker; `tests/keycloak/` admin-API; `tests/
|
||||
matrix-synapse/` `db`-service psql marker). **Do not edit the shared `tests/conftest.py` /
|
||||
`runner/harness/` to add a recipe** — set per-recipe knobs in `recipe_meta.py`:
|
||||
`runner/harness/` to add a recipe** — set per-recipe knobs in `recipe_meta.py` (the COMPLETE key
|
||||
reference is the generated table in `docs/recipe-customization.md` §4; unknown ALL-CAPS keys are
|
||||
hard errors, recipe-private constants are underscore-prefixed `_FOO`):
|
||||
|
||||
```python
|
||||
HEALTH_PATH = "/realms/master" # path that returns a healthy status (default "/")
|
||||
@ -51,9 +55,7 @@ HEALTH_OK = (200,) # acceptable status codes (default 200/301/302)
|
||||
DEPLOY_TIMEOUT = 600 # seconds for services to converge (default 600)
|
||||
HTTP_TIMEOUT = 600 # seconds for the app to answer (default 300)
|
||||
BACKUP_CAPABLE = True # override backup-capability auto-detect (default: scan compose)
|
||||
EXTRA_ENV = {"KEY": "value"} # or EXTRA_ENV(domain) -> dict; extra .env keys set at deploy
|
||||
SKIP_GENERIC = ["upgrade"] # per-recipe opt-out from the generic floor for the listed ops
|
||||
# ("all"/"*" = every op); rarely needed — generic is the floor
|
||||
EXTRA_ENV = {"KEY": "value"} # or EXTRA_ENV(ctx) -> dict; extra .env keys set at deploy
|
||||
```
|
||||
|
||||
Useful `harness.lifecycle` helpers for overlays: `http_get`, `http_fetch`, `http_body`,
|
||||
@ -76,9 +78,10 @@ Beyond the lifecycle overlays, each recipe carries (plan §4.1):
|
||||
- **`playwright/`** — browser flows where the recipe's core UX is a UI (P6).
|
||||
|
||||
The orchestrator's **custom** tier discovers `test_*.py` in `tests/<recipe>/{functional,playwright}/`
|
||||
(recursive, via `runner/harness/discovery.custom_tests`) and runs each as its own pytest against
|
||||
the same `live_app` shared deployment. Lifecycle-named files (`test_install.py`/etc.) are
|
||||
**excluded** from the custom tier — they live at the top level and run as lifecycle overlays.
|
||||
ONLY (the placement rule, via `runner/harness/discovery.custom_tests` — a top-level `test_*.py`
|
||||
is a lifecycle overlay and nothing else) and runs each as its own pytest against the same
|
||||
`live_app` shared deployment. Lifecycle-named files (`test_install.py`/etc.) are **excluded**
|
||||
from the custom tier even inside those subdirs (safety net against double-running).
|
||||
|
||||
### 2.2 Recipe-test dependencies — DEPS = [...] (Phase 2 Q2.3)
|
||||
|
||||
@ -89,23 +92,28 @@ them in `recipe_meta.py`:
|
||||
DEPS = ["keycloak"] # one entry per dep recipe name (cc-ci tests/<dep>/ must exist + work)
|
||||
```
|
||||
|
||||
The orchestrator (plan §4.2):
|
||||
1. Reads `DEPS` BEFORE deploying the recipe under test.
|
||||
2. Deploys each dep at a per-run domain `<dep[:4]>-<6hex>.ci.commoninternet.net` (the 6hex is
|
||||
hashed from `parent_recipe + pr + ref + dep_recipe` so two recipes' deps of the same kind do
|
||||
not collide on a single node).
|
||||
3. Waits each dep healthy using its own `recipe_meta.py` (HEALTH_PATH/HEALTH_OK/timeouts).
|
||||
4. Persists `[{"recipe": "<dep>", "domain": "<dep-domain>"}, ...]` to `$CCCI_DEPS_FILE`.
|
||||
5. Deploys + tests the recipe under test as usual.
|
||||
6. Tears down the dep LAST in `finally` (reverse declaration order, with `verify=True` — leaked
|
||||
The orchestrator (plan §4.2; install-time provisioning is the ONLY mode):
|
||||
1. Reads `DEPS` and provisions every dep **BEFORE the single deploy** of the recipe under test —
|
||||
each dep at a per-run domain `<dep[:4]>-<6hex>.ci.commoninternet.net` (the 6hex is hashed from
|
||||
`parent_recipe + pr + ref + dep_recipe` so two recipes' deps of the same kind do not collide on
|
||||
a single node), waited healthy using the dep's own `recipe_meta.py`.
|
||||
2. Persists the full per-dep identity + SSO creds dict to `$CCCI_DEPS_FILE` (jq-readable JSON,
|
||||
`{"<dep>": {"domain": ..., "realm": ..., "client_secret": ..., ...}}`).
|
||||
3. Deploys the recipe under test — its `install_steps.sh` reads `$CCCI_DEPS_FILE` and wires
|
||||
OIDC env into that ONE deploy (no post-deploy redeploy). A dep-provisioning failure does NOT
|
||||
block the run: the recipe deploys alone, generic tiers run, and `requires_deps` tests skip
|
||||
with a counted reason (F2-11).
|
||||
4. Tears down the dep LAST in `finally` (reverse declaration order, with `verify=True` — leaked
|
||||
deps fail the run loudly per §9 teardown sacred / F2-5 fix).
|
||||
|
||||
Tests access dep domains via the **`deps_apps` pytest fixture** (`tests/conftest.py`):
|
||||
Tests access deps via the **`deps` pytest fixture** (`tests/conftest.py`) — entries expose
|
||||
`.domain` plus the full creds dict (attribute or dict-style):
|
||||
|
||||
```python
|
||||
def test_my_recipe_uses_keycloak(live_app, deps_apps):
|
||||
assert "keycloak" in deps_apps, f"keycloak dep not deployed; {deps_apps}"
|
||||
kc_domain = deps_apps["keycloak"]
|
||||
@pytest.mark.requires_deps
|
||||
def test_my_recipe_uses_keycloak(live_app, deps):
|
||||
assert "keycloak" in deps, f"keycloak dep not deployed; {deps}"
|
||||
kc_domain = deps["keycloak"].domain
|
||||
…
|
||||
```
|
||||
|
||||
@ -120,7 +128,7 @@ For OIDC-dependent recipes, the shared `runner/harness/sso.py` provides:
|
||||
from harness import sso
|
||||
|
||||
creds = sso.setup_keycloak_realm(
|
||||
kc_domain, # = deps_apps["keycloak"]
|
||||
kc_domain, # = deps["keycloak"].domain
|
||||
realm="my-realm",
|
||||
client_id="my-client",
|
||||
redirect_uris=[f"https://{live_app}/*"],
|
||||
@ -144,10 +152,10 @@ ARE provider-pluggable.
|
||||
Not every recipe is a single HTTP app. `recipe_meta.py` + a few harness mechanisms cover the harder
|
||||
shapes (proven on mumble, mailu, and the SSO-dependent suite):
|
||||
|
||||
- **`EXTRA_ENV`** — a dict **or** a `callable(domain) -> dict`. The callable form derives values from
|
||||
the per-run domain (e.g. `MAIL_DOMAIN`/`HOSTNAMES` for mailu, `SANDBOX_DOMAIN` for cryptpad). Applied
|
||||
at every deploy (`abra.env_set`), so a recipe enrolls with NO shared-harness change.
|
||||
- **`READY_PROBE(domain) -> [...]`** — readiness signals beyond replica-convergence + the app's
|
||||
- **`EXTRA_ENV`** — a dict **or** a `callable(ctx) -> dict`. The callable form derives values from
|
||||
the per-run domain (`ctx.domain` — e.g. `MAIL_DOMAIN`/`HOSTNAMES` for mailu, `SANDBOX_DOMAIN` for
|
||||
cryptpad). Applied at every deploy (`abra.env_set`), so a recipe enrolls with NO shared-harness change.
|
||||
- **`READY_PROBE(ctx) -> [...]`** — readiness signals beyond replica-convergence + the app's
|
||||
`HEALTH_PATH`. Two probe shapes:
|
||||
- HTTP: `{"host": "...", "path": "/...", "ok": (200,)}` (e.g. lasuite-drive collabora WOPI discovery).
|
||||
- **TCP**: `{"tcp_host": "127.0.0.1", "tcp_port": 64738, "stable": 3}` — polls a socket connect N
|
||||
@ -155,16 +163,16 @@ shapes (proven on mumble, mailu, and the SSO-dependent suite):
|
||||
service (mumble: the mumble-web sidecar serves HTTP 200 while the voice server on 64738 is still
|
||||
rebinding after an upgrade redeploy — the TCP probe gates the backup tier until the voice server is
|
||||
actually up). Runs after install AND after the upgrade chaos redeploy.
|
||||
- **`CHAOS_BASE_DEPLOY = True`** — make the pinned base deploy use `--chaos` (skips abra's clean-tree +
|
||||
lint gates, still deploys the explicitly-checked-out pinned version, NOT latest). Needed when an
|
||||
`install_steps.sh` adds an UNTRACKED file to the recipe checkout (e.g. mumble copies a
|
||||
`compose.host-ports.yml` into versions that predate it) — abra's pinned-deploy clean-tree check would
|
||||
otherwise FATA. `abra.recipe_checkout` force-checks-out (`-f`) so the upgrade tier's re-checkout to
|
||||
PR-head overwrites such overlays cleanly.
|
||||
- **`compose.ccci.yml`** (first-class at `tests/<recipe>/compose.ccci.yml`) — a CI-only compose
|
||||
overlay the harness itself copies into the recipe checkout before the base deploy, automatically
|
||||
using `--chaos` for that deploy (the untracked file would otherwise trip abra's pinned-deploy
|
||||
clean-tree check). Reference it from `EXTRA_ENV`'s `COMPOSE_FILE`. Minimal, justified fallback
|
||||
only (e.g. ghost's 15m `start_period` grace). `abra.recipe_checkout` force-checks-out (`-f`) so
|
||||
the upgrade tier's re-checkout to PR-head overwrites such overlays cleanly.
|
||||
- **`install_steps.sh`** (auto-discovered at `tests/<recipe>/install_steps.sh`) — runs after
|
||||
`abra app new` + EXTRA_ENV + secret-generate, BEFORE the single deploy, with `CCCI_APP_DOMAIN` /
|
||||
`CCCI_APP_ENV` / `CCCI_RECIPE` (and `CCCI_DEPS_FILE` when DEPS are provisioned at install). Use it to
|
||||
drop a cc-ci-owned compose overlay into the checkout, wire dep-derived env/secrets, etc.
|
||||
`CCCI_APP_ENV` / `CCCI_RECIPE` (and `CCCI_DEPS_FILE` when the recipe declares DEPS — deps are
|
||||
always provisioned before the deploy). Use it to wire dep-derived env/secrets, seed config, etc.
|
||||
|
||||
**Non-HTTP protocol tests (mumble).** Reach a TCP service published `mode: host` (via a host-ports
|
||||
overlay) at `127.0.0.1:<port>` — cc-ci runs tests on-host (cc-ci-run). mumble ships a stdlib protocol
|
||||
@ -227,9 +235,10 @@ RECIPE=<recipe> PR=<n> REF=<sha-or-branch> SRC=recipe-maintainers/<recipe> \
|
||||
|
||||
```
|
||||
tests/lasuite-docs/
|
||||
├── recipe_meta.py # HEALTH_PATH="/", DEPLOY_TIMEOUT=900, EXTRA_ENV(domain) for cold-pull,
|
||||
├── recipe_meta.py # HEALTH_PATH="/", DEPLOY_TIMEOUT=900, EXTRA_ENV(ctx) for cold-pull,
|
||||
│ # DEPS=["keycloak"] ← Phase 2 dep declaration
|
||||
├── ops.py # pre_<op> seed hooks (volume marker for backup/restore data-integrity)
|
||||
├── install_steps.sh # wires OIDC env from $CCCI_DEPS_FILE into the single deploy
|
||||
├── ops.py # pre_<op>(ctx) seed hooks (volume marker for backup/restore data-integrity)
|
||||
├── test_install.py # lifecycle install overlay (Playwright frontend SPA load)
|
||||
├── test_upgrade.py # lifecycle upgrade overlay (marker survives chaos redeploy)
|
||||
├── test_backup.py # lifecycle backup overlay (marker captured)
|
||||
@ -239,12 +248,14 @@ tests/lasuite-docs/
|
||||
├── test_health_check.py # parity port (SOURCE comment cites recipe-info file)
|
||||
├── test_auth_required.py # specific: /api/v1.0/users/me/ → 401 without auth
|
||||
└── test_oidc_with_keycloak.py # specific: full OIDC flow against the dep keycloak (uses
|
||||
# harness.sso primitives + deps_apps["keycloak"])
|
||||
# harness.sso primitives + the `deps` fixture)
|
||||
```
|
||||
|
||||
`!testme` on a lasuite-docs PR drives the orchestrator to:
|
||||
1. Deploy the per-run keycloak dep (`keyc-<6hex>.ci.commoninternet.net`) and wait healthy.
|
||||
2. Deploy lasuite-docs (`lasu-<6hex>.ci.commoninternet.net`).
|
||||
1. Provision the per-run keycloak dep (`keyc-<6hex>.ci.commoninternet.net`), wait healthy, write
|
||||
creds to `$CCCI_DEPS_FILE` — BEFORE the recipe deploy.
|
||||
2. Deploy lasuite-docs (`lasu-<6hex>.ci.commoninternet.net`); `install_steps.sh` wires the OIDC
|
||||
env into that one deploy.
|
||||
3. Run install / upgrade / backup / restore + the 3 functional tests against the shared
|
||||
deployment (custom tier).
|
||||
4. Teardown lasuite-docs, then the keycloak dep (LAST), both with verify=True.
|
||||
@ -254,12 +265,13 @@ tests/lasuite-docs/
|
||||
### Other shapes (concrete references)
|
||||
|
||||
- **TCP / voice recipe — `tests/mumble/`**: `recipe_meta.py` (EXTRA_ENV sets
|
||||
`COMPOSE_FILE=compose.yml:compose.mumbleweb.yml:compose.host-ports.yml`, `WELCOME_TEXT`/`USERS`
|
||||
markers, `CHAOS_BASE_DEPLOY=True`, `READY_PROBE` TCP 64738), `install_steps.sh` (provides the
|
||||
host-ports overlay to older versions), `functional/_mumble_proto.py` + the protocol/config-round-trip
|
||||
`COMPOSE_FILE=compose.yml:compose.mumbleweb.yml` for the base; `UPGRADE_EXTRA_ENV` adds the
|
||||
native `compose.host-ports.yml` at PR-head so 64738 is host-published on latest; private
|
||||
`_WELCOME_TEXT_MARKER`/`_MAX_USERS` constants; `READY_PROBE(ctx)` TCP 64738 — phase-aware via
|
||||
the live COMPOSE_FILE), `functional/_mumble_proto.py` + the protocol/config-round-trip
|
||||
tests, `ops.py`/`test_backup.py`/`test_restore.py` (sqlite P4). See §2.4.
|
||||
- **Multi-service, dep-less, in-container functional — `tests/mailu/`**: `recipe_meta.py`
|
||||
(`EXTRA_ENV(domain)` with `TLS_FLAVOR=notls` + `MAIL_DOMAIN`/`HOSTNAMES`/`TRAEFIK_STACK_NAME`),
|
||||
(`EXTRA_ENV(ctx)` with `TLS_FLAVOR=notls` + `MAIL_DOMAIN`/`HOSTNAMES`/`TRAEFIK_STACK_NAME`),
|
||||
`functional/_mailu.py` (flask-CLI helpers), `test_mailbox.py` (create→config-export read-back),
|
||||
`test_mail_flow.py` (in-container sendmail→doveadm delivery). No backupbot → P4 N/A (PARITY.md +
|
||||
DEFERRED.md). See §2.4.
|
||||
|
||||
Reference in New Issue
Block a user