recipe-maintainers/cc-ci
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(1d): DG8 — docs/testing.md (generic suite + overlay convention + install-steps hook); update enroll-recipe.md to the deploy-once contract; README pointer

Browse Source 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
autonomic-bot
2026-05-28 01:35:16 +01:00
parent afd75a48db
commit b756e72cc2
3 changed files with 153 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
docs/enroll-recipe.md
View File

@ -14,26 +14,40 @@ those are discovered and run against the live app (D4 — see below).
```
tests/<recipe>/
├── recipe_meta.py # optional per-recipe harness config (see below)
├── test_install.py # install-stage assertions (health + Playwright)
├── test_upgrade.py # upgrade-stage assertions (data survives)
└── test_backup.py # backup→mutate→restore assertions
├── install_steps.sh # optional custom install-steps hook (pre-deploy setup)
├── test_install.py # optional install overlay (else the generic install tier runs)
├── test_upgrade.py # optional upgrade overlay (else the generic upgrade tier runs)
├── test_backup.py # optional backup overlay (else the generic backup tier runs)
└── test_restore.py # optional restore overlay (else the generic restore tier runs)
```
Copy from an existing recipe (e.g. `tests/custom-html/` for a simple app, `tests/keycloak/` for a
DB-backed one). The shared fixtures live in `tests/conftest.py` + `runner/harness/` — **do not edit
them to add a recipe**; instead set per-recipe config in `recipe_meta.py`:
**A recipe is testable with ZERO config:** with no overlay files, the **generic lifecycle suite**
runs (install/upgrade/backup/restore) against a single shared deployment — see `docs/testing.md` for
the full model (tiers, deploy-once, override-vs-extend, precedence, the install-steps hook). The
per-recipe dir only holds the bits where the recipe needs *more* than the generic.
To add recipe-specific coverage, drop a `tests/<recipe>/test_<op>.py` **overlay** (it OVERRIDES the
generic for that op; absent ⇒ generic runs). Overlays are **assertion-only** against the shared live
deployment (the `live_app` fixture; they never deploy), and reuse the generic op + serving check by
composition (`from harness import generic; generic.do_upgrade(...)` etc.), adding recipe-specific
assertions. Copy an existing overlay (`tests/custom-html/` simple/volume marker; `tests/keycloak/`
admin-API; `tests/matrix-synapse/` `db`-service psql marker). **Do not edit the shared
`tests/conftest.py` / `runner/harness/` to add a recipe** — set per-recipe config in `recipe_meta.py`:
```python
HEALTH_PATH = "/realms/master" # path that returns a healthy status (default "/")
HEALTH_OK = (200,) # acceptable status codes (default 200/301/302)
DEPLOY_TIMEOUT = 600 # seconds for services to converge (default 600)
HTTP_TIMEOUT = 600 # seconds for the app to answer (default 300)
BACKUP_CAPABLE = True # override backup-capability auto-detect (default: scan compose)
EXTRA_ENV = {"KEY": "value"} # or EXTRA_ENV(domain) -> dict; extra .env keys set at deploy
```
The test files use the fixtures: `deployed_app` (install), `deployed` (function-scoped), and the
`harness.lifecycle` helpers (`http_get`, `http_body`, `exec_in_app`, `upgrade_app`, `backup_app`,
`restore_app`, `previous_version`). The harness forces `LETS_ENCRYPT_ENV=""` (no ACME) and a unique
short domain per run, and guarantees teardown.
Useful `harness.lifecycle` helpers for overlays: `http_get`, `http_fetch`, `http_body`,
`exec_in_app` (use this for data markers — volume/DB, robust to the serving layer); the lifecycle ops
themselves come from `harness.generic` (`assert_serving`, `do_upgrade`, `do_backup`, `do_restore`).
The harness forces `LETS_ENCRYPT_ENV=""` (no ACME), a unique short domain per run, and guarantees
teardown.
## 3. Recipe-local tests (D4)