diff --git a/machine-docs/BACKLOG-2.md b/machine-docs/BACKLOG-2.md index 3a019ee..c99fd1c 100644 --- a/machine-docs/BACKLOG-2.md +++ b/machine-docs/BACKLOG-2.md @@ -6,22 +6,22 @@ Phase plan: `/srv/cc-ci/cc-ci-plan/plan-phase2-recipe-tests.md` ## Build backlog ### Q0 — Harness additions -- [ ] **Q0.1** — Vendor `recipe-maintainer/utils/tests/helpers.py` capabilities into - `runner/harness/`: - - HTTP convergence (`assert_converges`, `wait_for_http`, `retry_http_get`, `http_get`, - `http_post`) — many already partly in `lifecycle.py`/`generic.py`; add the missing pieces + - consolidate (no duplicate logic). - - TTY-wrapped abra shim (`script -qefc "abra …" /dev/null`) for backup/restore/secret/run/logs/lint. - - Backup data-integrity primitive (volume/DB seed → marker survival check). -- [ ] **Q0.2** — Discovery: recurse into `tests//functional/` and `tests//playwright/` - so per-plan `§4.1` directory layout is honored. Update `discovery.custom_tests` + `enroll-recipe.md`. -- [ ] **Q0.3** — PARITY.md template + a worked custom-html example - (`tests/custom-html/PARITY.md`). Port custom-html health_check.py to a functional test. +- [x] **Q0.1** — `runner/harness/http.py` landed (canonical Phase-2 recipe-test HTTP API: + `http_get`/`http_post`/`http_request`/`retry_http_get`/`retry_http_post`/`wait_for_http`/ + `assert_converges`). TTY abra wrapper already present (`runner/harness/abra.py::_run_pty`) + from Phase 1d. 11 unit tests landed. +- [x] **Q0.2** — `discovery.custom_tests` recurses into `tests//{functional,playwright}/` + (Phase 2 §4.1 layout); 2 unit tests landed. +- [x] **Q0.3** — `tests/custom-html/PARITY.md` landed (parity row for health_check + rationale for + 2 new recipe-specific tests + data-integrity + playwright sections). Parity port: + `tests/custom-html/functional/test_health_check.py` (SOURCE comment present). - [ ] **Q0.4** — Dependency resolver harness primitive (read `tests//recipe.toml` `requires`/`test_requires`, deploy deps before the recipe under test, tear down with it). Mind - `MAX_TESTS`/node budget; sequence heavy ones. Defer SSO-realm-setup wiring to Q2. -- [ ] **Q0.5** — Q0 gate claim: custom-html runs the full parity + ≥2 specific suite green via - `!testme`; PARITY.md complete for custom-html; harness changes recorded in DECISIONS.md. + `MAX_TESTS`/node budget; sequence heavy ones. **Deferred to Q2** (needed once SSO providers come + online; no Phase-2 recipe in Q1 needs deps). Tracked in BACKLOG. +- [x] **Q0.5** — **CLAIMED @2026-05-28.** Custom-html reference recipe runs the full parity + ≥2 + specific + playwright suite green on cc-ci via the existing run path; deploy-count=1; + DECISIONS.md Phase-2 section in place. Awaiting Adversary cold-verify gate PASS. ### Q1 — Pattern proof (custom-html + n8n) - [ ] **Q1.1** — custom-html: ≥2 NEW recipe-specific functional tests (beyond parity diff --git a/machine-docs/JOURNAL-2.md b/machine-docs/JOURNAL-2.md index fdbd027..149fdc6 100644 --- a/machine-docs/JOURNAL-2.md +++ b/machine-docs/JOURNAL-2.md @@ -58,3 +58,65 @@ $REF). Per HC2, repo-local PR-authored code runs only for recipes on Will now seed DECISIONS, then begin Q0.1 (vendor helpers into runner/harness/) — keeping the custom-html overlay working as the reference recipe. The /loop will self-pace. + +## 2026-05-28 — Q0 + Q1.1 landed; Q0 gate CLAIMED + +Worked through Q0.1, Q0.2, Q0.3, Q1.1 in one stretch since they're tightly coupled: + +**Q0.1** — `runner/harness/http.py` is the canonical Phase-2 recipe-test HTTP API. Mirrors +`recipe-maintainer/utils/tests/helpers.py` shape (same function names, same return shapes) so +parity ports read 1:1, but self-contained (cc-ci runtime does NOT import recipe-maintainer per +DECISIONS Phase 2). Existing `lifecycle.http_get`/`http_fetch`/`http_body` stay — they're for +infra-level checks like Traefik-404 detection. `harness.http` is for recipe tests' API calls. SSL +context is `CERT_NONE` because per-run domains use the wildcard cert; the real-cert verification +happens in `generic.served_cert` once per run via the install tier. + +**Q0.2** — discovery now recurses into `functional/` + `playwright/` subdirs. Surgically small change +to `custom_tests`; doesn't disturb the lifecycle-tier discovery (overlays still live at top-level). +Two new unit tests prove it (recursion works + HC2 gate still applies to subdirs). Pre-existing 8 +discovery unit tests still pass. + +**Q0.3 / Q1.1** — custom-html as the reference recipe: +- `PARITY.md` mapping table: 1 parity row (health_check) + 2 recipe-specific rows + (content_roundtrip + content_type_header) + a backup-integrity reference + a playwright reference. +- `functional/test_health_check.py` — parity port with `SOURCE: recipe-info/custom-html/tests/health_check.py` comment for audit. +- `functional/test_content_roundtrip.py` — NEW: write a `uuid.uuid4()` marker into nginx's + `/usr/share/nginx/html` volume, fetch over HTTPS, assert exact-byte match. Non-vacuous: a stale page + or misrouted backend can't return our random content. +- `functional/test_content_type_header.py` — NEW: write `.html` + `.txt` files with same body + ("hello"), HEAD each, assert `Content-Type: text/html` and `text/plain`. Caught the case where nginx + MIME map breaks even when 200 still works. +- `playwright/test_browser_smoke.py` — P6: Chromium renders HTML, no console errors. + +**E2E cold-verifiable evidence on cc-ci** (log `/root/ccci-q0-customhtml-full.log`): +``` +RECIPE=custom-html cc-ci-run runner/run_recipe_ci.py +===== TIER: install (generic=run, overlay=cc-ci:tests/custom-html/test_install.py) ===== + ... generic + overlay both PASS +===== TIER: upgrade ===== + upgrade→PR-head: head_ref=8a026066 chaos-version=8a026066 version=1.10.0+1.28.0→1.11.0+1.29.0 + ... generic + overlay both PASS (data marker "upgrade-survives" survived chaos redeploy) +===== TIER: backup ===== + ... generic + overlay both PASS +===== TIER: restore ===== + ... generic + overlay both PASS (volume restored to "original") +===== TIER: custom ===== + ... 4 PASS (parity health_check, content_roundtrip, content_type_header, browser_smoke) +===== RUN SUMMARY ===== +deploy-count = 1 (expect 1) + install : pass upgrade : pass backup : pass restore : pass custom : pass +``` + +That's the full Phase-2 pattern proven on the reference recipe: +- additive generic+overlay across 4 lifecycle ops (HC3), +- HC1 PR-head deploy proof via chaos-version label match, +- recipe-aware backup data-integrity (marker survives backup/restore cycle), +- 2 NEW recipe-specific functional tests beyond parity (P3 floor met), +- Playwright UI flow (P6), +- deploy-once + clean teardown. + +**Q0.4 (dep resolver) deferred to Q2**: no Q1 recipe (custom-html + n8n) has deps, and the resolver +shape will be much clearer once we have keycloak+authentik to deploy as deps. Logged in BACKLOG-2. + +**Q0 gate now CLAIMED.** Working in parallel on Q1.2 (n8n) while the Adversary cold-verifies. + diff --git a/machine-docs/STATUS-2.md b/machine-docs/STATUS-2.md index a30c423..d02c670 100644 --- a/machine-docs/STATUS-2.md +++ b/machine-docs/STATUS-2.md @@ -49,11 +49,51 @@ tree must carry: - **Q5** — Completeness + docs; flip `## DONE`. ## In flight -**Q0 — Harness additions.** Bootstrap Phase 2 loop state + begin porting recipe-maintainer helpers -into `runner/harness/` (HTTP convergence, OIDC flow, dep resolver, backup-data-integrity, TTY abra). +**Q1 — Pattern proof.** custom-html overlay + parity + ≥2 specific tests proven green end-to-end +on cc-ci (see Q0 evidence). Next: n8n (Q1.2) — port `recipe-info/n8n/tests/health_check.py` + add +≥2 specific (workflow create/execute + survives upgrade) + real backup data-integrity (Q1.3). ## Gate -(none yet — Q0 has not been claimed) +**Gate: Q0 — CLAIMED, awaiting Adversary @2026-05-28** (commits `0d0fc6c` Q0.1/Q0.2 + Q0.3/Q1.1 in +HEAD). Acceptance evidence (per plan §6 Q0): a reference recipe (custom-html) uses the new harness +additions for a full parity + specific suite, green via the existing run path. + +**Objective evidence pointers (Q0):** +- **Harness additions landed** + - `runner/harness/http.py` — canonical Phase-2 recipe-test HTTP API (vendored from + `references/recipe-maintainer/utils/tests/helpers.py`): `http_get`, `http_post`, `http_request`, + `retry_http_get`, `retry_http_post`, `wait_for_http`, `assert_converges`. JSON + form bodies, + transport-failure → status=0. + - `runner/harness/discovery.custom_tests` recurses into `tests//functional/` and + `tests//playwright/` (Phase 2 §4.1 layout) while excluding lifecycle `test_.py` + names; HC2 repo-local gate continues to apply. + - TTY abra wrapper already present in `runner/harness/abra.py::_run_pty` (Phase 1d) — reused. +- **Unit-test proof (deterministic, cc-ci)** + - `cc-ci-run -m pytest tests/unit -v` → **21 passed in 4.93s**: + - 8× pre-existing `tests/unit/test_discovery.py` (overlay + HC2 gate, regressed). + - 2× new `tests/unit/test_discovery_phase2.py` (functional/+playwright/ recursion + HC2 + gate still applies to subdirs). + - 11× new `tests/unit/test_http.py` (in-process http.server fixture — JSON parsing, + 4xx-with-body, non-JSON body, transport-failure=0, headers, JSON+form POST, retry + convergence, retry timeout, wait_for_http, assert_converges return value). +- **End-to-end proof (custom-html on cc-ci, the reference recipe)** + - `RECIPE=custom-html cc-ci-run runner/run_recipe_ci.py` (log `/root/ccci-q0-customhtml-full.log`): + - install/upgrade/backup/restore/custom **all PASS**, deploy-count=1. + - HC1 PR-head proof: `head_ref=8a026066 == chaos-version=8a026066`, version `1.10.0→1.11.0`. + - 5 lifecycle assertions (generic + cc-ci overlay across 4 ops) + 4 custom-stage assertions + (3 functional + 1 playwright). Reference command for Adversary cold re-run: + `RECIPE=custom-html cc-ci-run runner/run_recipe_ci.py`. +- **Per-recipe contract artifact landed** + - `tests/custom-html/PARITY.md` — parity row for `health_check.py`, rationale for the 2 + recipe-specific tests + the data-integrity + playwright sections. + - `tests/custom-html/functional/{test_health_check.py,test_content_roundtrip.py,test_content_type_header.py}` — parity port + 2 NEW recipe-specific tests; each parity file carries the + `SOURCE: recipe-info/custom-html/tests/` comment for audit. + - `tests/custom-html/playwright/test_browser_smoke.py` — Phase-2 P6 home. + +**Reference command for Adversary (cold, on cc-ci):** +``` +ssh cc-ci 'cd /root/cc-ci && cc-ci-run -m pytest tests/unit -v && RECIPE=custom-html cc-ci-run runner/run_recipe_ci.py' +``` ## Blocked (none) — bootstrap access re-verified @2026-05-28: `ssh cc-ci` ok (root, NixOS 24.11), Gitea API