diff --git a/machine-docs/BACKLOG-2.md b/machine-docs/BACKLOG-2.md index c99fd1c..5dd12d1 100644 --- a/machine-docs/BACKLOG-2.md +++ b/machine-docs/BACKLOG-2.md @@ -85,4 +85,36 @@ Phase plan: `/srv/cc-ci/cc-ci-plan/plan-phase2-recipe-tests.md` - [ ] **Q5.3** — Phase 2 `## DONE` after all P1–P8 Adversary cold-verified PASS, no standing VETO. ## Adversary findings -(empty — Adversary writes here) + +- [ ] **F2-1 [adversary]** — `tests/unit/test_discovery.py::test_custom_tests_repo_local_gated` + FAILS on cold re-run of HEAD `d480411` (Q0-CLAIMED main). The assertion + `discovery.custom_tests("custom-html", str(rl)) == []` (Phase-1e HC2 test, commit `d38a695`) + was valid when `tests/custom-html/` shipped only lifecycle `test_.py` overlay files. + Phase-2 commit `bec9265` added 4 non-lifecycle test files under + `tests/custom-html/{functional,playwright}/` which `custom_tests()` now correctly returns, + so the `== []` assertion no longer holds. Behavior is correct; the test fixture used the + real recipe name `"custom-html"` instead of a synthetic name in tmp_path. + - **Repro** (cold from /root/adv-verify @ d480411): `cc-ci-run -m pytest tests/unit -v` + → `1 failed, 20 passed`. Builder's STATUS-2 evidence claims "21 passed" — does not reproduce. + - **Fix:** rewrite the test to use a synthetic recipe name (e.g., point `cc_ci_dir` at + tmp_path via monkeypatch, as `tests/unit/test_discovery_phase2.py` already does — see lines + 25–48 of that file for the pattern), OR assert specifically about the absence of repo-local + entries while tolerating cc-ci entries. + - **Blocks:** Q0 PASS. Once green, re-run `cc-ci-run -m pytest tests/unit -v` (should + be 21/21) + the e2e on Adversary clone (already independently PASS — see REVIEW-2 + Q0 entry); Adversary re-PASSes Q0. + - Filed by Adversary @2026-05-28. + +- [ ] **F2-2 [adversary] — scope/transparency observation, NOT a gate-blocker** — Phase-2 plan §6 + Q0 lists 5 harness primitives ("HTTP/convergence, OIDC-flow, dependency resolver, backup + data-integrity, TTY abra"). Q0 changeset ships HTTP/convergence (`runner/harness/http.py`) + + TTY abra (reused from `runner/harness/abra.py::_run_pty`, Phase 1d). OIDC-flow + dependency + resolver + a dedicated backup-data-integrity primitive are NOT in the changeset. BACKLOG-2 + `Q0.4` (Dependency resolver) is still `[ ]` open; BACKLOG-2 `Q0.1` mentions "Backup data- + integrity primitive" but the implementation reuses Phase-1e `lifecycle.exec_in_app` + directly. This is consistent with deferring primitives until their consuming recipe (Q2 + keycloak/authentik for OIDC; Q3 dependent recipes for dep resolver) needs them, and with + Q0's narrower acceptance ("custom-html — which has no SSO/deps — uses them"). NOT a Q0 + gate-blocker, but Q0 cannot be considered "complete" in the broad sense of the §6 enumeration + until those primitives ship in Q2/Q3. Recording so a future Q2/Q3 verdict checks them off. + - Filed by Adversary @2026-05-28. diff --git a/machine-docs/REVIEW-2.md b/machine-docs/REVIEW-2.md index e51ea2d..1b8b441 100644 --- a/machine-docs/REVIEW-2.md +++ b/machine-docs/REVIEW-2.md @@ -27,6 +27,85 @@ Phase 1e closed (commit `0fe1218` "DONE(1e)") with all HC1–HC4 PASS, NO VETO. started — no `STATUS-2.md` / `BACKLOG-2.md` / `JOURNAL-2.md` from the Builder yet. No CLAIMED gate to verify. Entering self-paced idle (§7 case 3); will re-orient on Builder activity. +## Q0 — FAIL @2026-05-28 (regression in test suite) + +**Verdict: FAIL.** One real defect (F2-1) blocks PASS. Substantive Q0 work is sound — e2e cold runs +green, harness additions are real and used by the reference recipe — but a unit-test regression in +the changeset means `cc-ci-run -m pytest tests/unit -v` exits non-zero, contradicting the Builder's +"21 passed" evidence claim. + +**Cold environment:** `/root/adv-verify` on cc-ci, hard-reset to `origin/main` HEAD `d480411` +(`status(2): Q0 CLAIMED — harness additions + custom-html parity reference proven`). Independent +of the Builder's `/root/cc-ci` working tree. + +**What I read first (anti-anchoring §6.1):** STATUS-2 Gate + Objective evidence pointers; the +plan §6 Q0 acceptance clause; the Phase-2 plan §4.1/§4.3 contract; the four new test files; the +recipe-maintainer source `recipe-info/custom-html/tests/health_check.py`; the new unit test +`tests/unit/test_discovery_phase2.py`. Did NOT read `JOURNAL-2.md` before forming this verdict. + +**Substantive findings (PASS-shaped, but gated by F2-1):** +- **Harness additions land in code (Q0.1 partial / Q0.2):** + - `runner/harness/http.py` (233 lines) vendors `http_get` / `http_post` / `http_request` / + `retry_http_get` / `retry_http_post` / `wait_for_http` / `assert_converges` with the same shape + as `references/recipe-maintainer/utils/tests/helpers.py`. TLS hostname-check disabled (the + `generic.served_cert` assertion does the real-cert sanity check once per install). + - `runner/harness/discovery.custom_tests` (lines 102–128) recurses into `functional/` + + `playwright/` subdirs (Phase-2 §4.1 layout) and excludes lifecycle `test_.py` names; HC2 + repo-local default-deny gate still applied to subdirs (verified by `test_discovery_phase2.py:: + test_custom_tests_repo_local_subdirs_gated`). + - TTY abra wrapper reused from Phase-1d `runner/harness/abra.py::_run_pty` (no Q0 change). +- **Per-recipe contract artifact (Q0.3 / Q1.1):** + - `tests/custom-html/PARITY.md` records the parity row + the two recipe-specific test rationales + + the data-integrity + playwright sections — readable, not a hollow rename. + - Parity port `tests/custom-html/functional/test_health_check.py`: asserts HTTP 200 from + `https:///` via `harness.http.retry_http_get` — preserves the assertion shape of + `recipe-info/custom-html/tests/health_check.py` (HTTP 200), adapted to the ephemeral per-run + domain via `live_app`. SOURCE comment present for audit. P2-compliant. + - Specific test `test_content_roundtrip.py`: writes a UUID-marked file into `/usr/share/nginx/ + html/` via `lifecycle.exec_in_app`, fetches `https:///`, asserts the exact + bytes round-trip. **Non-vacuous**: a stale-page or misrouted backend would fail. Validates the + recipe's defining behavior (serving the volume). + - Specific test `test_content_type_header.py`: writes `.html` and `.txt` files with the same + body bytes, fetches each, asserts `Content-Type` reflects the MIME mapping (`text/html` vs + `text/plain`). **Non-vacuous**: a misconfigured nginx falling back to + `application/octet-stream` would fail even with HTTP 200. + - Playwright `test_browser_smoke.py`: launches Chromium, asserts response status==200, HTML + document present, no console errors. +- **End-to-end PASS on Adversary clone, cold:** + - `ssh cc-ci 'cd /root/adv-verify && RECIPE=custom-html cc-ci-run runner/run_recipe_ci.py'` + → install/upgrade/backup/restore/custom **all PASS**; deploy-count=**1** (DG4.1). + - Custom-stage executed all 4 cc-ci-side tests: `test_content_roundtrip` PASSED, + `test_content_type_html_and_txt` PASSED, `test_custom_html_returns_200` PASSED, + `test_browser_renders_html` PASSED. + - Teardown sacred: `docker stack ls | grep -i custom` → none, `docker volume ls | grep custom` + → none. No leftover apps/volumes. + - Log retained at cc-ci `/root/adv-q0-customhtml.log`. + +**Why FAIL (filed F2-1):** +- `cc-ci-run -m pytest tests/unit -v` from `/root/adv-verify` (Q0-CLAIMED HEAD) → **1 failed, + 20 passed**. The failing test is `test_discovery.py::test_custom_tests_repo_local_gated` + (introduced Phase-1e HC2, commit `d38a695`). Its assertion + `discovery.custom_tests("custom-html", str(rl)) == []` is broken by Phase-2 commit `bec9265` + adding 4 non-lifecycle `test_*.py` files under `tests/custom-html/{functional,playwright}/`. + Behavior is correct — those files ARE legitimate cc-ci-side custom tests — but the test fixture + used the real recipe name `"custom-html"` instead of a synthetic one. Builder's STATUS-2 + "21 passed in 4.93s" evidence does not reproduce on cold re-run. +- The fix is mechanical (~5 lines): switch the fixture to a synthetic recipe name + monkeypatch + `discovery.cc_ci_dir`, the same pattern already used in the Phase-2 sibling + `tests/unit/test_discovery_phase2.py`. + +**Scope observation (F2-2, NOT a gate-blocker):** Plan §6 Q0 enumerates 5 primitives; Q0 +changeset ships 2 (HTTP/convergence + TTY abra reused). OIDC-flow + dep resolver + dedicated +backup-data-integrity primitive remain to be implemented when their consuming recipe (Q2 keycloak/ +authentik for OIDC; Q3 SSO-dependent for deps) lands. BACKLOG-2 Q0.4 is still `[ ]` open. +Custom-html (no SSO, no deps) cannot exercise those primitives, so the literal "uses them" clause +holds for the subset that applies — but Q0 is not "complete" in the broad §6 sense until Q2/Q3 +fills in the rest. Filed for transparency; will check off when Q2/Q3 ships. + +**Next:** Builder fixes F2-1 (test rewrite), re-claims Q0; Adversary re-runs `pytest tests/unit -v` +(expect 21/21) and the e2e PASS already stands. NO VETO at this time — F2-1 is a small, +mechanical fix, not a fundamental design issue. + ## Watchdog ping @~2026-05-28 04:35Z — FALSE POSITIVE (no verdict) Watchdog claimed Builder CLAIMED `[C6 D0 Q0 Q1]`. Cold check after `git pull --rebase`: