status(2): Q0 RE-CLAIMED — F2-1 fix verified cold (21/21 unit PASS)

Per Adversary cold (REVIEW-2 "Q0 FAIL"), F2-1 mechanical regression: the Phase-1e HC2 unit test
asserted custom_tests('custom-html', rl) == [] when the real custom-html dir had no functional/
tests. Phase-2 added 4 legit functional/playwright files there, so the assertion no longer holds.
Behavior is correct; the test fixture was brittle.

Fix landed commit 5741e88: switch the assertion to a synthetic recipe + monkeypatch cc_ci_dir
(same pattern as the Phase-2 sibling test_discovery_phase2.py). Cold re-run: 21/21 PASS.

F2-2 (Q0 scope observation): OIDC-flow + dep resolver primitives deferred to Q2/Q3 when consuming
recipes land; BACKLOG-2 Q0.4 explicitly tracks this — acknowledged in STATUS-2 gate text.

Q0 RE-CLAIMED, awaiting Adversary re-verify.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-28 06:33:41 +01:00
parent 5741e8838f
commit 0b834e90f2
3 changed files with 46 additions and 8 deletions

View File

@ -19,9 +19,10 @@ Phase plan: `/srv/cc-ci/cc-ci-plan/plan-phase2-recipe-tests.md`
`requires`/`test_requires`, deploy deps before the recipe under test, tear down with it). Mind
`MAX_TESTS`/node budget; sequence heavy ones. **Deferred to Q2** (needed once SSO providers come
online; no Phase-2 recipe in Q1 needs deps). Tracked in BACKLOG.
- [x] **Q0.5****CLAIMED @2026-05-28.** Custom-html reference recipe runs the full parity + ≥2
specific + playwright suite green on cc-ci via the existing run path; deploy-count=1;
DECISIONS.md Phase-2 section in place. Awaiting Adversary cold-verify gate PASS.
- [x] **Q0.5****RE-CLAIMED @2026-05-28** (commit `5741e88` adds F2-1 fix to original Q0).
Custom-html reference recipe runs the full parity + ≥2 specific + playwright suite green on
cc-ci; deploy-count=1; DECISIONS.md Phase-2 section in place. F2-1 closed by Builder; 21/21
unit tests PASS cold. Awaiting Adversary cold re-verify.
### Q1 — Pattern proof (custom-html + n8n)
- [ ] **Q1.1** — custom-html: ≥2 NEW recipe-specific functional tests (beyond parity

View File

@ -120,3 +120,35 @@ shape will be much clearer once we have keycloak+authentik to deploy as deps. Lo
**Q0 gate now CLAIMED.** Working in parallel on Q1.2 (n8n) while the Adversary cold-verifies.
## 2026-05-28 — F2-1 fix: synthetic-recipe fixture (Adversary FAIL on Q0)
The Adversary FAILed Q0 cold on F2-1: `tests/unit/test_discovery.py::test_custom_tests_repo_local_gated` (Phase-1e HC2 test) used the real recipe name `"custom-html"` and asserted
`custom_tests("custom-html", repo_local) == []`. Phase-2 commit `bec9265` added 4 legit non-lifecycle
tests under `tests/custom-html/{functional,playwright}/`, which `custom_tests()` now correctly
returns — so the `== []` assertion no longer holds. Behavior is right; the fixture was brittle.
My "21 passed" evidence was real on the Builder clone — but I had synced the new tests to cc-ci
**before** syncing the new custom-html functional/ tests, so at that moment the assertion still held.
The Adversary's cold re-run from origin/main pulled the full state and correctly caught the regression.
**Fix (commit `5741e88`):** switch to synthetic recipe + monkeypatch `discovery.cc_ci_dir` — same
pattern already used in the Phase-2 sibling `tests/unit/test_discovery_phase2.py`. 5-line change,
no behavior change. Cold-verifiable: `cc-ci-run -m pytest tests/unit -v` → 21/21 PASS.
F2-2 (scope observation) — the Adversary flagged that Q0.4 (dep resolver) and OIDC-flow primitive
are not yet implemented; explicitly deferred to Q2/Q3 in BACKLOG-2. Acknowledged in STATUS-2 gate
text.
**Lesson:** when adding new content to an existing recipe directory, scan the unit tests for any
that assume that directory is empty/lifecycle-only. The synthetic-recipe + monkeypatch pattern is
the right shape for all such unit tests; we should prefer it across the board.
**n8n probe ran in the background to validate endpoint shapes for Q1.2:**
- `/` → 200 text/html (the SPA)
- `/healthz` → 200 `{"status":"ok"}` (already used by install overlay)
- `/types/nodes.json` → 200 but size=31 bytes, not JSON (probably SPA fallback). REJECT this idea.
- Probe terminated before reaching `/rest/settings` / `/rest/login` (the JSON parse on
`/types/nodes.json` raised). Re-running probe now without the JSON gate.
Q0 re-claimed; awaiting Adversary re-verify. Continuing on Q1.2 (n8n) in parallel.

View File

@ -54,9 +54,13 @@ on cc-ci (see Q0 evidence). Next: n8n (Q1.2) — port `recipe-info/n8n/tests/hea
≥2 specific (workflow create/execute + survives upgrade) + real backup data-integrity (Q1.3).
## Gate
**Gate: Q0 — CLAIMED, awaiting Adversary @2026-05-28** (commits `0d0fc6c` Q0.1/Q0.2 + Q0.3/Q1.1 in
HEAD). Acceptance evidence (per plan §6 Q0): a reference recipe (custom-html) uses the new harness
additions for a full parity + specific suite, green via the existing run path.
**Gate: Q0 — RE-CLAIMED, awaiting Adversary @2026-05-28** (commit `5741e88` — F2-1 fix landed on
top of the original Q0 changeset). Acceptance evidence (per plan §6 Q0): a reference recipe
(custom-html) uses the new harness additions for a full parity + specific suite, green via the
existing run path. F2-1 (test_custom_tests_repo_local_gated stale assertion) closed by Builder; cold
re-run on cc-ci → **21/21 PASS** including the previously-failing test. F2-2 (scope observation:
OIDC-flow + dep resolver not in Q0) acknowledged — those primitives implement when Q2/Q3 consume
them; BACKLOG-2 Q0.4 remains open and explicitly deferred.
**Objective evidence pointers (Q0):**
- **Harness additions landed**
@ -68,8 +72,9 @@ additions for a full parity + specific suite, green via the existing run path.
`tests/<recipe>/playwright/` (Phase 2 §4.1 layout) while excluding lifecycle `test_<op>.py`
names; HC2 repo-local gate continues to apply.
- TTY abra wrapper already present in `runner/harness/abra.py::_run_pty` (Phase 1d) — reused.
- **Unit-test proof (deterministic, cc-ci)**
- `cc-ci-run -m pytest tests/unit -v`**21 passed in 4.93s**:
- **Unit-test proof (deterministic, cc-ci; post-F2-1 fix commit `5741e88`)**
- `cc-ci-run -m pytest tests/unit -v`**21 passed in 5.38s** (the previously-failing
`test_custom_tests_repo_local_gated` now passes; synthetic-recipe + monkeypatch fixture):
- 8× pre-existing `tests/unit/test_discovery.py` (overlay + HC2 gate, regressed).
- 2× new `tests/unit/test_discovery_phase2.py` (functional/+playwright/ recursion + HC2
gate still applies to subdirs).