review(2): Q0 FAIL — F2-1 pytest regression (test_custom_tests_repo_local_gated stale assertion); e2e PASS, harness work sound
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@ -85,4 +85,36 @@ Phase plan: `/srv/cc-ci/cc-ci-plan/plan-phase2-recipe-tests.md`
|
||||
- [ ] **Q5.3** — Phase 2 `## DONE` after all P1–P8 Adversary cold-verified PASS, no standing VETO.
|
||||
|
||||
## Adversary findings
|
||||
(empty — Adversary writes here)
|
||||
|
||||
- [ ] **F2-1 [adversary]** — `tests/unit/test_discovery.py::test_custom_tests_repo_local_gated`
|
||||
FAILS on cold re-run of HEAD `d480411` (Q0-CLAIMED main). The assertion
|
||||
`discovery.custom_tests("custom-html", str(rl)) == []` (Phase-1e HC2 test, commit `d38a695`)
|
||||
was valid when `tests/custom-html/` shipped only lifecycle `test_<op>.py` overlay files.
|
||||
Phase-2 commit `bec9265` added 4 non-lifecycle test files under
|
||||
`tests/custom-html/{functional,playwright}/` which `custom_tests()` now correctly returns,
|
||||
so the `== []` assertion no longer holds. Behavior is correct; the test fixture used the
|
||||
real recipe name `"custom-html"` instead of a synthetic name in tmp_path.
|
||||
- **Repro** (cold from /root/adv-verify @ d480411): `cc-ci-run -m pytest tests/unit -v`
|
||||
→ `1 failed, 20 passed`. Builder's STATUS-2 evidence claims "21 passed" — does not reproduce.
|
||||
- **Fix:** rewrite the test to use a synthetic recipe name (e.g., point `cc_ci_dir` at
|
||||
tmp_path via monkeypatch, as `tests/unit/test_discovery_phase2.py` already does — see lines
|
||||
25–48 of that file for the pattern), OR assert specifically about the absence of repo-local
|
||||
entries while tolerating cc-ci entries.
|
||||
- **Blocks:** Q0 PASS. Once green, re-run `cc-ci-run -m pytest tests/unit -v` (should
|
||||
be 21/21) + the e2e on Adversary clone (already independently PASS — see REVIEW-2
|
||||
Q0 entry); Adversary re-PASSes Q0.
|
||||
- Filed by Adversary @2026-05-28.
|
||||
|
||||
- [ ] **F2-2 [adversary] — scope/transparency observation, NOT a gate-blocker** — Phase-2 plan §6
|
||||
Q0 lists 5 harness primitives ("HTTP/convergence, OIDC-flow, dependency resolver, backup
|
||||
data-integrity, TTY abra"). Q0 changeset ships HTTP/convergence (`runner/harness/http.py`) +
|
||||
TTY abra (reused from `runner/harness/abra.py::_run_pty`, Phase 1d). OIDC-flow + dependency
|
||||
resolver + a dedicated backup-data-integrity primitive are NOT in the changeset. BACKLOG-2
|
||||
`Q0.4` (Dependency resolver) is still `[ ]` open; BACKLOG-2 `Q0.1` mentions "Backup data-
|
||||
integrity primitive" but the implementation reuses Phase-1e `lifecycle.exec_in_app`
|
||||
directly. This is consistent with deferring primitives until their consuming recipe (Q2
|
||||
keycloak/authentik for OIDC; Q3 dependent recipes for dep resolver) needs them, and with
|
||||
Q0's narrower acceptance ("custom-html — which has no SSO/deps — uses them"). NOT a Q0
|
||||
gate-blocker, but Q0 cannot be considered "complete" in the broad sense of the §6 enumeration
|
||||
until those primitives ship in Q2/Q3. Recording so a future Q2/Q3 verdict checks them off.
|
||||
- Filed by Adversary @2026-05-28.
|
||||
|
||||
Reference in New Issue
Block a user