review(2): F2-11 CLOSED — deploy-free cold proof (35 unit + real conftest skip-report stitched to predicate); consume inbox
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@ -115,8 +115,36 @@ Phase plan: `/srv/cc-ci/cc-ci-plan/plan-phase2-recipe-tests.md`
|
||||
|
||||
## Adversary findings
|
||||
|
||||
- [ ] **F2-11 [adversary] — SSO-dep "deps-not-ready" SKIP yields a GREEN `!testme` while the
|
||||
core OIDC test never ran (gate-integrity / P7, medium)** — Filed by Adversary @2026-05-28
|
||||
- [x] **F2-11 [adversary] — CLOSED @2026-05-28** by Builder commit `5b34496`. The deps-not-ready
|
||||
SKIP no longer yields a GREEN run; generic-tier failure-isolation is preserved (only the green
|
||||
SIGNAL is corrected). The fix: `conftest.pytest_collection_modifyitems` counts skipped
|
||||
`requires_deps` tests and appends the count to `$CCCI_DEPS_SKIP_REPORT`; `run_recipe_ci`
|
||||
sums it (`run_recipe_ci.py:582-585`), surfaces `(N requires_deps SKIPPED … SSO UNVERIFIED)`
|
||||
in the RUN SUMMARY, and the pure predicate `sso_dep_unverified(declared, deps_ready, skipped)`
|
||||
(`:48`) flips `overall=1` (`:633`) when a DEPS-declaring recipe skipped ≥1 SSO test.
|
||||
**Adversary cold re-verify @2026-05-28 on `/root/adv-verify` HEAD `0d6cd05` (deploy-free,
|
||||
rate-limit-independent):**
|
||||
- `cc-ci-run -m pytest tests/unit -q` → **35 passed** (28 prior + 7 new `test_f211_sso_skip.py`;
|
||||
read the bodies — non-vacuous: predicate true + 3 false cases, conftest skip/record/append/
|
||||
no-op with fakes).
|
||||
- **Real signal proof:** the actual `tests/lasuite-docs/functional/test_oidc_with_keycloak.py`
|
||||
(lasuite-docs declares `DEPS=["keycloak"]`) run with `CCCI_DEPS_READY=0` →
|
||||
`1 skipped`, **pytest-exit=0** (the original hazard — a skip-only file still exits 0) BUT
|
||||
`$CCCI_DEPS_SKIP_REPORT` content == `1`.
|
||||
- **Stitched to the real orchestrator predicate:** `sso_dep_unverified(["keycloak"], False, 1)
|
||||
= True` → `overall=1` (RED). Negatives correct: `deps_ready=True → False`, `no-deps → False`.
|
||||
- Runtime wiring verified by code-read: `main()` sets `CCCI_DEPS_SKIP_REPORT` (`:445`) before
|
||||
the custom tier; `_tier_env` returns `dict(os.environ, …)` so the pytest subprocess inherits
|
||||
`CCCI_DEPS_READY` + the report path; orchestrator reads the same `skipfile`.
|
||||
- **Residual (non-blocking):** the Builder honestly deferred the full live-deploy e2e (forced
|
||||
`setup_custom_tests` failure on a real deployed recipe → observe `overall=1` end-to-end)
|
||||
behind the Docker Hub pull rate limit. The decision logic + conftest→orchestrator signal it
|
||||
would exercise are already proven above; I will confirm the live path on the next SSO-dep
|
||||
deploy once pulls flow (belt-and-suspenders, not a re-open condition).
|
||||
Original FAIL detail retained below for audit.
|
||||
|
||||
- [ ] ~~**F2-11 [adversary] — SSO-dep "deps-not-ready" SKIP yields a GREEN `!testme` while the
|
||||
core OIDC test never ran (gate-integrity / P7, medium)**~~ — Filed by Adversary @2026-05-28
|
||||
as an independent break-it probe during the git.autonomic.zone outage (no gate claimed).
|
||||
|
||||
**The hazard chain (cold-proven, end-to-end):**
|
||||
|
||||
Reference in New Issue
Block a user