cc-ci/machine-docs/REVIEW-1e.md

REVIEW-1e — Adversary verdicts (Phase 1e: generic-harness corrections)

Adversary-owned, append-only. Phase plan: /srv/cc-ci/cc-ci-plan/plan-phase1e-harness-corrections.md. Definition of Done = HC1–HC4 each cold-verified PASS here (handshake per plan.md §6.1).

Definition-of-Done tracker

• HC1 — Upgrade tier upgrades to PR head (prev published → PR-head via abra app deploy --chaos), not a published tag; moved-assertion adapted; DG4.1 deploy-count guard reconciled.
• HC2 — Repo-local (PR-authored) test_*.py / install_steps.sh NOT executed unless recipe is on the cc-ci approval allowlist (default-deny). PASS @2026-05-28 (E0, commit c7ae296).
• HC3 — Generic runs by default alongside an overlay (additive); skipped only via explicit opt-out; op runs once.
• HC4 — No regression: D1–D10 / DG1–DG8 re-verified cold; deploy-once (DG4.1) holds; teardown sacred; three new behaviors demonstrated.

Maps to Builder milestones: E0=HC2, E1=HC3, E2=HC1, E3=HC4+docs.

Cold-start access (re-verified each phase)

• @2026-05-28 — ssh cc-ci OK (NixOS 24.11), dashboard HTTP 200 via SOCKS proxy 127.0.0.1:1055. Proxy/SSH path healthy.

Verdicts

E0 / HC2 — repo-local trust gate (default-deny) — PASS @2026-05-28

Builder claim (STATUS-1e, commit c7ae296 / feat d38a695): repo-local (PR-authored) test_*.py/install_steps.sh/ops.py consulted only for recipes on tests/repo-local-approved.txt (empty ⇒ deny); centralized _gated() in discovery.py; 8 unit tests pass.

Cold verification (own clone HEAD=c7ae296, shipped to cc-ci, run via cc-ci-run):

1. Unit suite, independent run: cd /tmp/adv-1e && cc-ci-run -m pytest tests/unit -v → 8 passed in 0.06s (incl. repo-local-ignored-when-unapproved / wins-when-approved for overlay+custom+install_steps+pre_op, and default-allowlist-is-empty).
2. My own break-it probe (hc2_probe.py, planted a HOSTILE repo-local install_steps.sh rm -rf / + ops.py os.system('id') + test_install.py):
   • real checked-in allowlist → approved_recipes() == set() (default-deny).
   • real-default → approved=False, overlay falls back to cc-ci, install_steps=None, pre_op=None (hostile repo-local code NOT selected).
   • lone * → DENY (not a wildcard, as the file header promises).
   • only-comment / whitespace lines → DENY.
   • approving a different recipe (hedgedoc) → custom-html still DENY (no leak).
   • custom-html listed → approved=True, overlay/install_steps/pre_op all flip to repo-local.
3. No bypass: every execution path in runner/run_recipe_ci.py routes through gated discovery.* (resolve_op→resolve_overlay_op, custom_tests, install_steps→lifecycle hook). snapshot_recipe_tests reads the repo-local dir ungated but only copies it (discover), never executes — matches the plan's "discovered-but-NOT-executed". pre_op_hook not yet wired into the orchestrator (E1/HC3 work); its discovery fn is already gated.

Verdict: PASS — default-secure, centralized gate, flips only on explicit per-recipe approval; hostile repo-local code provably not executed under the shipped default. No finding. Note (not a defect): orchestrator still uses single-file override resolve_op (1d semantics); the additive generic floor (HC3) is E1 in-flight — will re-check the gate survives the HC3 refactor.

E1 / HC3 — additive generic + op/assertion split — FAIL (PASS WITHHELD) @2026-05-28

Builder claim (STATUS-1e gate, commit b7e6cbd): generic runs additively alongside overlays; orchestrator owns each op (once); opt-out via CCCI_SKIP_GENERIC[_<OP>]/recipe_meta.SKIP_GENERIC; deploy-count stays 1; two e2e (default + opt-out) "clean."

Cold verification (own clone HEAD=b7e6cbd shipped to cc-ci /tmp/adv-1e, run via cc-ci-run):

• Structure (PASS): read the refactor — run_lifecycle_tier performs the op ONCE (_perform_op→generic.perform_{upgrade,backup,restore}, none call deploy_app), then runs generic (unless _skip_generic) + overlay as separate pytests vs the shared post-op state. Generic+overlay test files are assertion-only; seeding moved to ops.py pre_<op>. assert_upgraded keeps the non-vacuous move check (F1d-2). _record_deploy() lives only in deploy_app.
• Default e2e (custom-html, all stages): EVERY tier ran BOTH assert (generic) AND assert (cc-ci); pre_upgrade/backup/restore seeds fired; deploy-count=1; install/upgrade/backup/ restore all PASS; custom=skip; clean teardown (no leftover stack/volume). ✓ additive confirmed.
• Opt-out e2e (CCCI_SKIP_GENERIC=1): generic skipped on every tier (0 _generic/ files ran), overlay-only, deploy-count=1 ✓ — but backup=FAIL: test_backup_captures_state → AssertionError: '' == 'original'. Same code/recipe; only diff is the opt-out flag.

Verdict: FAIL — opt-out is not behavior-neutral. Opting out of the generic removes an accidental ~1s timing buffer (the generic pytest spawn) and surfaces a real race: the backup/restore overlays read the marker via exec_in_app immediately after a container-cycling op with no readiness/retry, and exec_in_app silently returns empty stdout on a failed docker exec (returncode ignored). A healthy recipe can thus be reported RED under opt-out. Filed F1e-1 [adversary] (BACKLOG-1e) with root cause

• repro + fix direction (check exec returncode + bounded readiness retry; do NOT weaken the assertion). Isolated (no-concurrency) reproduction in flight to rule out the parallel-Builder-run confound — which would itself be a concurrency-collision finding. HC3 PASS withheld until F1e-1 is fixed + re-verified cold under opt-out.