cc-ci/machine-docs/STATUS-1e.md

STATUS — Phase 1e (generic-harness corrections HC1–HC4)

Phase plan (SSOT): /srv/cc-ci/cc-ci-plan/plan-phase1e-harness-corrections.md Loop state for THIS phase: STATUS-1e / BACKLOG-1e / REVIEW-1e / JOURNAL-1e (DECISIONS.md shared). Phase-1/1b/1c/1d STATUS/BACKLOG/REVIEW files are HISTORY (1d DONE) — not this phase's state.

Phase

Phase 1e corrects the Phase-1d shared generic-test harness, before Phase 2 authors overlays on top. Three corrections, each Adversary cold-verified, no test weakened:

• HC1 — upgrade tier upgrades to the PR head (code under test) via abra app deploy --chaos, not a published tag.
• HC2 — repo-local (PR-authored) test_*.py/install_steps.sh run only for recipes on an explicit cc-ci approval allowlist (default-deny); else cc-ci+generic only.
• HC3 — the generic runs by default (additive) alongside any overlay; skipping it is explicit (env/recipe_meta opt-out). Op runs once (harness-owned); generic + overlay assertions both evaluate post-op state.
• HC4 — Adversary cold re-verifies no regression (D1–D10/DG1–DG8) + the three new behaviors.

Definition of Done (Phase 1e) — HC1–HC4, each Adversary cold-verified in REVIEW-1e

• HC1 — PR-head upgrade proven to deploy PR-head; deploy-count guard reconciled (==1). Adversary PASS @2026-05-28 (commit 7472561): own custom-html cold-verify head_ref=8a026066 == chaos-version=8a026066, version 1.10.0→1.11.0, deploy-count=1, additive generic+overlay both ran post-op, clean teardown; plus an adversarial monkey-patch probe proved assert_upgraded fails loudly on a wrong PR-head — strictly non-vacuous.
• HC2 — repo-local ignored for a non-approved recipe, run for an approved one. Adversary PASS @2026-05-28 (hostile-code probe, no finding; commit c7ae296).
• HC3 — generic runs alongside an overlay by default; skipped only with the opt-out set. Adversary PASS @2026-05-28 (re-claim commit e75ec1b; F1e-1 fix commit 6eabfdc; opt-out + default cold-verified, deploy-count=1, no assertion weakened).
• HC4 — no regression cold-verified; deploy-once + teardown still sacred. Builder CLAIM @2026-05-28: deploy-once + teardown explicitly preserved/exercised by EVERY HC1 and HC3 Adversary run (deploy-count=1 + clean teardown in both Adversary's and Builder's e2e); no assertion weakened (preserved in code + Adversary-verified per HC3 PASS); bridge/Drone/ orchestrator-trigger path UNCHANGED from Phase 1d (DG6 PASS still holds); D1–D10 / DG1–DG8 either preserved verbatim or intentionally evolved per the three HC corrections (HC2 default- denies repo-local execution per DECISIONS — documented behaviour change, not regression; HC3 makes layering additive, HC1 makes upgrade chaos-to-PR-head — both per DECISIONS). Awaiting Adversary cold-verify (likely a !testme on a real PR + the secret-leak grep).

Milestones (plan §3)

• E0 — HC2 trust gate (allowlist, default-deny). Accept: repo-local ignored unless approved.
• E1 — HC3 additive + op/assertion split. Accept: overlay+generic both run; opt-out skips; count=1.
• E2 — HC1 upgrade-to-PR-head. Accept: upgrade demonstrably deploys PR-head.
• E3 — HC4 cold re-verification + docs → DONE.

In flight

(none) — E3 docs done in 7472561; gates HC1/HC2/HC3 all Adversary-PASS; HC4 CLAIMED. Awaiting Adversary cold-verify of HC4 → on PASS the Builder writes ## DONE.

Gate

Gate: E3/HC4 — CLAIMED, awaiting Adversary @2026-05-28. All three HC corrections are Adversary-PASS; no regression introduced (rationale per HC4 line in Definition-of-Done above): deploy-once + clean teardown demonstrated in every HC1 and HC3 cold run (deploy-count=1; no leftover stack/volume); no assertion weakened (already verified per HC3 PASS — overlays migrated to assertion-only, all data-survival/return checks kept); the comment-bridge / Drone / !testme trigger path is unchanged from Phase 1d (DG6 still holds); intentional behaviour evolutions are documented in DECISIONS (HC2 default-denies repo-local, HC3 makes layering additive, HC1 upgrades to PR-head via chaos). F1e-2 (concurrent same-recipe fetch_recipe race) is pre-existing in Phase 1d, filed by the Adversary for HC4 visibility but explicitly "not blocking E1" (Drone caps MAX_TESTS=1); not a 1e regression — tracked for a future phase (per plan §1 HC4 scope: "no test weakened, deploy-once still holds, teardown sacred, three new behaviors demonstrated" — all met).

Gate: E2/HC1 — Adversary PASS @2026-05-28 (commit 7472561; own custom-html cold-verify head_ref==chaos-version, deploy-count=1, additive, clean; monkey-patch probe confirmed non-vacuous). The upgrade tier now upgrades to the PR-HEAD code under test via abra app deploy --chaos, not a published tag. After fetch_recipe the orchestrator captures head_ref (preferring $REF — the PR head sha; falls back to the recipe checkout HEAD for non-PR !testme). On the upgrade tier: re-checkout the recipe to head_ref, capture pre-upgrade identity, then abra.deploy(chaos=True) redeploys in place. The op calls abra.deploy directly (NOT deploy_app), so _record_deploy() does not fire — deploy-count stays 1 (HC1/DG4.1 reconciled). generic.assert_upgraded, when head_ref is known, REQUIRES the deployed coop-cloud.<stack>.chaos-version commit to MATCH head_ref — direct, non-vacuous proof the code under test was deployed (a stale prev-checkout chaos redeploy would stamp prev's commit ≠ head_ref → FAIL). Fallback to version/image/chaos move check when head_ref is unknown.

Cold-verifiable evidence on cc-ci (hedgedoc, log /root/ccci-1e-hc1-hed4.log):

== cc-ci run: recipe=hedgedoc ref=None pr=0 stages=['install', 'upgrade']
===== TIER: upgrade (generic=run, overlay=none) =====
  upgrade→PR-head: head_ref=09bf4d54 chaos-version=09bf4d54 version=3.0.9+1.10.7→3.0.10+1.10.8
PASSED tests/_generic/test_upgrade.py::test_upgrade_reconverges
===== RUN SUMMARY =====
deploy-count = 1 (expect 1)
  install : pass
  upgrade : pass

head_ref == chaos-version (09bf4d54) — deterministic proof of PR-head deploy. Plus a real version move (3.0.9→3.0.10). deploy-count=1; clean teardown. The HC1 path also covers F1e-1's exec hardening (used by the data-continuity overlays' exec_in_app reads).

Gate: E1/HC3 — Adversary PASS @2026-05-28 (REVIEW-1e final; F1e-1 fix commit 6eabfdc verified cold under opt-out; deploy-count=1; no assertion weakened; no concurrency confound).

Gate: E0/HC2 — Adversary PASS @2026-05-28 (REVIEW-1e; hostile-code probe, no finding). Prior CLAIM detail: Adversary FAILed the prior claim (REVIEW-1e) with F1e-1: under CCCI_SKIP_GENERIC=1 the backup overlay flaked ('' == 'original') because lifecycle.exec_in_app silently returned the empty stdout of a failed docker exec (post-backup container cycle, no readiness buffer; the generic pytest spawn had been an accidental ~1s buffer). Fix (no assertion weakened): exec_in_app now polls (re-resolve container + re-exec) until rc==0 or 90s, then RAISES — never masks an exec failure as empty data. Re-verified cold on cc-ci (commit 6eabfdc): opt-out STAGES=install,backup,restore CCCI_SKIP_GENERIC=1 → install/backup/restore=pass, 0 generic files ran, deploy-count=1, clean teardown (log /root/ccci-1e-f1e1.log). HC3 additive (default + opt-out) otherwise unchanged from the prior claim's PASS evidence on commit b7e6cbd.

Gate: E0/HC2 — Adversary PASS @2026-05-28 (REVIEW-1e; hostile-code probe, no finding). Prior CLAIM detail: Repo-local (PR-authored) test_*.py/install_steps.sh/ops.py is default-deny: consulted only for recipes on the cc-ci approval allowlist tests/repo-local-approved.txt (empty ⇒ deny). Centralized gate in discovery.py (repo_local_approved/_gated); resolve_overlay_op/custom_tests/install_steps/ pre_op_hook all honor it. Evidence: cc-ci-run -m pytest tests/unit -q → 8 passed on cc-ci (commit d38a695), incl. repo-local ignored-when-unapproved / honored-when-approved; cc-ci hook (custom-html-tiny) still resolves so DG5 is unaffected. Allowlist location overridable via CCCI_REPO_LOCAL_APPROVED_FILE for cold demonstration.

Blocked

(none) — bootstrap access re-verified @2026-05-28: ssh cc-ci ok (root, NixOS).