cc-ci/machine-docs/STATUS-2.md

STATUS — Phase 2 (per-recipe test authoring)

Phase plan (SSOT): /srv/cc-ci/cc-ci-plan/plan-phase2-recipe-tests.md Loop state for THIS phase: STATUS-2 / BACKLOG-2 / REVIEW-2 / JOURNAL-2 (DECISIONS.md shared). Phase 1/1b/1c/1d/1e STATUS/BACKLOG/REVIEW files are HISTORY (all DONE) — not this phase's state.

Phase

Phase 2 authors per-recipe test content on top of the corrected Phase 1/1d/1e shared harness. Per the plan, for every maintained Co-op Cloud recipe (§5 target set), the cc-ci tests/<recipe>/ tree must carry:

• Phase-1d/1e lifecycle overlays (assertion-only, additive) — test_install.py, test_upgrade.py, test_backup.py, test_restore.py + ops.py pre-op seeds.
• Parity-ported tests from references/recipe-maintainer/recipe-info/<recipe>/tests/*.py, one-to-one (P2), with a PARITY.md mapping table.
• ≥2 NEW recipe-specific functional tests (P3) — characteristic behavior, not just status==200.
• Real backup data-integrity (P4): seed → backup → mutate → restore → assert seeded data survived.
• Dependency resolution (P5): recipes that need other apps (SSO providers, DBs) deploy them in-run.
• Playwright (P6) where the app's core UX is a UI flow.
• Docs (P8): docs/enroll-recipe.md updated with the per-recipe test contract + worked example.

Definition of Done (Phase 2) — P1–P8, each Adversary cold-verified in REVIEW-2

• P1 — Coverage. Every recipe in §5 target set has a tests/<recipe>/ suite enrolled and a full green !testme run (install + upgrade + backup-restore).
• P2 — Parity port. Every recipe-info/<recipe>/tests/*.py has a comparable cc-ci test; tests/<recipe>/PARITY.md records the mapping; non-ports documented in DECISIONS.md.
• P3 — Recipe-specific depth. Each recipe has ≥2 new functional tests beyond parity (characteristic behavior, real assertions on app state/responses).
• P4 — Backup data-integrity is real. Seed → backup → mutate → restore → assert seeded data survived (recipe-aware, not health-only). Pattern already proven in Phase 1e on custom-html.
• P5 — Dependencies handled. Recipes with deps declare them; harness deploys deps within the run (respecting MAX_TESTS); SSO setup runs automatically.
• P6 — Browser flows where they matter (D3). UI-centric recipes have a Playwright test of the core flow (login, create-an-object, etc.).
• P7 — No weakened tests, no corners cut. Every assertion is real; nothing skip/xfail'd, mocked, or health-only stand-in. Any "untestable" claim is a true env-level blocker with Adversary sign-off.
• P8 — Docs. docs/enroll-recipe.md updated with the per-recipe test contract (§4.1) and a worked example; a new engineer can add a recipe's full suite from the docs.

Milestones (plan §6)

• Q0 — Harness additions (HTTP/convergence, OIDC-flow, dep resolver, backup data-integrity, TTY abra). Reference recipe (custom-html) uses them for full parity+specific suite, green via !testme.
• Q1 — Pattern proof (custom-html + n8n): full parity + ≥2 specific + real backup data-integrity.
• Q2 — SSO providers (keycloak + authentik); reusable SSO-setup/OIDC-flow harness e2e.
• Q3 — SSO-dependent suite (lasuite-docs, lasuite-drive, lasuite-meet, cryptpad, immich); deps auto-deployed, SSO setup automated, parity + specific.
• Q4 — Remaining recipes (matrix-synapse, mumble, bluesky-pds, ghost, mattermost-lts, discourse, plausible, uptime-kuma, mailu, drone).
• Q5 — Completeness + docs; flip ## DONE.

In flight

Q1 — Pattern proof. custom-html overlay + parity + ≥2 specific tests proven green end-to-end on cc-ci (see Q0 evidence). Next: n8n (Q1.2) — port recipe-info/n8n/tests/health_check.py + add ≥2 specific (workflow create/execute + survives upgrade) + real backup data-integrity (Q1.3).

Gate

Gate: Q0 — RE-CLAIMED, awaiting Adversary @2026-05-28 (commit 5741e88 — F2-1 fix landed on top of the original Q0 changeset). Acceptance evidence (per plan §6 Q0): a reference recipe (custom-html) uses the new harness additions for a full parity + specific suite, green via the existing run path. F2-1 (test_custom_tests_repo_local_gated stale assertion) closed by Builder; cold re-run on cc-ci → 21/21 PASS including the previously-failing test. F2-2 (scope observation: OIDC-flow + dep resolver not in Q0) acknowledged — those primitives implement when Q2/Q3 consume them; BACKLOG-2 Q0.4 remains open and explicitly deferred.

Objective evidence pointers (Q0):

• Harness additions landed
  • runner/harness/http.py — canonical Phase-2 recipe-test HTTP API (vendored from references/recipe-maintainer/utils/tests/helpers.py): http_get, http_post, http_request, retry_http_get, retry_http_post, wait_for_http, assert_converges. JSON + form bodies, transport-failure → status=0.
  • runner/harness/discovery.custom_tests recurses into tests/<recipe>/functional/ and tests/<recipe>/playwright/ (Phase 2 §4.1 layout) while excluding lifecycle test_<op>.py names; HC2 repo-local gate continues to apply.
  • TTY abra wrapper already present in runner/harness/abra.py::_run_pty (Phase 1d) — reused.
• Unit-test proof (deterministic, cc-ci; post-F2-1 fix commit 5741e88)
  • cc-ci-run -m pytest tests/unit -v → 21 passed in 5.38s (the previously-failing test_custom_tests_repo_local_gated now passes; synthetic-recipe + monkeypatch fixture):
    • 8× pre-existing tests/unit/test_discovery.py (overlay + HC2 gate, regressed).
    • 2× new tests/unit/test_discovery_phase2.py (functional/+playwright/ recursion + HC2 gate still applies to subdirs).
    • 11× new tests/unit/test_http.py (in-process http.server fixture — JSON parsing, 4xx-with-body, non-JSON body, transport-failure=0, headers, JSON+form POST, retry convergence, retry timeout, wait_for_http, assert_converges return value).
• End-to-end proof (custom-html on cc-ci, the reference recipe)
  • RECIPE=custom-html cc-ci-run runner/run_recipe_ci.py (log /root/ccci-q0-customhtml-full.log):
    • install/upgrade/backup/restore/custom all PASS, deploy-count=1.
    • HC1 PR-head proof: head_ref=8a026066 == chaos-version=8a026066, version 1.10.0→1.11.0.
    • 5 lifecycle assertions (generic + cc-ci overlay across 4 ops) + 4 custom-stage assertions (3 functional + 1 playwright). Reference command for Adversary cold re-run: RECIPE=custom-html cc-ci-run runner/run_recipe_ci.py.
• Per-recipe contract artifact landed
  • tests/custom-html/PARITY.md — parity row for health_check.py, rationale for the 2 recipe-specific tests + the data-integrity + playwright sections.
  • tests/custom-html/functional/{test_health_check.py,test_content_roundtrip.py,test_content_type_header.py} — parity port + 2 NEW recipe-specific tests; each parity file carries the SOURCE: recipe-info/custom-html/tests/<file> comment for audit.
  • tests/custom-html/playwright/test_browser_smoke.py — Phase-2 P6 home.

Reference command for Adversary (cold, on cc-ci):

ssh cc-ci 'cd /root/cc-ci && cc-ci-run -m pytest tests/unit -v && RECIPE=custom-html cc-ci-run runner/run_recipe_ci.py'

Blocked

(none) — bootstrap access re-verified @2026-05-28: ssh cc-ci ok (root, NixOS 24.11), Gitea API HTTP 200, wildcard DNS resolves to gateway 143.244.213.108.

Carryover from Phase 1e (not blockers for Phase 2)

• F1e-2 [adversary] — concurrent same-recipe abra recipe fetch race in runner/run_recipe_ci.py::fetch_recipe. Pre-existing in Phase 1d; not a 1e regression. Drone caps MAX_TESTS=1 today, so practical impact bounded. Tracked for Phase-2 breadth-ramp if concurrent recipe runs become routine.