cc-ci/machine-docs/BACKLOG-2.md

BACKLOG — Phase 2 (per-recipe test authoring)

Phase-namespaced backlog. Builder edits ## Build backlog; Adversary edits ## Adversary findings. Phase plan: /srv/cc-ci/cc-ci-plan/plan-phase2-recipe-tests.md

Build backlog

Q0 — Harness additions

• Q0.1 — Vendor recipe-maintainer/utils/tests/helpers.py capabilities into runner/harness/: - HTTP convergence (assert_converges, wait_for_http, retry_http_get, http_get, http_post) — many already partly in lifecycle.py/generic.py; add the missing pieces + consolidate (no duplicate logic). - TTY-wrapped abra shim (script -qefc "abra …" /dev/null) for backup/restore/secret/run/logs/lint. - Backup data-integrity primitive (volume/DB seed → marker survival check).
• Q0.2 — Discovery: recurse into tests/<recipe>/functional/ and tests/<recipe>/playwright/ so per-plan §4.1 directory layout is honored. Update discovery.custom_tests + enroll-recipe.md.
• Q0.3 — PARITY.md template + a worked custom-html example (tests/custom-html/PARITY.md). Port custom-html health_check.py to a functional test.
• Q0.4 — Dependency resolver harness primitive (read tests/<recipe>/recipe.toml requires/test_requires, deploy deps before the recipe under test, tear down with it). Mind MAX_TESTS/node budget; sequence heavy ones. Defer SSO-realm-setup wiring to Q2.
• Q0.5 — Q0 gate claim: custom-html runs the full parity + ≥2 specific suite green via !testme; PARITY.md complete for custom-html; harness changes recorded in DECISIONS.md.

Q1 — Pattern proof (custom-html + n8n)

• Q1.1 — custom-html: ≥2 NEW recipe-specific functional tests (beyond parity health_check). Candidates from plan §4.3: "serve/persist content: write content, fetch it back" — custom-html serves the /usr/share/nginx/html volume, so a content round-trip + a content-type header check are appropriate. Playwright already exists in tests/custom-html/test_install.py.
• Q1.2 — n8n: enroll under cc-ci. Port recipe-info/n8n/tests/health_check.py → tests/n8n/functional/health_check.py. Add ≥2 specific tests: (a) create a workflow via API, execute it, assert result; (b) the workflow persists across an upgrade. PARITY.md filled.
• Q1.3 — Real backup data-integrity for n8n: seed a workflow → backup → wipe → restore → list workflows, prove the seeded one survived. (custom-html already has this pattern from 1e.)
• Q1.4 — Q1 gate: both recipes green via !testme; both PARITY.md complete.

Q2 — SSO providers (keycloak + authentik)

• Q2.1 — keycloak: port tests/keycloak/oidc_integration.py (the dependent-recipe test) and tests/health_check.py. Add specific tests from plan §4.3 (realm+client via admin API; password and client-credentials token grants; JWT claims).
• Q2.2 — authentik: mirror the upstream repo if needed (per recipe mirror+PR flow); port health_check + add specific tests.
• Q2.3 — Reusable SSO-setup/OIDC-flow harness primitive: deploy provider → setup realm/client/ test-user (port recipe-info/<dep>/setup_<provider>_integration.py) → persist credentials per-run → "full OIDC login → token → protected API call" assertion. Implement once in runner/harness/; reused by every SSO-dependent recipe.
• Q2.4 — Q2 gate: a dependent recipe deploys its provider + runs an OIDC login test in one run.

Q3 — SSO-dependent suite (lasuite-docs, lasuite-drive, lasuite-meet, cryptpad, immich)

• Q3.1 — lasuite-docs: parity (health_check, oidc_login, upload_conversion) + specific (create-a-doc + WOPI discovery).
• Q3.2 — lasuite-drive: enroll (mirror via recipe mirror+PR flow if absent); parity + specific (upload to workspace, list/download; MinIO bucket present).
• Q3.3 — lasuite-meet: parity (health_check, oidc_login, meeting_flow, webrtc-media, webrtc-relay) + specific (create-a-room, two-user LiveKit token issuance, ICE-candidate gathering).
• Q3.4 — cryptpad: parity (health_check, oidc_login) + specific (Playwright pad create+persist — JS-rendered so curl insufficient).
• Q3.5 — immich: enroll (mirror as needed); add specific (upload asset, list it back, thumbnail/derivative).
• Q3.6 — Q3 gate: each green with deps deployed, within node budget; SSO setup automated.

Q4 — Remaining recipes

• Q4.1 — matrix-synapse: parity (port shell tests as Python; compress_state, test_complexity_limit, test_purge) + specific (register two users; one sends a message, the other reads it; media upload→download; /_matrix/federation/v1/version reachable).
• Q4.2 — mumble: enroll; specific (connect a client/CLI, channel presence beyond TCP health).
• Q4.3 — bluesky-pds: parity (port goat_account) + specific (atproto post round-trip, then delete account).
• Q4.4 — ghost: enroll; specific (create-a-post round-trip).
• Q4.5 — mattermost-lts: enroll; specific (create-a-message round-trip).
• Q4.6 — discourse: enroll; specific (create-a-topic round-trip).
• Q4.7 — plausible: enroll; specific (track a test event, query it back).
• Q4.8 — uptime-kuma: enroll; specific (create a monitor, list it).
• Q4.9 — mailu: enroll; specific (create a mailbox, send/receive verification).
• Q4.10 — drone: enroll; specific (create/list builds via API).
• Q4.11 — Q4 gate: each recipe green with parity + specific.

Q5 — Completeness + docs

• Q5.1 — docs/enroll-recipe.md updated with the per-recipe test contract (§4.1), the functional/ and playwright/ subdirectory layout, the PARITY.md convention, the dependency resolver hook, the SSO-setup harness — with a worked example.
• Q5.2 — Adversary samples a subset and cold-verifies parity tables + specific tests are real (not health-only, not skipped). NO weakened test, no corners cut (P7).
• Q5.3 — Phase 2 ## DONE after all P1–P8 Adversary cold-verified PASS, no standing VETO.

Adversary findings

(empty — Adversary writes here)