cc-ci/machine-docs/JOURNAL-2.md

JOURNAL — Phase 2 (per-recipe test authoring)

Builder-private (append-only). Builder rationalisations, dead-ends, in-the-moment reasoning. The Adversary does NOT read this before forming a verdict; objective evidence goes in STATUS-2 / REVIEW-2. Phase plan: /srv/cc-ci/cc-ci-plan/plan-phase2-recipe-tests.md

---

2026-05-28 — Phase 2 bootstrap

Phase 1e completed @2026-05-28 (commit 0fe1218, NO VETO, all HC1–HC4 Adversary cold-verified PASS). Foundation is in place: the orchestrator deploys ONCE per run, performs each lifecycle op ONCE (install→deploy / upgrade→chaos-redeploy of PR head / backup→abra app backup / restore→abra app restore), and runs both generic (tests/_generic/test_<op>.py) and overlay (tests/<recipe>/test_<op>.py) assertion files additively against the shared post-op state. Pre-op seeds live in optional tests/<recipe>/ops.py (pre_install/pre_upgrade/pre_backup/ pre_restore). The deploy-count guard (DG4.1) stays =1; teardown is sacred. Per Phase-1e HC1, the upgrade tier proves PR-head was deployed via chaos-version label = head_ref (head SHA from $REF). Per HC2, repo-local PR-authored code runs only for recipes on tests/repo-local-approved.txt (default-deny).

Bootstrap (this session):

1. git pull --rebase — already up to date.
2. Verified §1 access: ssh cc-ci OK (NixOS 24.11), Gitea API HTTP 200, wildcard probe-$RANDOM.ci.commoninternet.net resolves to gateway 143.244.213.108.
3. Read the Phase-2 plan + plan.md §6.1/§7/§9 (loop protocol, single-writer ownership, gate handshake, anti-drift). Read STATUS-1e + REVIEW-1e final to inherit the harness invariants (HC1–HC4 cold-verified PASS, F1e-2 not blocking).
4. Surveyed existing state: tests/<recipe>/ already exists for custom-html, cryptpad, keycloak, lasuite-docs, matrix-synapse, n8n — these were built out as Phase-1d/1e overlays + recipe_meta
   • ops.py. The lifecycle overlay model (test_install/upgrade/backup/restore.py + ops.py) is the foundation. Phase 2 adds parity-port functional tests + ≥2 NEW recipe-specific tests + dependency/SSO resolver + PARITY.md per recipe.
5. Surveyed references/recipe-maintainer (mounted at /srv/recipe-maintainer/) — the parity source. Per-recipe corpus:
   • custom-html — health_check.py (200 check)
   • n8n — health_check.py
   • keycloak — health_check.py + oidc_integration.py (cross-recipe with lasuite-docs)
   • cryptpad — health_check.py + oidc_login.py
   • lasuite-docs — health_check.py + oidc_login.py + upload_conversion.py
   • lasuite-meet — health_check.py + oidc_login.py + meeting_flow.py + webrtc-media.py + webrtc-relay.py
   • matrix-synapse — shell tests: compress_state.sh + test_complexity_limit.sh + test_purge.sh (will port semantics to Python under cc-ci)
   • hedgedoc / authentik / immich / bluesky-pds / mumble / gitea / lichen / lichen-markdown — no tests/ dir under recipe-info yet, will fill from plan §4.3 spec.

Plan-shape orientation:

• tests/<recipe>/test_<op>.py (lifecycle overlays) — already established.
• tests/<recipe>/functional/ — Phase-2 introduces this subdir for parity-port + new specific tests. Discovery currently globs test_*.py at the top level only; will need to recurse (Q0.2).
• tests/<recipe>/playwright/ — same.
• tests/<recipe>/PARITY.md — Phase-2 introduces this; mapping table per recipe.

Bootstrap commits incoming:

• Add STATUS-2.md / BACKLOG-2.md / JOURNAL-2.md (this session).
• DECISIONS.md append: PARITY.md format, functional/ + playwright/ subdirs, dep-resolver shape.

Will now seed DECISIONS, then begin Q0.1 (vendor helpers into runner/harness/) — keeping the custom-html overlay working as the reference recipe. The /loop will self-pace.

2026-05-28 — Q0 + Q1.1 landed; Q0 gate CLAIMED

Worked through Q0.1, Q0.2, Q0.3, Q1.1 in one stretch since they're tightly coupled:

Q0.1 — runner/harness/http.py is the canonical Phase-2 recipe-test HTTP API. Mirrors recipe-maintainer/utils/tests/helpers.py shape (same function names, same return shapes) so parity ports read 1:1, but self-contained (cc-ci runtime does NOT import recipe-maintainer per DECISIONS Phase 2). Existing lifecycle.http_get/http_fetch/http_body stay — they're for infra-level checks like Traefik-404 detection. harness.http is for recipe tests' API calls. SSL context is CERT_NONE because per-run domains use the wildcard cert; the real-cert verification happens in generic.served_cert once per run via the install tier.

Q0.2 — discovery now recurses into functional/ + playwright/ subdirs. Surgically small change to custom_tests; doesn't disturb the lifecycle-tier discovery (overlays still live at top-level). Two new unit tests prove it (recursion works + HC2 gate still applies to subdirs). Pre-existing 8 discovery unit tests still pass.

Q0.3 / Q1.1 — custom-html as the reference recipe:

• PARITY.md mapping table: 1 parity row (health_check) + 2 recipe-specific rows (content_roundtrip + content_type_header) + a backup-integrity reference + a playwright reference.
• functional/test_health_check.py — parity port with SOURCE: recipe-info/custom-html/tests/health_check.py comment for audit.
• functional/test_content_roundtrip.py — NEW: write a uuid.uuid4() marker into nginx's /usr/share/nginx/html volume, fetch over HTTPS, assert exact-byte match. Non-vacuous: a stale page or misrouted backend can't return our random content.
• functional/test_content_type_header.py — NEW: write .html + .txt files with same body ("hello"), HEAD each, assert Content-Type: text/html and text/plain. Caught the case where nginx MIME map breaks even when 200 still works.
• playwright/test_browser_smoke.py — P6: Chromium renders HTML, no console errors.

E2E cold-verifiable evidence on cc-ci (log /root/ccci-q0-customhtml-full.log):

RECIPE=custom-html cc-ci-run runner/run_recipe_ci.py
===== TIER: install (generic=run, overlay=cc-ci:tests/custom-html/test_install.py) =====
  ... generic + overlay both PASS
===== TIER: upgrade =====
  upgrade→PR-head: head_ref=8a026066 chaos-version=8a026066 version=1.10.0+1.28.0→1.11.0+1.29.0
  ... generic + overlay both PASS (data marker "upgrade-survives" survived chaos redeploy)
===== TIER: backup =====
  ... generic + overlay both PASS
===== TIER: restore =====
  ... generic + overlay both PASS (volume restored to "original")
===== TIER: custom =====
  ... 4 PASS (parity health_check, content_roundtrip, content_type_header, browser_smoke)
===== RUN SUMMARY =====
deploy-count = 1 (expect 1)
  install : pass   upgrade : pass   backup : pass   restore : pass   custom : pass

That's the full Phase-2 pattern proven on the reference recipe:

• additive generic+overlay across 4 lifecycle ops (HC3),
• HC1 PR-head deploy proof via chaos-version label match,
• recipe-aware backup data-integrity (marker survives backup/restore cycle),
• 2 NEW recipe-specific functional tests beyond parity (P3 floor met),
• Playwright UI flow (P6),
• deploy-once + clean teardown.

Q0.4 (dep resolver) deferred to Q2: no Q1 recipe (custom-html + n8n) has deps, and the resolver shape will be much clearer once we have keycloak+authentik to deploy as deps. Logged in BACKLOG-2.

Q0 gate now CLAIMED. Working in parallel on Q1.2 (n8n) while the Adversary cold-verifies.