recipe-maintainers/cc-ci
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(2): F2-4 + F2-3 — n8n workflow round-trip + Playwright exception catch

Browse Source 
F2-4 (P3/§4.3 floor — gate-blocker on Q1):
  tests/n8n/functional/test_workflow_roundtrip.py: plan §4.3 prescribed test.
    POST /rest/owner/setup with class-B run-scoped owner email+password (plan
    §4.4-B); capture auth cookie; POST /rest/workflows with a minimal Manual-
    Trigger workflow; GET /rest/workflows/<id>; assert the round-trip (id,
    name, nodes payload all preserved). Removes the prohibited 'needs owner
    setup' excuse; exercises n8n's defining persistence + retrieval surface.

F2-3 (cold-run flake on install):
  tests/n8n/test_install.py: wrap page.goto(...) in try/except PlaywrightError
    inside the retry loop so net::ERR_* / connection resets trigger a retry
    instead of an immediate test failure. Same pattern as F1e-1's exec_in_app
    poll+raise hardening.

PARITY.md updated: 3 recipe-specific tests now listed; workflow_roundtrip
called out as the plan §4.3 prescribed create+read-back; rationale for keeping
test_rest_settings / test_login_state retained.

Cold-verifiable on cc-ci (log /root/ccci-q1-n8n-r4.log):
  RECIPE=n8n cc-ci-run runner/run_recipe_ci.py
  all 5 stages PASS, deploy-count=1, head_ref=63dd3e0f==chaos-version=63dd3e0f.
  Custom tier ran 4 PASS: health_check, login_state, rest_settings, AND the
  new workflow_create_and_read_back.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
autonomic-bot
2026-05-28 07:07:34 +01:00
parent 90e95270a0
commit fc89552347
3 changed files with 213 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
tests/n8n/PARITY.md
View File

@ -9,18 +9,21 @@ file side-by-side.
|---|---|---|---|
| `recipe-info/n8n/tests/health_check.py` | `tests/n8n/functional/test_health_check.py` | The app is reachable over HTTPS and returns a successful response (the original asserted HTTP 200 against a persistent `n8n.<suffix>` host). The cc-ci port preserves the assertion shape — HTTP 200 from the served root — and adapts to the ephemeral per-run domain via the `live_app` fixture. | **ported** |
## Recipe-specific tests (Phase-2 P3, ≥2 beyond parity)
## Recipe-specific tests (Phase-2 P3 §4.3 floor: "create-an-object + read-it-back, and one more")
n8n's characteristic behavior is **a working REST API on top of a working workflow engine**. /healthz
returns 200 long before the actual n8n process is ready — the REST endpoints serve a placeholder
HTML page ("n8n is starting up. Please wait") with status 200 during early boot. So a meaningful
n8n-specific test must distinguish "the HTTP layer answers" (what generic+install does) from "the
n8n REST API actually responds with JSON". Two new functional tests:
n8n's defining behavior is **the workflow engine**. The plan §4.3 names the canonical test
directly: "create a workflow via API, execute it, assert the result." So:
| cc-ci file | what's verified | rationale |
|---|---|---|
| `tests/n8n/functional/test_rest_settings.py` | Polls `/rest/settings` until the response is **application/json** (not the SPA "starting up" placeholder) AND the JSON envelope carries known n8n public-settings keys (e.g. `endpointWebhook`, `versionCli`, `n8nMetadata`, `instanceId`). | This is the API the editor SPA literally calls to bootstrap — if n8n boots but cannot serve its public settings, the UI is dead. Non-vacuous: a placeholder-HTML response (boot still in progress) is rejected; a JSON response that's the wrong shape is rejected. |
| `tests/n8n/functional/test_login_state.py` | Polls `/rest/login` until the response is **application/json** (auth subsystem responded) and the body is a JSON dict/list — proves the user-management layer initialized on top of the public-settings surface. | Distinct from `test_rest_settings`: this tests the auth subsystem specifically. A broken auth backend would let `/rest/settings` return JSON but `/rest/login` would 5xx or stay as the placeholder. |
| `tests/n8n/functional/test_workflow_roundtrip.py` | Owner setup via `POST /rest/owner/setup` with a per-run-generated email + password (class-B run-scoped secret, plan §4.4-B); then `POST /rest/workflows` creates a Manual-Trigger workflow with a unique name; then `GET /rest/workflows/<id>` reads it back; asserts the returned id matches, name matches, nodes payload preserved (type/name of the one node). | **Plan §4.3 prescribed test** — create-an-object + read-it-back, exercising n8n's persistence + retrieval. Non-vacuous: a broken persistence layer would round-trip with wrong shape; a wedged engine that serves the SPA but rejects workflow POSTs fails at the create step. |
| `tests/n8n/functional/test_rest_settings.py` | Polls `/rest/settings` until response is **application/json** (rejects the "n8n is starting up" SPA placeholder); asserts known public-settings keys (`userManagement`, `defaultLocale`, `authCookie`) in the `data` envelope. | The editor SPA's primary API contract — proves bootstrap surface is intact. Distinct from `test_workflow_roundtrip.py` (which proves persistence); this proves the SPA can come up at all. |
| `tests/n8n/functional/test_login_state.py` | Polls `/rest/login` until response is **application/json**; asserts JSON dict/list shape — proves the user-management/auth subsystem initialized. | Auth subsystem readiness; distinct from settings (a broken auth backend would let settings return JSON but login would 5xx). |
Three specific tests, exceeding the ≥2 floor — `test_workflow_roundtrip.py` is the plan §4.3
prescribed "create + read-back"; the other two are bootstrap-readiness assertions retained from
the earlier draft because they catch boot-window failure modes the workflow test (which assumes
post-owner-setup state) doesn't.
Both tests run in the **custom** tier against the same `live_app` shared deployment as the
lifecycle overlays — no extra deploy, no extra teardown.