feat(2): Q1.2 — n8n Phase-2 parity + functional + robust install (full e2e green)

- tests/n8n/PARITY.md: parity table (health_check ported) + 2 recipe-specific
  functional tests with rationale + data-integrity section pointing to
  Phase-1d/1e lifecycle overlays.
- tests/n8n/functional/test_health_check.py: parity port of
  recipe-info/n8n/tests/health_check.py — SOURCE comment.
- tests/n8n/functional/test_rest_settings.py: NEW recipe-specific — polls
  /rest/settings until response is application/json (not the 'n8n is starting
  up' SPA placeholder); asserts known n8n public-settings keys
  (userManagement/defaultLocale/authCookie) in the 'data' envelope. Proves the
  editor SPA's primary API contract is intact.
- tests/n8n/functional/test_login_state.py: NEW recipe-specific — polls
  /rest/login until response is JSON; proves the user-management/auth subsystem
  initialized on top of the public-settings layer.
- tests/n8n/test_install.py: install overlay's Playwright now polls page.goto
  until status==200 (n8n's / route can return 404 briefly while the SPA route
  registers on top of /healthz=200). Bounded poll, no bare sleep, raise on
  persistent failure — same robustness pattern as Phase-1e exec_in_app.

Cold-verifiable on cc-ci (log /root/ccci-q1-n8n-r3.log):
  RECIPE=n8n cc-ci-run runner/run_recipe_ci.py
  all 5 stages PASS, deploy-count=1, head_ref=63dd3e0f==chaos-version=63dd3e0f,
  version 3.1.0+2.9.4 -> 3.2.0+2.20.6 (HC1 non-vacuous), 5 lifecycle assertions
  + 3 custom-stage assertions all PASS.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-28 06:48:00 +01:00
parent 5ab25c3dea
commit 2f3d5aa78f
5 changed files with 232 additions and 9 deletions

View File

@ -11,13 +11,16 @@ file side-by-side.
## Recipe-specific tests (Phase-2 P3, ≥2 beyond parity)
n8n is a workflow-automation engine — its characteristic behavior is **running a workflow engine
with a registry of node types and a queryable settings/state surface**. Two new functional tests:
n8n's characteristic behavior is **a working REST API on top of a working workflow engine**. /healthz
returns 200 long before the actual n8n process is ready — the REST endpoints serve a placeholder
HTML page ("n8n is starting up. Please wait") with status 200 during early boot. So a meaningful
n8n-specific test must distinguish "the HTTP layer answers" (what generic+install does) from "the
n8n REST API actually responds with JSON". Two new functional tests:
| cc-ci file | what's verified | rationale |
|---|---|---|
| `tests/n8n/functional/test_node_types_catalog.py` | Fetches `/types/nodes.json` and asserts the response is a JSON list, contains a meaningful number (>= 50) of node-type definitions, and includes specific n8n-built-in node names such as `n8n-nodes-base.set`, `n8n-nodes-base.if`, and a webhook/HTTP node — proves the n8n runtime bootstrapped its **node registry** (the workflow engine's core capability), not just "the HTTP server is up". | The node registry is what makes n8n n8n; an n8n that boots but loads no nodes is broken. |
| `tests/n8n/functional/test_rest_settings.py` | Fetches `/rest/settings` and asserts the JSON response carries the expected n8n public settings keys (e.g. `data.endpointWebhook`, `data.versionCli`) — the public settings surface is what every editor SPA loads to bootstrap, and proves the n8n REST API initialized + the SPA can talk to it. | Proves the editor SPA's primary API contract is intact — distinct from "/" returning HTML, which only proves a static asset was served. |
| `tests/n8n/functional/test_rest_settings.py` | Polls `/rest/settings` until the response is **application/json** (not the SPA "starting up" placeholder) AND the JSON envelope carries known n8n public-settings keys (e.g. `endpointWebhook`, `versionCli`, `n8nMetadata`, `instanceId`). | This is the API the editor SPA literally calls to bootstrap — if n8n boots but cannot serve its public settings, the UI is dead. Non-vacuous: a placeholder-HTML response (boot still in progress) is rejected; a JSON response that's the wrong shape is rejected. |
| `tests/n8n/functional/test_login_state.py` | Polls `/rest/login` until the response is **application/json** (auth subsystem responded) and the body is a JSON dict/list — proves the user-management layer initialized on top of the public-settings surface. | Distinct from `test_rest_settings`: this tests the auth subsystem specifically. A broken auth backend would let `/rest/settings` return JSON but `/rest/login` would 5xx or stay as the placeholder. |
Both tests run in the **custom** tier against the same `live_app` shared deployment as the
lifecycle overlays — no extra deploy, no extra teardown.

View File

@ -0,0 +1,23 @@
"""n8n — parity port of recipe-maintainer's health_check.py (Phase 2 P2).
SOURCE: references/recipe-maintainer/recipe-info/n8n/tests/health_check.py
The original asserted HTTP 200 from `https://n8n.<DOMAIN_SUFFIX>`. The cc-ci port preserves the
assertion shape (HTTP 200 from `/`), adapted to the ephemeral per-run domain via the `live_app`
fixture. Runs in the custom tier against the shared post-install live deployment.
"""
from __future__ import annotations
import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "..", "runner"))
from harness import http as harness_http # noqa: E402
def test_n8n_returns_200(live_app):
"""Parity with recipe-info/n8n/tests/health_check.py: HTTP 200 from the app root."""
url = f"https://{live_app}/"
status, _ = harness_http.retry_http_get(url, expect_status=200, max_wait=60, interval=3)
assert status == 200, f"n8n at {url} returned HTTP {status} (expected 200)"

View File

@ -0,0 +1,96 @@
"""n8n — recipe-specific functional test (Phase 2 P3, ≥2 beyond parity).
n8n's `/rest/login` reports the auth/owner state — what the editor SPA calls right after settings
to decide between "show the owner-setup wizard" vs "show the login page" vs "we already have a
session". A fresh-deploy n8n with user-management enabled MUST return a JSON body indicating no
owner exists yet (status=200 + a no-owner / no-session shape). A still-starting n8n returns the
"n8n is starting up" placeholder HTML with the same 200; this test polls until a real JSON shape
arrives and asserts the auth subsystem responded — proving the user-management layer initialized
on top of the public settings (distinct from test_rest_settings.py, which only proves the public
settings endpoint).
Runs in the custom tier against the shared post-install deployment.
"""
from __future__ import annotations
import json
import os
import ssl
import sys
import urllib.request
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "..", "runner"))
from harness import http as harness_http # noqa: E402
def _raw_get(url: str, timeout: int = 15):
"""Same shape as test_rest_settings._raw_get_json: returns (status, content_type, json_or_None,
raw_excerpt_or_err)."""
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
req = urllib.request.Request(url, method="GET")
try:
resp = urllib.request.urlopen(req, timeout=timeout, context=ctx)
except urllib.error.HTTPError as e:
# n8n's /rest/login may return 4xx on a fresh install (no session); still informative
ct = e.headers.get("Content-Type", "") if e.headers else ""
try:
raw = e.read()
except Exception: # noqa: BLE001
return e.code, ct, None, "<read-failed>"
if "application/json" not in ct:
return e.code, ct, None, raw.decode(errors="replace")[:80]
try:
return e.code, ct, json.loads(raw), None
except (json.JSONDecodeError, ValueError) as je:
return e.code, ct, None, str(je)
except Exception as e: # noqa: BLE001
return 0, "", None, str(e)
ct = resp.headers.get("Content-Type", "")
raw = resp.read()
resp.close()
if "application/json" not in ct:
return resp.status, ct, None, raw.decode(errors="replace")[:80]
try:
return resp.status, ct, json.loads(raw), None
except (json.JSONDecodeError, ValueError) as je:
return resp.status, ct, None, str(je)
def test_login_endpoint_returns_json(live_app):
"""Poll /rest/login until it returns a real JSON response (not the SPA placeholder). Assert the
response indicates the auth subsystem is up — either a JSON error envelope (no session) or a
user / state object. n8n's exact response shape evolves: pre-1.x returned a flat error; recent
versions return {"code":..., "message":..., "hint":...} for the no-session case. We accept any
JSON object — but reject the placeholder HTML."""
url = f"https://{live_app}/rest/login"
state: dict = {}
def _ready():
status, ct, body, err = _raw_get(url)
state.update({"status": status, "ct": ct, "err": err, "body": body})
if status == 0:
return None
# The "starting up" placeholder is text/html with 200 -> body is None, ct doesn't contain
# application/json. Reject until ct says JSON.
if "application/json" not in ct:
return None
return body if body is not None else None
body = harness_http.assert_converges(
_ready,
f"GET {url} returns JSON (n8n auth subsystem ready)",
max_wait=180,
interval=5,
)
# The auth endpoint returned a JSON body — that's the proof. The shape varies, but JSON it is.
assert body is not None, f"/rest/login returned no parseable JSON: state={state}"
# If it's a dict, it's the expected envelope; if it's a list, n8n shouldn't do that on this
# endpoint, but accept either; only reject obvious non-shapes.
assert isinstance(body, (dict, list)), (
f"/rest/login returned unexpected JSON type {type(body).__name__}: {body!r}"
)

View File

@ -0,0 +1,88 @@
"""n8n — recipe-specific functional test (Phase 2 P3, ≥2 beyond parity).
n8n's editor SPA bootstraps by calling `/rest/settings` to obtain its public settings (webhook URL
base, version, defaults). A working n8n returns a JSON body from this endpoint; a still-starting
n8n returns the placeholder HTML page "n8n is starting up. Please wait" with the SAME 200 status.
So this test polls until the response is real JSON (not the SPA fallback) and asserts known n8n
public-settings keys are present. Non-vacuous: a broken n8n that boots but cannot serve its public
settings (the API the editor SPA literally requires) would fail here, distinguishing it from a
"the HTTP layer is up but n8n itself is dead" failure that /healthz can't catch.
Runs in the custom tier against the shared post-install deployment (live_app).
"""
from __future__ import annotations
import json
import os
import ssl
import sys
import urllib.request
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "..", "runner"))
from harness import http as harness_http # noqa: E402
def _raw_get_json(url: str, timeout: int = 15):
"""GET the URL and parse the body as JSON only if Content-Type is application/json — n8n's
"starting up" placeholder returns text/html with status 200, so a content-type check
distinguishes a real JSON response from the SPA fallback without relying on path-specific shape.
Returns (status, content_type, json_or_None)."""
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
req = urllib.request.Request(url, method="GET")
try:
with urllib.request.urlopen(req, timeout=timeout, context=ctx) as resp:
ct = resp.headers.get("Content-Type", "")
raw = resp.read()
except Exception as e: # noqa: BLE001
return 0, "", None, str(e)
if "application/json" not in ct:
return resp.status, ct, None, raw.decode(errors="replace")[:80]
try:
return resp.status, ct, json.loads(raw), None
except (json.JSONDecodeError, ValueError) as e:
return resp.status, ct, None, str(e)
def test_rest_settings_returns_json_with_known_keys(live_app):
"""Poll /rest/settings until it returns real JSON (not the "starting up" SPA placeholder), then
assert known n8n public-settings keys are present in the response — proves the editor's primary
API contract is intact."""
url = f"https://{live_app}/rest/settings"
last_state: dict = {}
def _ready():
status, ct, body, err = _raw_get_json(url)
last_state.update({"status": status, "ct": ct, "err": err})
return body if (status == 200 and body is not None) else None
body = harness_http.assert_converges(
_ready,
f"GET {url} returns JSON (n8n REST ready)",
max_wait=180,
interval=5,
)
# n8n /rest/settings shape: {"data": {<settings dict>}, ...}. The exact key set evolves over
# versions; assert STRUCTURE (the "data" envelope) + a few stable bootstrap keys that have been
# part of /rest/settings since n8n introduced user-management — verified on this live deploy
# (e.g. version 3.2.0+2.20.6).
assert isinstance(body, dict), f"/rest/settings returned non-dict JSON: {type(body).__name__}"
data = body.get("data") if "data" in body else body
assert isinstance(data, dict), (
f"/rest/settings response missing 'data' envelope: keys={list(body.keys())[:10]}"
)
# Bootstrap keys the editor SPA relies on across versions:
# - `userManagement`: the auth-mode dict (whether owner-setup is needed, smtp/email mode).
# - `defaultLocale`: i18n bootstrap; present on every n8n install.
# - `authCookie`: the auth-cookie configuration; consumed by the SPA on every page load.
expected_any = ("userManagement", "defaultLocale", "authCookie")
present = [k for k in expected_any if k in data]
assert present, (
f"/rest/settings 'data' missing all of {expected_any}; "
f"got keys: {sorted(data.keys())[:20]} — n8n version may have rotated these names; "
"update the test."
)

View File

@ -20,6 +20,10 @@ def test_serving_and_editor(live_app, meta):
assert status == 200, f"expected 200 from {live_app}/healthz, got {status}"
# A real browser loads the live n8n editor SPA over HTTPS.
# n8n's boot is staged: /healthz returns 200 before / route is registered. So we may briefly
# get 404 from /, then a 200 with the "starting up" placeholder, then the actual SPA. Poll
# page.goto until status==200 (route registered) — Phase 1e exec_in_app pattern: bounded poll,
# no bare sleep, raise on persistent failure.
from playwright.sync_api import sync_playwright
url = f"https://{live_app}/"
@ -28,11 +32,20 @@ def test_serving_and_editor(live_app, meta):
try:
ctx = browser.new_context(ignore_https_errors=True)
page = ctx.new_page()
resp = page.goto(url, wait_until="domcontentloaded", timeout=60000)
assert resp is not None and resp.status in (
200,
304,
), f"page status {resp and resp.status}"
import time
deadline = time.time() + 120
resp = None
last_status = 0
while time.time() < deadline:
resp = page.goto(url, wait_until="domcontentloaded", timeout=30000)
last_status = resp.status if resp is not None else 0
if last_status in (200, 304):
break
time.sleep(3)
assert resp is not None and last_status in (200, 304), (
f"page status {last_status} after polling — n8n route never came up"
)
body = page.content().lower()
assert "n8n" in body or "<html" in body, "no n8n content served"
finally: