autonomic-bot
5741e8838f
The Phase-1e HC2 test asserted custom_tests('custom-html', repo-local) == [] when only the
repo-local dir was set + custom-html had no cc-ci-side functional tests. Phase-2 commit bec9265
added 4 legitimate non-lifecycle test_*.py files under tests/custom-html/{functional,playwright}/
which custom_tests() now correctly returns — breaking the == [] assertion.
The custom_tests behavior is correct; the test fixture was using the real recipe name. Fix: switch
to a synthetic recipe + monkeypatch cc_ci_dir (same pattern already used in the Phase-2 sibling
test_discovery_phase2.py). 5-line change, no behavior change.
Cold-verifiable on cc-ci: cc-ci-run -m pytest tests/unit -v -> 21 passed in 5.38s
(Adversary's F2-1 repro now PASSes; no other regression).
Also: tests/n8n/PARITY.md drafted for the in-flight Q1.2 work (n8n parity port).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3.1 KiB
Parity — n8n
Phase-2 P2 mapping table: every references/recipe-maintainer/recipe-info/n8n/tests/*.py has a
comparable cc-ci test under tests/n8n/functional/, asserting the same thing (not a renamed
file). The Adversary cold-verifies parity by reading the source recipe-info/<file> and the cc-ci
file side-by-side.
| recipe-maintainer file | cc-ci file | what's verified | status |
|---|---|---|---|
recipe-info/n8n/tests/health_check.py |
tests/n8n/functional/test_health_check.py |
The app is reachable over HTTPS and returns a successful response (the original asserted HTTP 200 against a persistent n8n.<suffix> host). The cc-ci port preserves the assertion shape — HTTP 200 from the served root — and adapts to the ephemeral per-run domain via the live_app fixture. |
ported |
Recipe-specific tests (Phase-2 P3, ≥2 beyond parity)
n8n is a workflow-automation engine — its characteristic behavior is running a workflow engine with a registry of node types and a queryable settings/state surface. Two new functional tests:
| cc-ci file | what's verified | rationale |
|---|---|---|
tests/n8n/functional/test_node_types_catalog.py |
Fetches /types/nodes.json and asserts the response is a JSON list, contains a meaningful number (>= 50) of node-type definitions, and includes specific n8n-built-in node names such as n8n-nodes-base.set, n8n-nodes-base.if, and a webhook/HTTP node — proves the n8n runtime bootstrapped its node registry (the workflow engine's core capability), not just "the HTTP server is up". |
The node registry is what makes n8n n8n; an n8n that boots but loads no nodes is broken. |
tests/n8n/functional/test_rest_settings.py |
Fetches /rest/settings and asserts the JSON response carries the expected n8n public settings keys (e.g. data.endpointWebhook, data.versionCli) — the public settings surface is what every editor SPA loads to bootstrap, and proves the n8n REST API initialized + the SPA can talk to it. |
Proves the editor SPA's primary API contract is intact — distinct from "/" returning HTML, which only proves a static asset was served. |
Both tests run in the custom tier against the same live_app shared deployment as the
lifecycle overlays — no extra deploy, no extra teardown.
Backup data-integrity (P4)
Already exercised by the lifecycle overlays from Phase 1d/1e:
tests/n8n/test_backup.py + test_restore.py + ops.py (pre_backup seeds "original" into
/home/node/.n8n/ci-marker.txt; pre_restore mutates to "mutated"; restore must return the
volume to "original"). Read via lifecycle.exec_in_app since n8n state isn't HTTP-served.
Playwright (P6)
n8n's UI is the editor SPA — already exercised inline by tests/n8n/test_install.py::test_serving_and_editor (lifecycle install overlay), which loads the live editor via Chromium and asserts the
HTML carries the n8n shell. Adequate for P6 — the editor renders is the canonical browser flow.
Non-ports
None — recipe-maintainer's n8n tests/ directory contains only health_check.py, which is fully
ported above.