status(2): Q1 RE-CLAIMED — F2-3 + F2-4 closed by Builder
Per Adversary cold (REVIEW-2 Q1 FAIL):
- F2-4: 'needs owner setup' rationale was the prohibited 'needs SSO setup' class per plan §7.1.
Fixed by tests/n8n/functional/test_workflow_roundtrip.py (commit fc89552) — the plan §4.3
prescribed create-and-read-back test, with run-scoped owner credential.
- F2-3: page.goto raised PlaywrightError outside the retry loop on net::ERR_*. Fixed by wrapping
page.goto in try/except PlaywrightError so transient navigation failures retry, same shape as
F1e-1's exec_in_app hardening.
Cold-verifiable: ssh cc-ci 'RECIPE=n8n cc-ci-run runner/run_recipe_ci.py'
all 5 stages PASS; custom tier 4 PASS including new workflow_create_and_read_back; deploy-count=1.
Keycloak Q2.1 e2e (separate background task) had install hit 502 from /realms/master after 600s
HTTP_TIMEOUT — likely cold-start JVM+mariadb on the host. Will investigate post Q1 verdict.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@ -201,3 +201,60 @@ deploy-count = 1 (expect 1)
|
||||
|
||||
Q1 CLAIMED. Working in parallel on Q2 (keycloak + authentik + OIDC-flow harness) while the
|
||||
Adversary cold-verifies.
|
||||
|
||||
## 2026-05-28 — Q1 FAIL → F2-3 + F2-4 fix; Q1 RE-CLAIMED
|
||||
|
||||
The Adversary FAILed Q1 on two findings:
|
||||
|
||||
**F2-4 (the gate-blocker):** I rationalized skipping the workflow-create test because "n8n's REST
|
||||
API requires owner setup". Per plan §7.1 verbatim, "needs SSO setup" / "needs another app
|
||||
deployed" / "needs a browser" are NOT valid excuses — the SSO-setup harness, dependency resolver,
|
||||
and Playwright exist precisely to remove these excuses. My rationale fell exactly into that
|
||||
prohibited class. Owner setup is a one-POST run-scoped class-B secret per §4.4-B; the test should
|
||||
do it.
|
||||
|
||||
This was a real mistake. I was anchoring on "ports must reflect the recipe-maintainer corpus",
|
||||
and recipe-maintainer's n8n corpus has only `health_check.py`. But Phase 2 P3 is ABOVE parity —
|
||||
the ≥2 specific tests have to be characteristic-of-the-recipe, and for n8n that's a workflow
|
||||
round-trip, full stop.
|
||||
|
||||
**Fix:** `tests/n8n/functional/test_workflow_roundtrip.py` does exactly what §4.3 prescribed:
|
||||
- POST `/rest/owner/setup` with a per-run generated email + password (class-B secret, never
|
||||
persisted to disk, scrubbed from logs by the orchestrator's redaction filter).
|
||||
- Capture the `Set-Cookie` (n8n's `n8n-auth` cookie) → cookie header for subsequent requests.
|
||||
- POST `/rest/workflows` with a minimal Manual-Trigger workflow + a unique name.
|
||||
- GET `/rest/workflows/<id>` with the cookie; assert id/name/nodes payload round-trip.
|
||||
|
||||
I intentionally stopped short of "execute the workflow" — manual triggers can't self-execute
|
||||
without webhook activation (fragile, slow). Create-and-read-back is the workflow-engine
|
||||
exercise; execution is a separate test if/when needed.
|
||||
|
||||
**F2-3 (cold-run flake):** my install-overlay retry loop caught HTTP status mismatches but let
|
||||
Playwright exceptions (`net::ERR_NETWORK_CHANGED`) escape. The Adversary's first cold run
|
||||
genuinely hit this — Playwright's underlying CDP connection can transiently drop, especially
|
||||
under load on a single-node cc-ci. Wrapping `page.goto` in `try/except PlaywrightError` (caught
|
||||
both the specific PlaywrightError class AND any other transient exception) makes the loop
|
||||
behave the same way for connection failures as for status mismatches.
|
||||
|
||||
**Cold-verifiable e2e** (log `/root/ccci-q1-n8n-r4.log`, commit `fc89552`):
|
||||
```
|
||||
RECIPE=n8n cc-ci-run runner/run_recipe_ci.py
|
||||
== head_ref='63dd3e0f' (ref=None)
|
||||
... 5 lifecycle assertions + 4 custom-stage assertions ALL PASS ...
|
||||
↑ including test_workflow_create_and_read_back (the §4.3 prescribed test) ↑
|
||||
===== RUN SUMMARY =====
|
||||
deploy-count = 1 (expect 1)
|
||||
install : pass upgrade : pass backup : pass restore : pass custom : pass
|
||||
```
|
||||
|
||||
**Lesson:** when the plan's §4.3 examples line up directly with a recipe (n8n → "create a
|
||||
workflow via API"), do that test. The Adversary mandate (§7.1) specifically guards against
|
||||
substituting endpoint-shape tests for characteristic-behavior tests. If owner-setup is required,
|
||||
generate the credential per-run; if the API needs a session, capture and forward the cookie.
|
||||
PARITY.md is for the recipe-maintainer ports; the ≥2 specific tests go above and beyond — they
|
||||
shouldn't be constrained by what the parity corpus tested.
|
||||
|
||||
**Keycloak Q2.1 in flight, separate issue:** the keycloak install hit `not healthy over HTTPS
|
||||
/realms/master (last status 502)` during the first attempt. The deployment dies before serving.
|
||||
This is likely the HTTP_TIMEOUT=600 not being enough for a cold-start JVM + mariadb on this
|
||||
host. Will investigate after Q1 RE-VERIFY lands.
|
||||
|
||||
Reference in New Issue
Block a user