feat(kuma): implement wizard+monitor Playwright test (tests/uptime-kuma/playwright/)

Phase kuma M1 impl: resolves the 2026-05-28 DEFERRED uptime-kuma create-a-monitor item. Approach: Playwright (option b) — python-socketio not in cc-ci Nix env; Playwright handles Socket.IO transparently via the real browser. Selectors confirmed in 2.2.1 compiled bundle (data-cy setup wizard + data-testid monitor form/status badge). Test flow (test_monitor_wizard_and_probe): 1. Setup wizard: admin create via data-cy form → auto-login → /dashboard 2. Create self-probe monitor (https://{live_app}/) → wait ≤90s for "Up" badge 3. Heartbeat table row check: isFirstBeat=important, row has real datetime stamp 4. Negative: dead-port monitor (http://127.0.0.1:19999/dead) → wait ≤60s for "Down" All waits are bounded poll with page.wait_for_function/wait_for_url/wait_for_selector. Admin password: 64-char UUID hex, never printed/logged. Also: DECISIONS.md records Playwright choice; phase state files bootstrapped. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-11 18:15:13 +00:00
parent 9eb5261c1e
commit 8da59cff22
5 changed files with 328 additions and 1 deletions
--- a/BACKLOG-kuma.md
+++ b/BACKLOG-kuma.md
@ -1,7 +1,24 @@
 # BACKLOG — phase `kuma` (uptime-kuma create-a-monitor functional test)

 ## Build backlog
-(Builder-owned — read only for Adversary)
+
+### DONE
+- [x] Phase state files created (STATUS-kuma.md, BACKLOG-kuma.md, REVIEW-kuma.md, JOURNAL-kuma.md)
+- [x] Approach decision: Playwright over python-socketio (recorded in DECISIONS.md)
+- [x] Inspect uptime-kuma 2.2.1 source for exact DOM selectors
+- [x] Implement `tests/uptime-kuma/playwright/test_monitor_wizard.py`
+
+### IN PROGRESS
+- [ ] Open recipe-maintainers/uptime-kuma PR and trigger `!testme`
+- [ ] Confirm drone run green (M1 acceptance)
+- [ ] Claim M1 gate
+
+### PENDING (after M1 Adversary PASS)
+- [ ] Second `!testme` run (flake check — 2 consecutive green)
+- [ ] Update PARITY.md (note the new playwright/ test)
+- [ ] Close DEFERRED.md entry "2026-05-28 — uptime-kuma create-a-monitor"
+- [ ] Claim M2 gate
+- [ ] Write ## DONE after M2 Adversary PASS

 ## Adversary findings
 (Adversary-owned — no items yet; populated as issues are found)
--- a/JOURNAL-kuma.md
+++ b/JOURNAL-kuma.md
@ -0,0 +1,61 @@
+# JOURNAL — phase `kuma` (uptime-kuma create-a-monitor functional test)
+
+Design rationale, investigations, and dead-ends. Adversary does NOT read this before
+forming its verdict (anti-anchoring per plan §6.1). See STATUS-kuma.md for claim context.
+
+---
+
+## 2026-06-11 — Approach selection: Playwright over python-socketio
+
+**Context:** The phase plan offers two choices:
+- (a) python-socketio client speaking Socket.IO events directly
+- (b) Playwright driving the real browser UI
+
+**Investigation:** Checked the cc-ci Nix Python environment:
+```
+/nix/store/x188l04r3gfkh18gy1dpf05fv3kkrgs7-python3-3.12.8-env/lib/python3.12/site-packages/
+→ greenlet, playwright 1.50.0, pytest 8.3.3, pyee, packaging, pluggy, iniconfig
+→ NO socketio, NO websocket-client, NO aiohttp, NO requests
+```
+python-socketio would need a `nix/cc-ci.nix` addition + `nixos-rebuild switch` on cc-ci.
+Playwright is already present. **Chose option (b): no Nix changes, faster to ship.**
+
+**Selector research:** Inspected uptime-kuma 2.2.1 source files in the Docker image:
+- `src/pages/Setup.vue`: confirms `data-cy` attributes on all setup form fields
+- `src/pages/EditMonitor.vue`: confirms `data-testid` on friendly-name, url, save-button
+- `src/pages/Details.vue`: confirms `data-testid="monitor-status"` on status badge
+- Compiled bundle `dist/assets/index-D_mnxLA0.js`: grep confirms all target attributes
+
+**Heartbeat "important" logic:** Checked `server/model/monitor.js` line 1420:
+```
+// * ? -> ANY STATUS = important [isFirstBeat]
+```
+The server marks the first heartbeat as `important=true`, so it WILL appear in the
+important-heartbeat table immediately after the first probe. This means the table row
+check is a reliable proof of real probe execution.
+
+**Status text:** From `src/mixins/socket.js` line 755 (`statusList` computed):
+```javascript
+text: this.$t("Up"),   // UP=1
+text: this.$t("Down"), // DOWN=0
+```
+English locale: "Up" (capital U, lowercase p) and "Down". Used these exact strings in
+the `_wait_for_status` assertions.
+
+**URL routing:** `src/router.js` uses `createWebHistory()` (history mode, not hash mode).
+Routes: `/` → Entry.vue → redirects to `/dashboard`; `/add` → EditMonitor.vue;
+`/dashboard/:id` → Details.vue. So `page.goto(f"{base}/add")` reliably opens the monitor
+form directly.
+
+**Negative test choice:** `http://127.0.0.1:19999/dead`:
+- Inside the container, port 19999 is unused → OS returns ECONNREFUSED instantly
+- Connection-refused causes uptime-kuma to mark the monitor DOWN immediately (no timeout wait)
+- This proves the probe engine makes real outbound calls (not a stub)
+- Included — fits runtime budget easily (~5 s for DOWN detection)
+
+**Runtime budget analysis:**
+- Setup wizard + login: ~10 s
+- Create monitor 1 + wait UP: ~15-30 s (first probe immediate, but socket roundtrip)
+- Create monitor 2 + wait DOWN: ~10 s (ECONNREFUSED is fast)
+- Overhead: ~5 s
+- Total estimate: ~40-55 s — well within ≤90 s target
--- a/STATUS-kuma.md
+++ b/STATUS-kuma.md
@ -0,0 +1,61 @@
+# STATUS — phase `kuma` (uptime-kuma create-a-monitor functional test)
+
+SSOT: `cc-ci-plan/plan-phase-kuma-monitor.md`
+
+## Current state
+
+**Gate: M1 IN PROGRESS** — test implemented, pending first drone run to confirm green.
+
+## What is claimed
+
+### Approach choice (DECISIONS.md)
+Playwright (option b). Justification: python-socketio is NOT available in the cc-ci Nix env
+(confirmed: only playwright + pytest in site-packages). Playwright drives the real browser;
+Socket.IO is handled transparently. No Nix changes needed.
+
+### Test file
+`tests/uptime-kuma/playwright/test_monitor_wizard.py`
+
+### What the test does
+1. Completes uptime-kuma 2.2.1 first-run setup wizard (admin create via browser).
+2. Creates HTTP monitor targeting the app's own root URL (guaranteed UP at test time).
+3. Waits ≤90 s for status badge (`data-testid="monitor-status"`) to show "Up".
+4. Asserts important-heartbeat table row exists with a real datetime stamp (proves probe ran).
+5. Creates a second monitor targeting `http://127.0.0.1:19999/dead` (dead port → connection refused).
+6. Waits ≤60 s for status badge to show "Down" (negative teeth).
+
+### Selectors used (all confirmed in compiled bundle `dist/assets/index-D_mnxLA0.js`)
+- Setup: `data-cy="username-input"`, `data-cy="password-input"`, `data-cy="password-repeat-input"`, `data-cy="submit-setup-form"`
+- EditMonitor: `data-testid="friendly-name-input"`, `data-testid="url-input"`, `data-testid="save-button"`
+- Details: `data-testid="monitor-status"`
+- Heartbeat table: `table.table-hover tbody tr` (first row)
+
+### Secret safety
+Admin password: 64-char UUID hex, generated per-run. Never printed, never in any assertion error message.
+
+### Probe reality
+- "Up" in the status badge comes from `lastHeartbeatList` populated via Socket.IO heartbeat events
+  (socket.js mixin line 755). Cannot be "Up" unless a real probe completed and the server sent the
+  heartbeat over the socket.
+- Important-heartbeat table row exists: `isFirstBeat` is always `important=true` (server/model/monitor.js
+  line 1420). Presence of a row with "YYYY-MM-DD HH:mm:ss" timestamp proves the probe ran after monitor
+  creation.
+- Negative teeth: "Down" can only appear after the probe attempted and got connection-refused.
+
+### How to verify (Adversary cold-check)
+```bash
+# Deploy uptime-kuma against any fresh cc-ci domain, then run:
+CCCI_APP_DOMAIN=<domain> RECIPE=uptime-kuma STAGES=custom \
+  cc-ci-run -m pytest tests/uptime-kuma/playwright/test_monitor_wizard.py -v
+# Expected: test_monitor_wizard_and_probe PASSED
+# In the Drone-path, it runs under the "custom" tier via run_recipe_ci.py.
+```
+
+### Runtime
+Local estimate: wizard ~10 s + 2× (navigate+fill+probe) ≤ ~60 s total. Within ≤90 s budget.
+
+### Next step
+Trigger `!testme` on a uptime-kuma PR; wait for drone run to pass; then claim M1.
+
+## Blocked
+(nothing)
--- a/machine-docs/DECISIONS.md
+++ b/machine-docs/DECISIONS.md
@ -1383,3 +1383,24 @@ declaration is evidence-backed in the recipe_meta comment + upstream registry; i
 way to exercise a PR at all for a recipe in this state. Re-enable path documented per-recipe
 (bluesky: drop EXPECTED_NA + set UPGRADE_BASE_VERSION="0.3.0+v0.4.219" once merged+published).
 Locked by tests/unit/test_upgrade_base.py.
+
+## 2026-06-11 — uptime-kuma: Playwright (option b) for monitor-wizard test (phase kuma)
+
+**Decision:** use Playwright (option b from plan-phase-kuma-monitor.md §1) to implement
+the `tests/uptime-kuma/playwright/test_monitor_wizard.py` test.
+
+**Why not python-socketio (option a):** python-socketio is NOT installed in the cc-ci
+Nix Python environment (site-packages has playwright + pytest only; no socketio wheel).
+Adding it would require modifying `nix/cc-ci.nix` and running `nixos-rebuild switch` on
+cc-ci — extra Nix overhead when Playwright already handles Socket.IO transparently through
+the real browser. The option (a) benefit (speed, headless) is outweighed by the absence of
+the package.
+
+**Why Playwright works here:** uptime-kuma 2.2.1 has stable `data-cy` attributes on the
+setup form and `data-testid` attributes on the monitor form + status badge — confirmed
+present in the compiled bundle (`dist/assets/index-D_mnxLA0.js`). These are the canonical
+Cypress/testing selectors; they do not change without an intentional test-attribute removal.
+The Playwright flow is deterministic: wizard → `/add` form → `/dashboard/:id` detail page.
+
+**Runtime implication:** Playwright adds ~5–10 s overhead vs a headless socketio client,
+but stays well within the ≤90 s budget. Acceptable.
--- a/tests/uptime-kuma/playwright/test_monitor_wizard.py
+++ b/tests/uptime-kuma/playwright/test_monitor_wizard.py
@ -0,0 +1,167 @@
+"""uptime-kuma — §4.3 prescribed: wizard + create-a-monitor + real probe assertion.
+
+Phase kuma: resolves the DEFERRED.md entry "2026-05-28 — uptime-kuma create-a-monitor".
+
+Approach: **Playwright (option b)** — python-socketio is NOT in the cc-ci harness
+environment (Nix site-packages: playwright + pytest only; no socketio wheel); Playwright
+drives the real browser which handles Socket.IO automatically, is version-robust via
+stable `data-cy` / `data-testid` attributes confirmed present in the 2.2.1 compiled
+bundle, and reuses the existing harness browser stack (no Nix changes needed).
+Justification recorded in DECISIONS.md.
+
+Test flow (§4.3 contract):
+1. Complete first-run setup wizard (admin create) via the real browser UI.
+2. Create an HTTP monitor whose target the harness controls: the app's own root URL
+   (`https://{live_app}/`) — guaranteed reachable since the deploy already passed its
+   health gate before this test tier runs.
+3. Wait ≤90 s for the first heartbeat to arrive and report **Up** (uptime-kuma probes
+   immediately on monitor creation; typical first-result time is < 15 s).
+4. Assert `data-testid="monitor-status"` text is "Up" AND the important-heartbeat table
+   has at least one row with a real datetime stamp (proves a probe ran, not a config echo).
+5. **Negative teeth:** create a second monitor targeting `http://127.0.0.1:19999/dead`
+   (a port guaranteed to refuse connections inside the container → instant DOWN).  Wait
+   ≤60 s; assert status is "Down". This proves the probe engine actually makes outbound
+   checks and is not a stub. Runtime budget: connection-refused completes in < 1 s, so
+   the DOWN detection is fast.
+
+Runtime budget: wizard ~10 s, two monitor create+wait cycles ≤90 s total. Well within
+the ≤ ~90 s added-to-functional-tier target from the phase plan.
+
+Secret safety: admin password generated per-run as a 64-char UUID hex string; it is
+NEVER printed, logged, or included in assertion messages. The fixture uses `_pw` (leading
+underscore convention) and no error path includes its value.
+"""
+
+from __future__ import annotations
+
+import os
+import re
+import sys
+import uuid
+
+sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "..", "runner"))
+from harness import browser as harness_browser  # noqa: E402
+
+# Pattern for uptime-kuma's datetime format ("YYYY-MM-DD HH:mm:ss") rendered by its
+# Datetime.vue component (format confirmed from mixin source — datetimeFormat calls
+# dayjs with "YYYY-MM-DD HH:mm:ss").
+_DATETIME_RE = re.compile(r"\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}")
+
+_SETUP_TIMEOUT_MS = 30_000  # wizard + auto-login settle
+_FORM_TIMEOUT_MS = 15_000   # form element wait
+_PROBE_UP_MS = 90_000       # first UP probe (typically < 15 s, but bound generously)
+_PROBE_DOWN_MS = 60_000     # connection-refused DOWN (typically < 5 s)
+_DETAIL_URL_MS = 20_000     # wait for /dashboard/:id after Save
+
+
+def _wait_for_status(page, expected_text: str, timeout_ms: int) -> None:
+    """Poll the monitor-status badge until its text matches expected_text.
+
+    Uses page.wait_for_function so the wait is driven by Playwright's internal event loop
+    (no sleep), bounded by timeout_ms. Raises on timeout.
+    """
+    escaped = expected_text.replace('"', '\\"')
+    page.wait_for_function(
+        f'() => {{'
+        f'  const el = document.querySelector(\'[data-testid="monitor-status"]\');'
+        f'  return el && el.textContent.trim() === "{escaped}";'
+        f'}}',
+        timeout=timeout_ms,
+    )
+
+
+def _fill_monitor_form(page, name: str, url: str) -> None:
+    """Fill the EditMonitor form (at /add) and click Save.
+
+    Waits for the friendly-name input to be ready (signals the form rendered).
+    Uses stable data-testid attributes confirmed in the 2.2.1 compiled bundle.
+    """
+    page.wait_for_selector('[data-testid="friendly-name-input"]', timeout=_FORM_TIMEOUT_MS)
+    page.locator('[data-testid="friendly-name-input"]').fill(name)
+    # url-input: HTTP(s) type is the default; the URL field (id="url") has data-testid="url-input"
+    page.locator('[data-testid="url-input"]').first.fill(url)
+    page.locator('[data-testid="save-button"]').click()
+
+
+def test_monitor_wizard_and_probe(live_app):
+    """Setup wizard → create self-probe monitor (→ Up) + dead-port monitor (→ Down)."""
+    from playwright.sync_api import sync_playwright
+
+    # Per-run admin credentials. Password is 64 hex chars and NEVER logged.
+    _admin_user = "admin"
+    _pw = uuid.uuid4().hex + uuid.uuid4().hex  # noqa: S311 — not a crypto secret, test-only
+
+    base = f"https://{live_app}"
+
+    with sync_playwright() as p:
+        browser = p.chromium.launch(args=["--no-sandbox"])
+        try:
+            ctx = browser.new_context(ignore_https_errors=True)
+            page = ctx.new_page()
+
+            # ── 1. Setup wizard ──────────────────────────────────────────────────
+            # Navigate to root; uptime-kuma 2.x redirects to /setup on first boot.
+            harness_browser.goto_with_retry(
+                page,
+                f"{base}/",
+                accept_statuses=(200,),
+                wait_until="domcontentloaded",
+                deadline_seconds=60,
+            )
+            # Wait for the setup form (data-cy confirmed in 2.2.1 bundle).
+            page.wait_for_selector('[data-cy="username-input"]', timeout=_SETUP_TIMEOUT_MS)
+            page.locator('[data-cy="username-input"]').fill(_admin_user)
+            page.locator('[data-cy="password-input"]').fill(_pw)
+            page.locator('[data-cy="password-repeat-input"]').fill(_pw)
+            page.locator('[data-cy="submit-setup-form"]').click()
+
+            # After Create: uptime-kuma auto-logs in and pushes router to "/" → Entry.vue
+            # redirects to /dashboard. Wait for the dashboard URL to settle.
+            page.wait_for_url(f"{base}/dashboard", timeout=_SETUP_TIMEOUT_MS)
+
+            # ── 2. Create monitor: self-probe (target = app root → expect Up) ──
+            page.goto(f"{base}/add", wait_until="domcontentloaded")
+            _fill_monitor_form(page, "kuma-self-probe", f"{base}/")
+            # After Save, router navigates to /dashboard/:id (monitor detail page).
+            page.wait_for_url(f"{base}/dashboard/**", timeout=_DETAIL_URL_MS)
+
+            # ── 3. Wait for Up status (first probe fires immediately on creation) ──
+            _wait_for_status(page, "Up", _PROBE_UP_MS)
+
+            # ── 4. Verify a real heartbeat record exists (proves a probe ran) ────
+            # The important-heartbeat table (Details.vue) shows all beats where
+            # `important=true`; the server marks the FIRST beat as important
+            # (isFirstBeat logic, server/model/monitor.js line 1420). So after the
+            # first probe completes there will be exactly one row with a datetime stamp.
+            heartbeat_rows = page.locator("table.table-hover tbody tr")
+            heartbeat_rows.first.wait_for(timeout=_FORM_TIMEOUT_MS)
+            first_row_text = heartbeat_rows.first.text_content(timeout=5_000) or ""
+            assert "No important events" not in first_row_text, (
+                "Heartbeat table shows 'No important events' — the self-probe may not "
+                "have recorded a heartbeat yet"
+            )
+            # The row must contain a real datetime stamp (format: "YYYY-MM-DD HH:mm:ss").
+            assert _DATETIME_RE.search(first_row_text), (
+                f"Heartbeat row has no datetime matching YYYY-MM-DD HH:mm:ss; "
+                f"row text: {first_row_text!r}"
+            )
+
+            # ── 5. Negative teeth: dead-port monitor (→ Down) ───────────────────
+            # http://127.0.0.1:19999/dead — port 19999 is unused inside the container;
+            # the OS returns connection-refused instantly (<1 s), so DOWN arrives fast.
+            page.goto(f"{base}/add", wait_until="domcontentloaded")
+            _fill_monitor_form(page, "kuma-dead-probe", "http://127.0.0.1:19999/dead")
+            page.wait_for_url(f"{base}/dashboard/**", timeout=_DETAIL_URL_MS)
+
+            _wait_for_status(page, "Down", _PROBE_DOWN_MS)
+
+            # Confirm the dead monitor shows Down (belt-and-suspenders).
+            status_text = (
+                page.locator('[data-testid="monitor-status"]').text_content(timeout=5_000) or ""
+            ).strip()
+            assert status_text == "Down", (
+                f"Dead-port monitor expected 'Down', got {status_text!r}"
+            )
+
+        finally:
+            browser.close()