From 8da59cff22a11a51cd8be183ecef9a2ae163ffc8 Mon Sep 17 00:00:00 2001 From: autonomic-bot Date: Thu, 11 Jun 2026 18:15:13 +0000 Subject: [PATCH] feat(kuma): implement wizard+monitor Playwright test (tests/uptime-kuma/playwright/) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase kuma M1 impl: resolves the 2026-05-28 DEFERRED uptime-kuma create-a-monitor item. Approach: Playwright (option b) — python-socketio not in cc-ci Nix env; Playwright handles Socket.IO transparently via the real browser. Selectors confirmed in 2.2.1 compiled bundle (data-cy setup wizard + data-testid monitor form/status badge). Test flow (test_monitor_wizard_and_probe): 1. Setup wizard: admin create via data-cy form → auto-login → /dashboard 2. Create self-probe monitor (https://{live_app}/) → wait ≤90s for "Up" badge 3. Heartbeat table row check: isFirstBeat=important, row has real datetime stamp 4. Negative: dead-port monitor (http://127.0.0.1:19999/dead) → wait ≤60s for "Down" All waits are bounded poll with page.wait_for_function/wait_for_url/wait_for_selector. Admin password: 64-char UUID hex, never printed/logged. Also: DECISIONS.md records Playwright choice; phase state files bootstrapped. Co-Authored-By: Claude Sonnet 4.6 --- BACKLOG-kuma.md | 19 +- JOURNAL-kuma.md | 61 +++++++ STATUS-kuma.md | 61 +++++++ machine-docs/DECISIONS.md | 21 +++ .../playwright/test_monitor_wizard.py | 167 ++++++++++++++++++ 5 files changed, 328 insertions(+), 1 deletion(-) create mode 100644 JOURNAL-kuma.md create mode 100644 STATUS-kuma.md create mode 100644 tests/uptime-kuma/playwright/test_monitor_wizard.py diff --git a/BACKLOG-kuma.md b/BACKLOG-kuma.md index e15bbfa..88c53cc 100644 --- a/BACKLOG-kuma.md +++ b/BACKLOG-kuma.md @@ -1,7 +1,24 @@ # BACKLOG — phase `kuma` (uptime-kuma create-a-monitor functional test) ## Build backlog -(Builder-owned — read only for Adversary) + +### DONE +- [x] Phase state files created (STATUS-kuma.md, BACKLOG-kuma.md, REVIEW-kuma.md, JOURNAL-kuma.md) +- [x] Approach decision: Playwright over python-socketio (recorded in DECISIONS.md) +- [x] Inspect uptime-kuma 2.2.1 source for exact DOM selectors +- [x] Implement `tests/uptime-kuma/playwright/test_monitor_wizard.py` + +### IN PROGRESS +- [ ] Open recipe-maintainers/uptime-kuma PR and trigger `!testme` +- [ ] Confirm drone run green (M1 acceptance) +- [ ] Claim M1 gate + +### PENDING (after M1 Adversary PASS) +- [ ] Second `!testme` run (flake check — 2 consecutive green) +- [ ] Update PARITY.md (note the new playwright/ test) +- [ ] Close DEFERRED.md entry "2026-05-28 — uptime-kuma create-a-monitor" +- [ ] Claim M2 gate +- [ ] Write ## DONE after M2 Adversary PASS ## Adversary findings (Adversary-owned — no items yet; populated as issues are found) diff --git a/JOURNAL-kuma.md b/JOURNAL-kuma.md new file mode 100644 index 0000000..5c3a5fd --- /dev/null +++ b/JOURNAL-kuma.md @@ -0,0 +1,61 @@ +# JOURNAL — phase `kuma` (uptime-kuma create-a-monitor functional test) + +Design rationale, investigations, and dead-ends. Adversary does NOT read this before +forming its verdict (anti-anchoring per plan §6.1). See STATUS-kuma.md for claim context. + +--- + +## 2026-06-11 — Approach selection: Playwright over python-socketio + +**Context:** The phase plan offers two choices: +- (a) python-socketio client speaking Socket.IO events directly +- (b) Playwright driving the real browser UI + +**Investigation:** Checked the cc-ci Nix Python environment: +``` +/nix/store/x188l04r3gfkh18gy1dpf05fv3kkrgs7-python3-3.12.8-env/lib/python3.12/site-packages/ +→ greenlet, playwright 1.50.0, pytest 8.3.3, pyee, packaging, pluggy, iniconfig +→ NO socketio, NO websocket-client, NO aiohttp, NO requests +``` +python-socketio would need a `nix/cc-ci.nix` addition + `nixos-rebuild switch` on cc-ci. +Playwright is already present. **Chose option (b): no Nix changes, faster to ship.** + +**Selector research:** Inspected uptime-kuma 2.2.1 source files in the Docker image: +- `src/pages/Setup.vue`: confirms `data-cy` attributes on all setup form fields +- `src/pages/EditMonitor.vue`: confirms `data-testid` on friendly-name, url, save-button +- `src/pages/Details.vue`: confirms `data-testid="monitor-status"` on status badge +- Compiled bundle `dist/assets/index-D_mnxLA0.js`: grep confirms all target attributes + +**Heartbeat "important" logic:** Checked `server/model/monitor.js` line 1420: +``` +// * ? -> ANY STATUS = important [isFirstBeat] +``` +The server marks the first heartbeat as `important=true`, so it WILL appear in the +important-heartbeat table immediately after the first probe. This means the table row +check is a reliable proof of real probe execution. + +**Status text:** From `src/mixins/socket.js` line 755 (`statusList` computed): +```javascript +text: this.$t("Up"), // UP=1 +text: this.$t("Down"), // DOWN=0 +``` +English locale: "Up" (capital U, lowercase p) and "Down". Used these exact strings in +the `_wait_for_status` assertions. + +**URL routing:** `src/router.js` uses `createWebHistory()` (history mode, not hash mode). +Routes: `/` → Entry.vue → redirects to `/dashboard`; `/add` → EditMonitor.vue; +`/dashboard/:id` → Details.vue. So `page.goto(f"{base}/add")` reliably opens the monitor +form directly. + +**Negative test choice:** `http://127.0.0.1:19999/dead`: +- Inside the container, port 19999 is unused → OS returns ECONNREFUSED instantly +- Connection-refused causes uptime-kuma to mark the monitor DOWN immediately (no timeout wait) +- This proves the probe engine makes real outbound calls (not a stub) +- Included — fits runtime budget easily (~5 s for DOWN detection) + +**Runtime budget analysis:** +- Setup wizard + login: ~10 s +- Create monitor 1 + wait UP: ~15-30 s (first probe immediate, but socket roundtrip) +- Create monitor 2 + wait DOWN: ~10 s (ECONNREFUSED is fast) +- Overhead: ~5 s +- Total estimate: ~40-55 s — well within ≤90 s target diff --git a/STATUS-kuma.md b/STATUS-kuma.md new file mode 100644 index 0000000..cbeb96f --- /dev/null +++ b/STATUS-kuma.md @@ -0,0 +1,61 @@ +# STATUS — phase `kuma` (uptime-kuma create-a-monitor functional test) + +SSOT: `cc-ci-plan/plan-phase-kuma-monitor.md` + +## Current state + +**Gate: M1 IN PROGRESS** — test implemented, pending first drone run to confirm green. + +## What is claimed + +### Approach choice (DECISIONS.md) +Playwright (option b). Justification: python-socketio is NOT available in the cc-ci Nix env +(confirmed: only playwright + pytest in site-packages). Playwright drives the real browser; +Socket.IO is handled transparently. No Nix changes needed. + +### Test file +`tests/uptime-kuma/playwright/test_monitor_wizard.py` + +### What the test does +1. Completes uptime-kuma 2.2.1 first-run setup wizard (admin create via browser). +2. Creates HTTP monitor targeting the app's own root URL (guaranteed UP at test time). +3. Waits ≤90 s for status badge (`data-testid="monitor-status"`) to show "Up". +4. Asserts important-heartbeat table row exists with a real datetime stamp (proves probe ran). +5. Creates a second monitor targeting `http://127.0.0.1:19999/dead` (dead port → connection refused). +6. Waits ≤60 s for status badge to show "Down" (negative teeth). + +### Selectors used (all confirmed in compiled bundle `dist/assets/index-D_mnxLA0.js`) +- Setup: `data-cy="username-input"`, `data-cy="password-input"`, `data-cy="password-repeat-input"`, `data-cy="submit-setup-form"` +- EditMonitor: `data-testid="friendly-name-input"`, `data-testid="url-input"`, `data-testid="save-button"` +- Details: `data-testid="monitor-status"` +- Heartbeat table: `table.table-hover tbody tr` (first row) + +### Secret safety +Admin password: 64-char UUID hex, generated per-run. Never printed, never in any assertion error message. + +### Probe reality +- "Up" in the status badge comes from `lastHeartbeatList` populated via Socket.IO heartbeat events + (socket.js mixin line 755). Cannot be "Up" unless a real probe completed and the server sent the + heartbeat over the socket. +- Important-heartbeat table row exists: `isFirstBeat` is always `important=true` (server/model/monitor.js + line 1420). Presence of a row with "YYYY-MM-DD HH:mm:ss" timestamp proves the probe ran after monitor + creation. +- Negative teeth: "Down" can only appear after the probe attempted and got connection-refused. + +### How to verify (Adversary cold-check) +```bash +# Deploy uptime-kuma against any fresh cc-ci domain, then run: +CCCI_APP_DOMAIN= RECIPE=uptime-kuma STAGES=custom \ + cc-ci-run -m pytest tests/uptime-kuma/playwright/test_monitor_wizard.py -v +# Expected: test_monitor_wizard_and_probe PASSED +# In the Drone-path, it runs under the "custom" tier via run_recipe_ci.py. +``` + +### Runtime +Local estimate: wizard ~10 s + 2× (navigate+fill+probe) ≤ ~60 s total. Within ≤90 s budget. + +### Next step +Trigger `!testme` on a uptime-kuma PR; wait for drone run to pass; then claim M1. + +## Blocked +(nothing) diff --git a/machine-docs/DECISIONS.md b/machine-docs/DECISIONS.md index 9f6c1c5..df11297 100644 --- a/machine-docs/DECISIONS.md +++ b/machine-docs/DECISIONS.md @@ -1383,3 +1383,24 @@ declaration is evidence-backed in the recipe_meta comment + upstream registry; i way to exercise a PR at all for a recipe in this state. Re-enable path documented per-recipe (bluesky: drop EXPECTED_NA + set UPGRADE_BASE_VERSION="0.3.0+v0.4.219" once merged+published). Locked by tests/unit/test_upgrade_base.py. + +## 2026-06-11 — uptime-kuma: Playwright (option b) for monitor-wizard test (phase kuma) + +**Decision:** use Playwright (option b from plan-phase-kuma-monitor.md §1) to implement +the `tests/uptime-kuma/playwright/test_monitor_wizard.py` test. + +**Why not python-socketio (option a):** python-socketio is NOT installed in the cc-ci +Nix Python environment (site-packages has playwright + pytest only; no socketio wheel). +Adding it would require modifying `nix/cc-ci.nix` and running `nixos-rebuild switch` on +cc-ci — extra Nix overhead when Playwright already handles Socket.IO transparently through +the real browser. The option (a) benefit (speed, headless) is outweighed by the absence of +the package. + +**Why Playwright works here:** uptime-kuma 2.2.1 has stable `data-cy` attributes on the +setup form and `data-testid` attributes on the monitor form + status badge — confirmed +present in the compiled bundle (`dist/assets/index-D_mnxLA0.js`). These are the canonical +Cypress/testing selectors; they do not change without an intentional test-attribute removal. +The Playwright flow is deterministic: wizard → `/add` form → `/dashboard/:id` detail page. + +**Runtime implication:** Playwright adds ~5–10 s overhead vs a headless socketio client, +but stays well within the ≤90 s budget. Acceptable. diff --git a/tests/uptime-kuma/playwright/test_monitor_wizard.py b/tests/uptime-kuma/playwright/test_monitor_wizard.py new file mode 100644 index 0000000..a25411c --- /dev/null +++ b/tests/uptime-kuma/playwright/test_monitor_wizard.py @@ -0,0 +1,167 @@ +"""uptime-kuma — §4.3 prescribed: wizard + create-a-monitor + real probe assertion. + +Phase kuma: resolves the DEFERRED.md entry "2026-05-28 — uptime-kuma create-a-monitor". + +Approach: **Playwright (option b)** — python-socketio is NOT in the cc-ci harness +environment (Nix site-packages: playwright + pytest only; no socketio wheel); Playwright +drives the real browser which handles Socket.IO automatically, is version-robust via +stable `data-cy` / `data-testid` attributes confirmed present in the 2.2.1 compiled +bundle, and reuses the existing harness browser stack (no Nix changes needed). +Justification recorded in DECISIONS.md. + +Test flow (§4.3 contract): +1. Complete first-run setup wizard (admin create) via the real browser UI. +2. Create an HTTP monitor whose target the harness controls: the app's own root URL + (`https://{live_app}/`) — guaranteed reachable since the deploy already passed its + health gate before this test tier runs. +3. Wait ≤90 s for the first heartbeat to arrive and report **Up** (uptime-kuma probes + immediately on monitor creation; typical first-result time is < 15 s). +4. Assert `data-testid="monitor-status"` text is "Up" AND the important-heartbeat table + has at least one row with a real datetime stamp (proves a probe ran, not a config echo). +5. **Negative teeth:** create a second monitor targeting `http://127.0.0.1:19999/dead` + (a port guaranteed to refuse connections inside the container → instant DOWN). Wait + ≤60 s; assert status is "Down". This proves the probe engine actually makes outbound + checks and is not a stub. Runtime budget: connection-refused completes in < 1 s, so + the DOWN detection is fast. + +Runtime budget: wizard ~10 s, two monitor create+wait cycles ≤90 s total. Well within +the ≤ ~90 s added-to-functional-tier target from the phase plan. + +Secret safety: admin password generated per-run as a 64-char UUID hex string; it is +NEVER printed, logged, or included in assertion messages. The fixture uses `_pw` (leading +underscore convention) and no error path includes its value. +""" + +from __future__ import annotations + +import os +import re +import sys +import uuid + +sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "..", "runner")) +from harness import browser as harness_browser # noqa: E402 + +# Pattern for uptime-kuma's datetime format ("YYYY-MM-DD HH:mm:ss") rendered by its +# Datetime.vue component (format confirmed from mixin source — datetimeFormat calls +# dayjs with "YYYY-MM-DD HH:mm:ss"). +_DATETIME_RE = re.compile(r"\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}") + +_SETUP_TIMEOUT_MS = 30_000 # wizard + auto-login settle +_FORM_TIMEOUT_MS = 15_000 # form element wait +_PROBE_UP_MS = 90_000 # first UP probe (typically < 15 s, but bound generously) +_PROBE_DOWN_MS = 60_000 # connection-refused DOWN (typically < 5 s) +_DETAIL_URL_MS = 20_000 # wait for /dashboard/:id after Save + + +def _wait_for_status(page, expected_text: str, timeout_ms: int) -> None: + """Poll the monitor-status badge until its text matches expected_text. + + Uses page.wait_for_function so the wait is driven by Playwright's internal event loop + (no sleep), bounded by timeout_ms. Raises on timeout. + """ + escaped = expected_text.replace('"', '\\"') + page.wait_for_function( + f'() => {{' + f' const el = document.querySelector(\'[data-testid="monitor-status"]\');' + f' return el && el.textContent.trim() === "{escaped}";' + f'}}', + timeout=timeout_ms, + ) + + +def _fill_monitor_form(page, name: str, url: str) -> None: + """Fill the EditMonitor form (at /add) and click Save. + + Waits for the friendly-name input to be ready (signals the form rendered). + Uses stable data-testid attributes confirmed in the 2.2.1 compiled bundle. + """ + page.wait_for_selector('[data-testid="friendly-name-input"]', timeout=_FORM_TIMEOUT_MS) + page.locator('[data-testid="friendly-name-input"]').fill(name) + # url-input: HTTP(s) type is the default; the URL field (id="url") has data-testid="url-input" + page.locator('[data-testid="url-input"]').first.fill(url) + page.locator('[data-testid="save-button"]').click() + + +def test_monitor_wizard_and_probe(live_app): + """Setup wizard → create self-probe monitor (→ Up) + dead-port monitor (→ Down).""" + from playwright.sync_api import sync_playwright + + # Per-run admin credentials. Password is 64 hex chars and NEVER logged. + _admin_user = "admin" + _pw = uuid.uuid4().hex + uuid.uuid4().hex # noqa: S311 — not a crypto secret, test-only + + base = f"https://{live_app}" + + with sync_playwright() as p: + browser = p.chromium.launch(args=["--no-sandbox"]) + try: + ctx = browser.new_context(ignore_https_errors=True) + page = ctx.new_page() + + # ── 1. Setup wizard ────────────────────────────────────────────────── + # Navigate to root; uptime-kuma 2.x redirects to /setup on first boot. + harness_browser.goto_with_retry( + page, + f"{base}/", + accept_statuses=(200,), + wait_until="domcontentloaded", + deadline_seconds=60, + ) + # Wait for the setup form (data-cy confirmed in 2.2.1 bundle). + page.wait_for_selector('[data-cy="username-input"]', timeout=_SETUP_TIMEOUT_MS) + page.locator('[data-cy="username-input"]').fill(_admin_user) + page.locator('[data-cy="password-input"]').fill(_pw) + page.locator('[data-cy="password-repeat-input"]').fill(_pw) + page.locator('[data-cy="submit-setup-form"]').click() + + # After Create: uptime-kuma auto-logs in and pushes router to "/" → Entry.vue + # redirects to /dashboard. Wait for the dashboard URL to settle. + page.wait_for_url(f"{base}/dashboard", timeout=_SETUP_TIMEOUT_MS) + + # ── 2. Create monitor: self-probe (target = app root → expect Up) ── + page.goto(f"{base}/add", wait_until="domcontentloaded") + _fill_monitor_form(page, "kuma-self-probe", f"{base}/") + # After Save, router navigates to /dashboard/:id (monitor detail page). + page.wait_for_url(f"{base}/dashboard/**", timeout=_DETAIL_URL_MS) + + # ── 3. Wait for Up status (first probe fires immediately on creation) ── + _wait_for_status(page, "Up", _PROBE_UP_MS) + + # ── 4. Verify a real heartbeat record exists (proves a probe ran) ──── + # The important-heartbeat table (Details.vue) shows all beats where + # `important=true`; the server marks the FIRST beat as important + # (isFirstBeat logic, server/model/monitor.js line 1420). So after the + # first probe completes there will be exactly one row with a datetime stamp. + heartbeat_rows = page.locator("table.table-hover tbody tr") + heartbeat_rows.first.wait_for(timeout=_FORM_TIMEOUT_MS) + first_row_text = heartbeat_rows.first.text_content(timeout=5_000) or "" + assert "No important events" not in first_row_text, ( + "Heartbeat table shows 'No important events' — the self-probe may not " + "have recorded a heartbeat yet" + ) + # The row must contain a real datetime stamp (format: "YYYY-MM-DD HH:mm:ss"). + assert _DATETIME_RE.search(first_row_text), ( + f"Heartbeat row has no datetime matching YYYY-MM-DD HH:mm:ss; " + f"row text: {first_row_text!r}" + ) + + # ── 5. Negative teeth: dead-port monitor (→ Down) ─────────────────── + # http://127.0.0.1:19999/dead — port 19999 is unused inside the container; + # the OS returns connection-refused instantly (<1 s), so DOWN arrives fast. + page.goto(f"{base}/add", wait_until="domcontentloaded") + _fill_monitor_form(page, "kuma-dead-probe", "http://127.0.0.1:19999/dead") + page.wait_for_url(f"{base}/dashboard/**", timeout=_DETAIL_URL_MS) + + _wait_for_status(page, "Down", _PROBE_DOWN_MS) + + # Confirm the dead monitor shows Down (belt-and-suspenders). + status_text = ( + page.locator('[data-testid="monitor-status"]').text_content(timeout=5_000) or "" + ).strip() + assert status_text == "Down", ( + f"Dead-port monitor expected 'Down', got {status_text!r}" + ) + + finally: + browser.close()