Files
cc-ci/runner/harness/screenshot.py
T
autonomic-bot 934ec8b8f0
continuous-integration/drone/push Build is passing
runner: never embed a fossil screenshot from a collided run dir
!testme gitea PR #10 (drone build 35, 2026-10-05) rendered a summary card whose
app screenshot was custom-html-tiny's — a fossil from a May-31 run dir with the
same number. Root cause chain:
- /var/lib/cc-ci-runs/ keeps old-era run dirs numbered up to 1377; drone's
  build counter restarted when its DB was re-created (2026-09-27), so new
  builds collide with old dirs (35 was one).
- the run reused the collided dir without cleaning it: fresh results.json/
  summary.* landed NEXT TO the old screenshot.png.
- screenshot.capture()'s SCREENSHOT-hook branch skipped the actual snap when
  out_path already existed ('the hook may have saved it' — no hook does), so
  capture() 'succeeded' without writing and the card embedded the fossil.

Fix, cosmetics-only (R7 — verdict logic untouched):
- results.fresh_artifact_dir(): at run start, remove ONLY the files/dirs this
  run owns (results.json, summary.*, badge.svg, lint.txt, screenshot.png,
  junit/) from its artifact dir; anything else is left alone; never raises.
- run_recipe_ci calls it right after computing run_artifact_dir.
- screenshot.capture(): the hook branch ALWAYS snaps (same settle/blank-retry
  path as the default branch) — a pre-existing file is never trusted.

Unit tests: fresh_artifact_dir (owned-only removal, dir creation, odd entries)
+ capture() fake-playwright regression proving a fossil out_path is overwritten.
2026-10-05 17:37:43 +00:00

175 lines
9.0 KiB
Python

"""Phase 3 — app screenshot capture (plan-phase3-results-ux.md §4.2, R4/U1).
Captures a real screenshot of the deployed app while it is up (before teardown), reusing the Phase-1
Playwright browser already in the harness — no new heavy dep. The PNG is embedded in the summary
card (R3) and the dashboard (R5).
Secret-safety (R7, the cardinal screenshot guardrail): the screenshot step must NEVER capture a page
that displays generated credentials (an install wizard showing the initial admin password, a secrets
page, etc.). The DEFAULT capture is the app's **landing page** (a login form shows fields, not the
password) — safe for every recipe. A recipe that needs a post-login view opts in via a recipe-meta
`SCREENSHOT` hook: a callable `SCREENSHOT(page, ctx) -> None` that drives Playwright to a
safe, credential-free view and is responsible for not landing on a secrets page. The harness never
auto-fills a wizard.
Robustness (R7, cosmetics never block): every entry point is best-effort — any failure (Playwright
missing, app slow, navigation error) is swallowed and returns None so the run/verdict is unaffected.
"""
from __future__ import annotations
import contextlib
import os
from . import browser as harness_browser
from . import meta as meta_mod
# Default viewport for the captured screenshot — a desktop-ish frame that crops well into the card.
VIEWPORT = {"width": 1280, "height": 800}
# Hard cap so a wedged app can never hang the run on the screenshot step (R7 / Phase-1 timeouts).
NAV_DEADLINE_S = 45
# ---- post-navigation settle (phase-shot fix, 2026-06-11) ----
# SPAs (immich, n8n, cryptpad, the keycloak admin console, lasuite-*, mumble-web, mattermost) fire
# `domcontentloaded` on their empty HTML shell and only paint after the JS bundle loads — snapping
# immediately produced solid blank frames (byte-stable 4801-2 B) or loading spinners. After nav,
# wait for network-idle up to SETTLE_TIMEOUT_MS (apps that never go idle — continuous polling —
# simply spend the cap; bounded, never raises), then RENDER_GRACE_MS for the final paint.
SETTLE_TIMEOUT_MS = 10_000
RENDER_GRACE_MS = 500
# A 1280x800 PNG below this is near-certainly a solid frame or a bare loading spinner (phase-shot
# audit: blank frames were 4801-2 B across three different apps, lone spinners 5.9-8.8 KB; the
# smallest real page was 12950 B). One bounded retry with an extra settle, then keep what we get —
# an honest late frame beats none, and the retry only ever replaces a tiny frame with a later one.
BLANK_SIZE_BYTES = 10_000
BLANK_RETRY_SETTLE_MS = 4_000
# Wait-budget arithmetic (plan-phase-shot §3 P3: step worst case ≤ ~60s): NAV_DEADLINE_S (45s,
# spent only while the app isn't serving yet) + SETTLE_TIMEOUT_MS + RENDER_GRACE_MS +
# BLANK_RETRY_SETTLE_MS + RENDER_GRACE_MS = 60s of bounded waiting; tested in unit tests.
def _settle(page, idle_timeout_ms: int) -> None:
"""Best-effort bounded settle: network-idle up to the cap, then a short render grace.
Never raises (R7) — a timeout just means the page kept polling; we snap what's painted."""
# cosmetic path (R7): a timeout on a never-idle app is expected — the cap IS the wait
with contextlib.suppress(Exception):
page.wait_for_load_state("networkidle", timeout=idle_timeout_ms)
with contextlib.suppress(Exception):
page.wait_for_timeout(RENDER_GRACE_MS)
def settle(page, idle_timeout_ms: int = SETTLE_TIMEOUT_MS) -> None:
"""Public settle for recipe SCREENSHOT hooks: after the hook navigates to its safe view, call
this so the snap happens post-paint. Same bounded best-effort contract as the default path."""
_settle(page, idle_timeout_ms)
def _snap_with_blank_retry(page, out_path: str) -> None:
"""Screenshot the page; if the PNG is blank/spinner-sized, retry ONCE after a longer settle.
The retry is snapped to a temp path and kept only if it is >= the first frame's size — later
is usually more painted, but a page can also regress (redirect, error overlay) and a worse
frame must never overwrite a better one (adversary finding A1)."""
page.screenshot(path=out_path, full_page=False)
try:
first = os.path.getsize(out_path)
except OSError:
return
if first >= BLANK_SIZE_BYTES:
return
print(
f" screenshot: frame looks blank/loading ({first} B < {BLANK_SIZE_BYTES} B) — "
"one retry after a longer settle",
flush=True,
)
_settle(page, BLANK_RETRY_SETTLE_MS)
retry_path = out_path + ".retry"
try:
page.screenshot(path=retry_path, full_page=False)
retry = os.path.getsize(retry_path)
if retry >= first:
os.replace(retry_path, out_path)
print(f" screenshot: retry frame kept ({retry} B >= {first} B)", flush=True)
else:
os.remove(retry_path)
print(f" screenshot: retry frame discarded ({retry} B < {first} B)", flush=True)
finally:
with contextlib.suppress(OSError):
os.remove(retry_path)
def screenshot_path(run_artifact_dir: str) -> str:
"""Canonical on-disk path for a run's app screenshot (pure)."""
return os.path.join(run_artifact_dir, "screenshot.png")
def _load_screenshot_hook(recipe_meta):
"""Return the recipe's optional SCREENSHOT hook (a callable) if it declared one, else None.
The hook drives Playwright to a safe post-login view; default is the landing page.
`recipe_meta` is the loaded RecipeMeta (rcust P1 — the single loader actually delivers
SCREENSHOT now; under the old L1 allowlist the key never arrived, spec §8 R2). A plain dict
is still accepted for direct/manual callers."""
if recipe_meta is None:
return None
if isinstance(recipe_meta, dict):
hook = recipe_meta.get("SCREENSHOT")
else:
hook = getattr(recipe_meta, "SCREENSHOT", None)
return hook if callable(hook) else None
def capture(domain: str, out_path: str, *, recipe_meta: dict | None = None) -> str | None:
"""Capture a screenshot of the live app at https://<domain>/ into out_path.
Default: navigate to the landing page and screenshot it (credential-free, safe for any recipe).
If the recipe declared a SCREENSHOT hook in recipe_meta, run it instead (post-login / app-specific
view, recipe-responsible for avoiding secret pages). Returns out_path on success, else None
(best-effort — never raises into the run; cosmetics never block, R7)."""
try:
from playwright.sync_api import sync_playwright
except ImportError: # pragma: no cover — playwright is always present in cc-ci-run
print(" screenshot: playwright unavailable — skipping (verdict unaffected)", flush=True)
return None
os.makedirs(os.path.dirname(out_path) or ".", exist_ok=True)
url = f"https://{domain}/"
hook = _load_screenshot_hook(recipe_meta)
try:
with sync_playwright() as p:
browser = p.chromium.launch(args=["--no-sandbox"])
try:
context = browser.new_context(ignore_https_errors=True, viewport=VIEWPORT)
page = context.new_page()
if hook is not None:
# Recipe-specific safe view (post-login etc.). The hook owns navigation +
# the no-secret-page guarantee; we ALWAYS snap the page after it returns.
# Never trust a pre-existing out_path: a recycled run artifact dir (drone
# build numbers restarted 2026-09-27 while old-era dirs linger) must not get
# its fossil screenshot embedded (observed 2026-10-05, !testme gitea #10
# build 35). SCREENSHOT(page, ctx) — the uniform ctx convention (rcust P3).
hook(page, meta_mod.hook_ctx(domain, recipe_meta))
_snap_with_blank_retry(page, out_path)
else:
# Default: landing page. Accept any rendered status (200 or an auth redirect to a
# login form) — both are credential-free and representative of "the app is up".
harness_browser.goto_with_retry(
page,
url,
accept_statuses=(200, 301, 302, 303, 401, 403),
deadline_seconds=NAV_DEADLINE_S,
wait_until="domcontentloaded",
)
# SPA paint race fix (phase-shot): settle before snapping, retry a blank frame.
_settle(page, SETTLE_TIMEOUT_MS)
_snap_with_blank_retry(page, out_path)
finally:
browser.close()
if os.path.exists(out_path) and os.path.getsize(out_path) > 0:
print(f" screenshot: captured {out_path}", flush=True)
return out_path
print(" screenshot: produced no file — skipping (verdict unaffected)", flush=True)
return None
except Exception as e: # noqa: BLE001 — screenshot is cosmetic; never fail/hang a run (R7)
print(f" screenshot: capture failed (non-fatal, verdict unaffected): {e}", flush=True)
return None