feat(harness): intentional skips + custom-html-tiny functional test; 4-rung ladder (#6)
Some checks failed
continuous-integration/drone/push Build is failing
Some checks failed
continuous-integration/drone/push Build is failing
Declare intentional skips + custom-html-tiny functional test; 4-rung level ladder
- recipe_meta.EXPECTED_NA = {rung: reason} lists intentionally-skipped rungs; any
essential rung skipped and not listed is unintentional. Skips still cap the level
(never inflate). results.json: skips:{intentional,unintentional} + level_cap_rung.
- Level ladder = the four essential rungs (install, upgrade, backup/restore,
functional; top = L4). integration & recipe-local are optional, not leveled
(SSO still enforced for the run verdict, unchanged).
- Card shows skipped rungs as INTENTIONAL SKIP (green, reason below) / UNINTENTIONAL
SKIP (amber); level badge gains an expected/gap? third segment.
- custom-html-tiny: functional serve test (exact-byte round-trip + 404); declares
backup_restore intentionally skipped (stateless static server).
Independently verified by the adversary: 138 unit tests pass cold; live full-stage
run on custom-html-tiny green (upgrade tier ran; level 2; correct skips/badge);
clean teardown.
This commit is contained in:
87
tests/custom-html-tiny/functional/test_serves_content.py
Normal file
87
tests/custom-html-tiny/functional/test_serves_content.py
Normal file
@ -0,0 +1,87 @@
|
||||
"""custom-html-tiny — recipe-specific functional test (static-web-server).
|
||||
|
||||
Proves the deployed static-web-server is *actually serving files from its `content` volume* with real
|
||||
file-server semantics, not merely returning 200 from a Traefik fallback or a generic stub:
|
||||
|
||||
1. exact-byte round-trip — write a uniquely-named file with random content into the served volume,
|
||||
fetch it over HTTPS, and assert the bytes come back verbatim. Non-vacuous: the content is random
|
||||
per run, so only a server that reads this file off the volume can pass.
|
||||
2. real 404 — a random non-existent path returns 404, proving directory/file semantics (a
|
||||
200-everything stub or mis-routed host would not 404).
|
||||
|
||||
The recipe's image (joseluisq/static-web-server) is shell-less (scratch-based) and its content volume
|
||||
is seeded via the install_steps.sh host-mountpoint mechanism — so this test writes its probe file the
|
||||
same way (resolve the swarm volume's mountpoint with `docker volume inspect`, write directly) rather
|
||||
than `docker exec`-ing in a container that has no shell.
|
||||
|
||||
Runs in the custom tier against the shared post-install deployment (the `live_app` fixture is its
|
||||
per-run domain). Mirrors install_steps.sh: the app's content volume is named `<stack>_content`, where
|
||||
`stack` is the domain with dots replaced by underscores; HTTP_SUBDIR is empty, so the volume root is
|
||||
served at `/`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import os
|
||||
import ssl
|
||||
import subprocess
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import uuid
|
||||
|
||||
|
||||
def _served_dir(domain: str) -> str:
|
||||
"""Host mountpoint of the app's served `content` volume (same naming as install_steps.sh)."""
|
||||
vol = f"{domain.replace('.', '_')}_content"
|
||||
out = subprocess.run(
|
||||
["docker", "volume", "inspect", vol, "--format", "{{.Mountpoint}}"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
mountpoint = out.stdout.strip()
|
||||
assert mountpoint, f"could not resolve mountpoint for volume {vol!r}"
|
||||
return mountpoint
|
||||
|
||||
|
||||
def _get(url: str) -> tuple[int, bytes]:
|
||||
"""GET the URL; return (status, body). A 4xx/5xx is returned, not raised (we assert on the code).
|
||||
TLS verification is relaxed: the served wildcard cert is validated separately by the infra check;
|
||||
here we care only about the app's response."""
|
||||
ctx = ssl.create_default_context()
|
||||
ctx.check_hostname = False
|
||||
ctx.verify_mode = ssl.CERT_NONE
|
||||
try:
|
||||
with urllib.request.urlopen(url, timeout=20, context=ctx) as resp:
|
||||
return resp.status, resp.read()
|
||||
except urllib.error.HTTPError as e:
|
||||
return e.code, e.read()
|
||||
|
||||
|
||||
def test_static_file_roundtrip_and_404(live_app):
|
||||
"""Write a random file into the served volume → fetch it → bytes match; and a missing path 404s."""
|
||||
served = _served_dir(live_app)
|
||||
token = uuid.uuid4().hex
|
||||
name = f"ccci-probe-{token}.txt"
|
||||
body = f"cc-ci-functional-{token}\n".encode()
|
||||
path = os.path.join(served, name)
|
||||
with open(path, "wb") as fh:
|
||||
fh.write(body)
|
||||
try:
|
||||
status, got = _get(f"https://{live_app}/{name}")
|
||||
assert status == 200, f"served probe file returned {status} (expected 200)"
|
||||
assert got == body, (
|
||||
f"content round-trip mismatch: served {got!r}, wrote {body!r} "
|
||||
"(static-web-server not serving the content volume?)"
|
||||
)
|
||||
|
||||
# A random non-existent path must 404 — proves real static-file semantics, distinguishing a
|
||||
# working server from a 200-everything stub or a mis-routed Traefik fallback.
|
||||
miss_status, _ = _get(f"https://{live_app}/ccci-missing-{uuid.uuid4().hex}.txt")
|
||||
assert miss_status == 404, (
|
||||
f"missing path returned {miss_status} (expected 404 — generic 200-returner / mis-route?)"
|
||||
)
|
||||
finally:
|
||||
with contextlib.suppress(OSError):
|
||||
os.remove(path)
|
||||
Reference in New Issue
Block a user