test(plausible): provision the site through the app, not SQL; add tests/STYLE.md
continuous-integration/drone/push Build is failing

The first cut of this fix added a team INSERT next to the existing sites INSERT.
That fixed the symptom and kept the cause: a fixture that knows the apps table
layout breaks whenever the app changes it, which is exactly what happened here.

_register_site now calls Plausible.Sites.create/2 through the app release console,
so the app provisions whatever its data model currently requires - including the
team it introduced in v3. Verified against BOTH versions on cc-ci: the identical
expression works on v2.0.0, which has no teams table at all, and on v3.2.1. No
version gate is needed because the fixture no longer depends on the schema.

The HTTP provisioning API (POST /api/v1/sites) would have been first choice, but
it is gated behind a paid plan and answers :upgrade_required on CE. That is
recorded in the code so the next person does not re-derive it.

tests/STYLE.md writes the rule down, along with the others this failure exercised:
gate on version rather than supporting both schemas (old-version tests can just be
deleted - the older version is only exercised through the upgrade tier); correct
the fixture or the wait but never the assertion; assert stored state rather than a
202 ack; size waits from the recipes declared readiness; and read the apps own
telemetry before deciding a test is stale.

Full cold suite against the recipe PR head: level 5 of 5, GREEN.
This commit is contained in:
cc-ci
2026-08-11 14:50:41 +00:00
parent eb1d6d9161
commit eecc4aaa51
2 changed files with 173 additions and 40 deletions
+57 -40
View File
@@ -14,10 +14,11 @@ Both assert real app state (the event reached the analytics store), not just the
plausible only ingests events for *known* sites — the in-memory `sites_cache` gates ingestion and
drops events for unregistered domains (empirically confirmed: an event for an unregistered domain
never appears in events_v2). From v3 (community-edition) a site is only "known" once it belongs to a
TEAM; a teamless site is dropped as `dropped_not_found` while the POST still acks 202, so the failure
looks like a silent ingestion stall. `_register_site` therefore provisions a team as well when the
schema has one. So each test first registers a site row in the metadata postgres, then
never appears in events_v2). Sites are therefore provisioned through plausible's OWN creation path
rather than by writing rows — under v3 a site must belong to a TEAM, and a teamless site is dropped as
`dropped_not_found` while the POST still acks 202, which reads as a silent ingestion stall. Letting the
app create the site sidesteps that entirely, and works unchanged on v2. So each test first provisions
the site, then
POSTs repeatedly while polling ClickHouse: the sites_cache must refresh to admit the new site and the
event write-buffer must flush to ClickHouse, so the first landing is not instantaneous. Re-POSTing the
same event is safe — we assert the row count is >= 1.
@@ -43,6 +44,10 @@ _UA = (
"(KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
)
# Identity the harness provisions sites under. Ephemeral per-run deploy, never a real account.
_HARNESS_EMAIL = "cc-ci@ci.invalid"
_HARNESS_PW = "ccci-harness-passphrase-2026"
def _ch(domain: str, sql: str) -> str:
"""Run a ClickHouse query against the `plausible_events_db` service; return stdout (stripped)."""
@@ -53,47 +58,59 @@ def _ch(domain: str, sql: str) -> str:
).strip()
# plausible's own provisioning path. Creating a site through the app (rather than INSERTing rows)
# means the app applies whatever its current data model requires — which is what makes this work
# unchanged across the v2→v3 jump, where sites gained a mandatory owning TEAM. Verified on cc-ci
# against BOTH v2.0.0 (no `teams` table at all) and v3.2.1: identical expression, site usable, events
# ingested. See tests/STYLE.md.
#
# The HTTP provisioning API (`POST /api/v1/sites`) would be the first choice, but it is gated behind
# a paid plan — on CE it answers `:upgrade_required` — so the app's release console is the closest
# public interface available here.
_PROVISION_SITE_EXS = """
pw = "__PW__"
email = "__EMAIL__"
user =
case Plausible.Auth.find_user_by(email: email) do
nil ->
{:ok, u} =
Plausible.Auth.User.new(%{name: "cc-ci", email: email, password: pw, password_confirmation: pw})
|> Plausible.Repo.insert()
u
u -> u
end
site = "__SITE__"
result =
case Plausible.Sites.get_by_domain(site) do
nil -> Plausible.Sites.create(user, %{"domain" => site, "timezone" => "UTC"})
s -> {:ok, s}
end
case result do
{:ok, _} -> IO.puts("CCCI_SITE_OK " <> site)
other -> IO.puts("CCCI_SITE_ERR " <> inspect(other))
end
"""
def _register_site(domain: str, site: str) -> None:
"""Register `site` in the metadata postgres so plausible will ingest events for it.
"""Provision `site` via plausible's own site-creation path, so it is a site the app will ingest for.
Idempotent. Works against BOTH schema generations, because the upgrade tier deploys an older
base version before upgrading:
Idempotent: an existing domain is reused rather than re-created.
* v2 (`plausible/analytics`) — a row in `sites` is sufficient.
* v3 (`ghcr.io/plausible/community-edition`) — sites belong to a TEAM, and ingestion drops
events for a site whose team is missing. The POST still acks 202 and the row still exists in
postgres, so the only visible symptom is that nothing ever reaches ClickHouse; the reason is
recorded in ClickHouse's own `ingest_counters` as `dropped_not_found`. Verified on cc-ci
against v3.2.1: identical site row, no team → `dropped_not_found`; with a team linked →
`buffered` and the row appears in `events_v2`.
The team block is guarded on the schema actually having teams, so this stays a no-op on v2
rather than branching on a version string.
Do NOT reach into postgres to do this. A `sites` INSERT was enough under v2, but v3 requires the
site to belong to a TEAM and silently discards events for a teamless site — `POST /api/event`
still acks 202 and the row still exists, so the only symptom is that nothing reaches ClickHouse
(ClickHouse's own `ingest_counters` records it as `dropped_not_found`). That is what put this
recipe RED on build 1224. Going through the app removes the whole class of problem: it provisions
the team itself.
"""
sql = (
"INSERT INTO sites (domain, timezone, inserted_at, updated_at, native_stats_start_at) "
f"VALUES ('{site}','UTC', now(), now(), now()) ON CONFLICT (domain) DO NOTHING; "
"DO $ccci$ "
"BEGIN "
" IF EXISTS (SELECT 1 FROM information_schema.tables WHERE table_name = 'teams') "
" AND EXISTS (SELECT 1 FROM information_schema.columns "
" WHERE table_name = 'sites' AND column_name = 'team_id') THEN "
" INSERT INTO teams (name, inserted_at, updated_at, accept_traffic_until, setup_complete) "
" SELECT 'cc-ci', now(), now(), now() + interval '365 days', true "
" WHERE NOT EXISTS (SELECT 1 FROM teams WHERE name = 'cc-ci'); "
" UPDATE sites "
" SET team_id = COALESCE(team_id, (SELECT id FROM teams WHERE name = 'cc-ci' LIMIT 1)), "
" accept_traffic_until = COALESCE(accept_traffic_until, now() + interval '365 days') "
f" WHERE domain = '{site}'; "
" END IF; "
"END "
"$ccci$; "
f"SELECT domain FROM sites WHERE domain = '{site}';"
exs = (
_PROVISION_SITE_EXS.replace("__PW__", _HARNESS_PW)
.replace("__EMAIL__", _HARNESS_EMAIL)
.replace("__SITE__", site)
)
out = lifecycle.exec_in_app(
domain, ["psql", "-q", "-U", "plausible", "-d", "plausible", "-tAc", sql], service="db"
).strip()
assert out == site, f"site {site!r} not registered in postgres (got {out!r})"
out = lifecycle.exec_in_app(domain, ["/app/bin/plausible", "rpc", exs], service="app")
assert f"CCCI_SITE_OK {site}" in out, f"could not provision site {site!r} via the app: {out.strip()[-400:]}"
def _post_event(base_domain: str, site: str, name: str, pathname: str) -> int: