diff --git a/cc-ci-plan/plan-server-regression-canaries.md b/cc-ci-plan/plan-server-regression-canaries.md index a8a96e3..5ff2cdc 100644 --- a/cc-ci-plan/plan-server-regression-canaries.md +++ b/cc-ci-plan/plan-server-regression-canaries.md @@ -61,10 +61,17 @@ For each good canary the test drives the real cold full suite and asserts the *o - **Determinism:** pin canary recipe versions/SHAs and the bad-fixture SHA. Record run artifacts (build URL, logs) on failure for triage. -## How it runs (dogfood) +## How it runs — CADENCE POLICY (important) +**Do NOT run these canaries on every commit/PR.** They are slow + resource-heavy (full lifecycle on +lasuite-docs is minutes, needs the live server/abra/Swarm). Run them **deliberately at milestones**: +- **polishing passes**, **code reviews**, and **releases** of the cc-ci server — i.e. before we trust a + batch of server changes, not on each incremental commit. - On-demand: `pytest -m canary` against the live cc-ci server (from the orchestrator or the host). -- Dogfood: cc-ci is itself enrolled in `!testme`, so a server-change PR on the cc-ci repo can trigger - the canary run automatically — a server regression shows up as a RED check on the very PR that caused it. +- They are explicitly **opt-in** (the `@pytest.mark.canary` marker keeps them out of any fast/default + run). If wired to `!testme` on the cc-ci repo, gate it behind a deliberate trigger (e.g. a + `run-canaries` label or a `!testme --canary`), **not** an automatic run on every cc-ci PR. +- Document this cadence in `tests/regression/README.md` so future contributors don't wire it into the + per-commit path. ## Definition of Done (Adversary-verified) 1. `tests/regression/` pytest suite exists and is committed (cc-ci repo PR).