plan: regression canaries are milestone-cadence (polish/review/release), not per-commit

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
autonomic-bot
2026-06-02 00:30:09 +00:00
parent 947e7f55b9
commit fd86baea2a

View File

@ -61,10 +61,17 @@ For each good canary the test drives the real cold full suite and asserts the *o
- **Determinism:** pin canary recipe versions/SHAs and the bad-fixture SHA. Record run artifacts
(build URL, logs) on failure for triage.
## How it runs (dogfood)
## How it runs — CADENCE POLICY (important)
**Do NOT run these canaries on every commit/PR.** They are slow + resource-heavy (full lifecycle on
lasuite-docs is minutes, needs the live server/abra/Swarm). Run them **deliberately at milestones**:
- **polishing passes**, **code reviews**, and **releases** of the cc-ci server — i.e. before we trust a
batch of server changes, not on each incremental commit.
- On-demand: `pytest -m canary` against the live cc-ci server (from the orchestrator or the host).
- Dogfood: cc-ci is itself enrolled in `!testme`, so a server-change PR on the cc-ci repo can trigger
the canary run automatically — a server regression shows up as a RED check on the very PR that caused it.
- They are explicitly **opt-in** (the `@pytest.mark.canary` marker keeps them out of any fast/default
run). If wired to `!testme` on the cc-ci repo, gate it behind a deliberate trigger (e.g. a
`run-canaries` label or a `!testme --canary`), **not** an automatic run on every cc-ci PR.
- Document this cadence in `tests/regression/README.md` so future contributors don't wire it into the
per-commit path.
## Definition of Done (Adversary-verified)
1. `tests/regression/` pytest suite exists and is committed (cc-ci repo PR).