From 6c91373357d6aa9f4f152e6ce056d5e9ae184c09 Mon Sep 17 00:00:00 2001 From: autonomic-bot Date: Tue, 11 Aug 2026 14:41:10 +0000 Subject: [PATCH] skills: point every test-editing path at tests/STYLE.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit /recipe-upgrade --with-tests, /ci-test-review and /cc-ci-tests-update all author test changes, and all three had only 'never weaken a test' as guidance. That did not stop the plausible failure: the fixture INSERTed rows into the app's database, which was correct for v2 and silently wrong for v3, where a site must belong to a team. Events were acked 202 and discarded; the recipe sat RED for six weeks. The rule that would have prevented it — set state up through the app's own interface, not its database — now lives in tests/STYLE.md in the cc-ci repo, and each of the three paths is told to read it before editing a test. --- .claude/skills/ci-test-review/SKILL.md | 4 ++++ .claude/skills/recipe-upgrade/SKILL.md | 6 ++++++ .opencode/skills/cc-ci-tests-update/SKILL.md | 8 ++++++++ 3 files changed, 18 insertions(+) diff --git a/.claude/skills/ci-test-review/SKILL.md b/.claude/skills/ci-test-review/SKILL.md index 0e7c54a..de06743 100644 --- a/.claude/skills/ci-test-review/SKILL.md +++ b/.claude/skills/ci-test-review/SKILL.md @@ -79,6 +79,10 @@ For each real (non-flaky) finding, write the actual fix and open a PR. **Never m it handles the mirror to `git.autonomic.zone/recipe-maintainers/` (upstream `git.coopcloud.tech`). Keep the change **bounded** to the diagnosed root cause; don't rewrite the recipe. +- **Before editing any test, read `tests/STYLE.md` in the cc-ci repo.** It encodes the rules a test + change must satisfy — set state up through the app's interface rather than its database, gate on + version instead of branching, correct the fixture/wait but NEVER the assertion, and diagnose from + the app's own telemetry before concluding a test is stale. - **CI-server-side fix → cc-ci PR.** Branch the cc-ci product repo (`recipe-maintainers/cc-ci`), apply the fix, and open the PR via the Gitea API (use the `GITEA_*` creds from `/srv/cc-ci/.testenv`). **Single-writer discipline:** work on a dedicated diff --git a/.claude/skills/recipe-upgrade/SKILL.md b/.claude/skills/recipe-upgrade/SKILL.md index 9d3f0cf..785b678 100644 --- a/.claude/skills/recipe-upgrade/SKILL.md +++ b/.claude/skills/recipe-upgrade/SKILL.md @@ -312,6 +312,12 @@ test change, and a test change is **gated by `--with-tests`**: Do **NOT** modify any test. Report `SUCCESS-PENDING-TESTS` (recipe PR open; `!testme` red on a stale test; operator to decide). - **`--with-tests` — open + verify a cc-ci test PR.** Make it the `ci-test-review` way: + 0. **READ `tests/STYLE.md` in the cc-ci repo FIRST.** It is the rulebook for changing a test, and + it is written against the failures this pipeline has actually produced. The two that matter most + here: **set state up through the app's own interface, never its database** (a plausible fixture + that INSERTed rows passed on v2 and silently broke on v3, holding the recipe RED for six weeks), + and **gate on version rather than writing a fixture that supports both** — old-version tests can + simply be deleted, since the older version is only exercised through the upgrade tier. 1. Branch `recipe-maintainers/cc-ci` in a **separate clone** (single-writer: never push `main`, never touch the build loops' `/cc-ci` `/cc-ci-adv` clones); update the test/overlay. 2. **Verify the recipe upgrade WITH the updated test applied.** `!testme` on the recipe PR uses the diff --git a/.opencode/skills/cc-ci-tests-update/SKILL.md b/.opencode/skills/cc-ci-tests-update/SKILL.md index cd0a211..d94bbcb 100644 --- a/.opencode/skills/cc-ci-tests-update/SKILL.md +++ b/.opencode/skills/cc-ci-tests-update/SKILL.md @@ -85,6 +85,14 @@ failure (AI — this is the `ci-test-review` step-3 diagnosis): ### 2. For each stale test — author the minimal test update (AI; never weaken) +> **Read `tests/STYLE.md` in the cc-ci repo before writing the update.** It is the rulebook for test +> changes, written from failures this pipeline actually produced. Most load-bearing: set state up +> through the app's **own interface, never its database** (a plausible fixture that INSERTed rows +> passed on v2 and silently broke on v3 — 202 acks, rows in postgres, nothing ingested — and held the +> recipe RED for six weeks), **gate on version rather than supporting both** (old-version tests can be +> deleted; the older version is only exercised via the upgrade tier), and correct the fixture or the +> wait but **never the assertion**. + Work on **one recipe at a time** (serialize — each verification deploys a recipe on the shared Swarm). For each `STALE_TESTS` entry: