skills: point every test-editing path at tests/STYLE.md

/recipe-upgrade --with-tests, /ci-test-review and /cc-ci-tests-update all author
test changes, and all three had only 'never weaken a test' as guidance. That did
not stop the plausible failure: the fixture INSERTed rows into the app's database,
which was correct for v2 and silently wrong for v3, where a site must belong to a
team. Events were acked 202 and discarded; the recipe sat RED for six weeks.

The rule that would have prevented it — set state up through the app's own
interface, not its database — now lives in tests/STYLE.md in the cc-ci repo, and
each of the three paths is told to read it before editing a test.
This commit is contained in:
autonomic-bot
2026-08-11 14:41:10 +00:00
parent 1db85a7e77
commit 6c91373357
3 changed files with 18 additions and 0 deletions
+4
View File
@@ -79,6 +79,10 @@ For each real (non-flaky) finding, write the actual fix and open a PR. **Never m
it handles the mirror to `git.autonomic.zone/recipe-maintainers/<recipe>` (upstream
`git.coopcloud.tech`). Keep the change **bounded** to the diagnosed root cause; don't rewrite the
recipe.
- **Before editing any test, read `tests/STYLE.md` in the cc-ci repo.** It encodes the rules a test
change must satisfy — set state up through the app's interface rather than its database, gate on
version instead of branching, correct the fixture/wait but NEVER the assertion, and diagnose from
the app's own telemetry before concluding a test is stale.
- **CI-server-side fix → cc-ci PR.** Branch the cc-ci product repo
(`recipe-maintainers/cc-ci`), apply the fix, and open the PR via the Gitea API (use the
`GITEA_*` creds from `/srv/cc-ci/.testenv`). **Single-writer discipline:** work on a dedicated
+6
View File
@@ -312,6 +312,12 @@ test change, and a test change is **gated by `--with-tests`**:
Do **NOT** modify any test. Report `SUCCESS-PENDING-TESTS` (recipe PR open; `!testme` red on a
stale test; operator to decide).
- **`--with-tests` — open + verify a cc-ci test PR.** Make it the `ci-test-review` way:
0. **READ `tests/STYLE.md` in the cc-ci repo FIRST.** It is the rulebook for changing a test, and
it is written against the failures this pipeline has actually produced. The two that matter most
here: **set state up through the app's own interface, never its database** (a plausible fixture
that INSERTed rows passed on v2 and silently broke on v3, holding the recipe RED for six weeks),
and **gate on version rather than writing a fixture that supports both** — old-version tests can
simply be deleted, since the older version is only exercised through the upgrade tier.
1. Branch `recipe-maintainers/cc-ci` in a **separate clone** (single-writer: never push `main`,
never touch the build loops' `/cc-ci` `/cc-ci-adv` clones); update the test/overlay.
2. **Verify the recipe upgrade WITH the updated test applied.** `!testme` on the recipe PR uses the
@@ -85,6 +85,14 @@ failure (AI — this is the `ci-test-review` step-3 diagnosis):
### 2. For each stale test — author the minimal test update (AI; never weaken)
> **Read `tests/STYLE.md` in the cc-ci repo before writing the update.** It is the rulebook for test
> changes, written from failures this pipeline actually produced. Most load-bearing: set state up
> through the app's **own interface, never its database** (a plausible fixture that INSERTed rows
> passed on v2 and silently broke on v3 — 202 acks, rows in postgres, nothing ingested — and held the
> recipe RED for six weeks), **gate on version rather than supporting both** (old-version tests can be
> deleted; the older version is only exercised via the upgrade tier), and correct the fixture or the
> wait but **never the assertion**.
Work on **one recipe at a time** (serialize — each verification deploys a recipe on the shared
Swarm). For each `STALE_TESTS` entry: