# STATUS — server regression canaries phase

**Phase:** server regression canaries (codified E2E self-tests)
**SSOT:** `/srv/cc-ci/cc-ci-plan/plan-server-regression-canaries.md`
**Builder loop started:** 2026-06-02
**Repo:** git.autonomic.zone/recipe-maintainers/cc-ci

---

## Current state

**Gate: D-initial CLAIMED — test suite written; awaiting first canary run**

The `tests/regression/` suite is committed. Before claiming the final gate (all DoD items
verified), the canaries need to actually run on the live server and return the expected verdicts.
Currently running the good-simple (custom-html-tiny) canary to confirm GREEN.

---

## What was built

`tests/regression/` committed in the cc-ci repo:
- `conftest.py` — `run_recipe_ci()` helper that invokes the real harness as subprocess, returns `(rc, results_dict, artifact_dir)`; `stage_has_passing_test()` / `stage_has_failing_test()` helpers for semantic checks
- `test_canaries.py` — parametrized `@pytest.mark.canary` test with three canaries (see below)
- `README.md` — cadence policy, how to run, how to add a canary

---

## Canaries defined

| ID | Recipe | SHA pinned | Expected |
|----|--------|-----------|----------|
| `good-simple` | `custom-html-tiny` | `435df8fc` (main 2026-06-02) | GREEN |
| `good-significant` | `lasuite-docs` | `290a8ad7` (main 2026-06-02) | GREEN |
| `bad-false-green` | `custom-html` | `71e7326a` (v5-stale-docroot) | RED |

---

## Semantic assertions (teeth)

Good canaries:
- `rc == 0` (harness exit)
- install tier: "pass"
- No tier is "fail"
- `flags.clean_teardown == True`
- `flags.no_secret_leak == True`
- Named test `test_serving` present + passing in install stage (custom-html-tiny)
- Named test `test_serving_and_frontend` present + passing in install stage (lasuite-docs)

Bad canary:
- `rc != 0` (PRIMARY — false-green catches here)
- Named test `test_content_type` present + FAILING in custom stage (proves guard not vacuated)

---

## How to verify (Adversary commands)

From cc-ci server root (requires the repo checked out at `/root/cc-ci` or similar):

```bash
# Good simple (fast ~2-5 min):
cc-ci-run python -m pytest tests/regression/ -m canary -k good-simple -v

# Bad canary (fast ~2-5 min, same recipe lifecycle):
cc-ci-run python -m pytest tests/regression/ -m canary -k bad-false-green -v

# Full suite (slow — lasuite-docs is 10-20 min):
cc-ci-run python -m pytest tests/regression/ -m canary -v
```

Expected outcomes:
- `good-simple`: test PASSES (harness returns GREEN, test_serving passes)
- `bad-false-green`: test PASSES (harness returns RED, test_content_type fails in custom stage)
- `good-significant`: test PASSES (harness returns GREEN, test_serving_and_frontend passes)

Verify teeth: tamper with an outcome to confirm the regression test fails:
- For good canary: unset `test_serving` (remove it) → `stage_has_passing_test` returns False → test fails
- For bad canary: change the assert to `rc == 0` → would fail if harness returns non-zero (teeth work)

---

## In-flight

- Canary `good-simple` running on cc-ci (started ~now)
- Status will be updated once run completes and we have actual output to paste