recipe-maintainers/cc-ci
0
0
Fork 0
Code Issues 1 Pull Requests 3 Actions Packages Projects Releases Wiki Activity

feat(regression): E2E canary regression suite — operator review #4

New Issue
Open
opened 2026-06-02 02:19:04 +00:00 by autonomic-bot · 0 comments
autonomic-bot commented 2026-06-02 02:19:04 +00:00
Owner
Copy Link

Summary

This represents the server regression canaries phase — a standing pytest suite that gives the cc-ci server codified E2E self-tests.

What was built

tests/regression/ — 7 parametrized canaries (@pytest.mark.canary + @pytest.mark.canary_fast):

Canary Recipe Expected Status
good-simple custom-html-tiny @ 435df8fc GREEN ✓ Verified
good-significant lasuite-docs @ 290a8ad7 GREEN Re-running (upgrade flakiness)
bad-false-green custom-html @ 71e7326a (v5-stale-docroot) RED (custom=fail) ✓ Verified
bad-install custom-html-tiny @ 4ae88661 RED (install=fail) ✓ Verified
bad-upgrade custom-html-tiny @ 4ae88661 RED (upgrade=fail) ✓ Verified
bad-backup custom-html-bkp-bad @ b6fe99de RED (backup=fail) ✓ Verified
bad-restore custom-html-rst-bad @ 9a73a184 RED (restore=fail) ✓ Verified

How to run

# Fast RED canaries (~2-3 min each):
cd /root/builder-clone && cc-ci-run -m pytest tests/regression/ -m canary_fast -v

# Full suite (includes lasuite-docs, 10-20 min):
cc-ci-run -m pytest tests/regression/ -m canary -v

Related recipe fixtures on Gitea

  • recipe-maintainers/custom-html-tiny branch regression-bad-image — bad image for install/upgrade RED
  • recipe-maintainers/custom-html branch v5-stale-docroot — existing bad MIME type fixture
  • recipe-maintainers/custom-html-bkp-bad — no pre_backup hook, test_backup.py fails
  • recipe-maintainers/custom-html-rst-bad — no pre_backup hook, ops.py+test_restore.py

Definition of Done (plan § DoD)

  1. ✓ tests/regression/ suite committed (7 tests, 0 collection errors)
  2. ✓ good-simple GREEN — install=pass, upgrade=pass, test_serving PASS in install stage
  3. ✓ bad-false-green RED — custom=fail, test_content_type fails (content-type guard has teeth)
  4. ✓ 4 per-tier RED canaries: each asserts RED at the intended tier with prior tiers PASS
  5. ✓ README.md with cadence policy and how to add a canary
  6. ✓ NOT merged — this PR is for operator review only (loops never merge)

good-significant (lasuite-docs) upgrade had a transient WOPI convergence race on first run; re-running now to confirm.

🤖 Generated with Claude Code

## Summary This represents the **server regression canaries** phase — a standing pytest suite that gives the cc-ci server codified E2E self-tests. ### What was built `tests/regression/` — 7 parametrized canaries (`@pytest.mark.canary` + `@pytest.mark.canary_fast`): | Canary | Recipe | Expected | Status | |--------|--------|----------|--------| | good-simple | custom-html-tiny @ 435df8fc | GREEN | ✓ Verified | | good-significant | lasuite-docs @ 290a8ad7 | GREEN | Re-running (upgrade flakiness) | | bad-false-green | custom-html @ 71e7326a (v5-stale-docroot) | RED (custom=fail) | ✓ Verified | | bad-install | custom-html-tiny @ 4ae88661 | RED (install=fail) | ✓ Verified | | bad-upgrade | custom-html-tiny @ 4ae88661 | RED (upgrade=fail) | ✓ Verified | | bad-backup | custom-html-bkp-bad @ b6fe99de | RED (backup=fail) | ✓ Verified | | bad-restore | custom-html-rst-bad @ 9a73a184 | RED (restore=fail) | ✓ Verified | ### How to run ```bash # Fast RED canaries (~2-3 min each): cd /root/builder-clone && cc-ci-run -m pytest tests/regression/ -m canary_fast -v # Full suite (includes lasuite-docs, 10-20 min): cc-ci-run -m pytest tests/regression/ -m canary -v ``` ### Related recipe fixtures on Gitea - `recipe-maintainers/custom-html-tiny` branch `regression-bad-image` — bad image for install/upgrade RED - `recipe-maintainers/custom-html` branch `v5-stale-docroot` — existing bad MIME type fixture - `recipe-maintainers/custom-html-bkp-bad` — no pre_backup hook, test_backup.py fails - `recipe-maintainers/custom-html-rst-bad` — no pre_backup hook, ops.py+test_restore.py ### Definition of Done (plan § DoD) 1. ✓ `tests/regression/` suite committed (7 tests, 0 collection errors) 2. ✓ good-simple GREEN — install=pass, upgrade=pass, test_serving PASS in install stage 3. ✓ bad-false-green RED — custom=fail, test_content_type fails (content-type guard has teeth) 4. ✓ 4 per-tier RED canaries: each asserts RED at the intended tier with prior tiers PASS 5. ✓ README.md with cadence policy and how to add a canary 6. ✓ NOT merged — this PR is for operator review only (loops never merge) good-significant (lasuite-docs) upgrade had a transient WOPI convergence race on first run; re-running now to confirm. --- 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
autonomic autonomic-bot notplants trav
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: recipe-maintainers/cc-ci#4
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?