recipe-maintainers/cc-ci
0
0
Fork 0
Code Issues 1 Pull Requests 3 Actions Packages Projects Releases Wiki Activity
Files
fd3db37c49dcbbb8fd31155b7d79e95bac96e4f2
cc-ci/machine-docs/BACKLOG-regression.md
autonomic-bot fd3db37c49
Some checks failed
continuous-integration/drone/push Build is failing
Details
feat(regression): add tests/regression/ E2E canary suite 
Three canaries (@pytest.mark.canary) drive the real cold CI lifecycle:
- good-simple: custom-html-tiny @ main (435df8fc) — fast signal, expects GREEN
- good-significant: lasuite-docs @ main (290a8ad7) — multi-service, expects GREEN
- bad-false-green: custom-html @ v5-stale-docroot (71e7326a) — expects RED

Semantic teeth: beyond exit-code, each test asserts that specific named tests
ran in results.json stages (test_serving, test_serving_and_frontend, test_content_type).
If an assertion is removed, the named test disappears → regression test fails.

Includes conftest (run_recipe_ci helper + stage_has_{passing,failing}_test),
README (cadence policy, how to run, how to add), and phase state files.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-02 01:25:55 +00:00

674 B
Raw Blame History

BACKLOG — server regression canaries phase

Build backlog

  • Create tests/regression/ suite (conftest + test_canaries + README)
  • Run good-simple canary (custom-html-tiny main) → confirm GREEN + test_serving passes
  • Run bad-false-green canary (custom-html v5-stale-docroot) → confirm RED + test_content_type fails
  • Run good-significant canary (lasuite-docs main) → confirm GREEN + test_serving_and_frontend passes
  • Open PR for operator review (DoD item 5: NOT merged)
  • Claim gate once all canary runs are GREEN/RED as expected + PR is open

Adversary findings

(None yet — will be appended as findings are discovered)