Commit Graph

5 Commits

Author SHA1 Message Date
3b7267cbee fix(regression): use custom-html-bkp-bad recipe for bad-backup canary
Some checks failed
continuous-integration/drone/push Build is failing
backupbot-two ignores nonexistent backup paths and backs up the whole
volume, making the bad-path approach unreliable. New approach:
- Create recipe-maintainers/custom-html-bkp-bad on Gitea (custom-html
  without backupbot.backup=true label) — SHA 4e584063a99a
- Add tests/custom-html-bkp-bad/recipe_meta.py with BACKUP_CAPABLE=True
  so the harness runs the backup tier despite auto-detect returning False
- Without a labeled container, backup-bot-two produces no snapshot →
  parse_snapshot_id=None → test_backup_artifact fails → backup=RED ✓

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-02 02:07:06 +00:00
090724ec80 fix(regression): correct SHAs for bad-backup/bad-restore (A-reg-3) + consume inbox
Some checks failed
continuous-integration/drone/push Build is failing
continuous-integration/drone Build is passing
Both compose.yml uploads had empty files due to a bash encoding bug.
Fixed via Python API upload; new SHAs:
- regression-bad-backup: cd52b3a (backupbot.backup.path=/nonexistent-path-cc-ci-canary-bad)
- regression-bad-restore: 7e03499 (backup targets .backup-data subdir + command creates it)

Adversary confirmed bad-install ✓ and bad-upgrade ✓ from run artifacts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-02 02:00:51 +00:00
cf405b4195 feat(regression): add 4 per-tier RED canaries (DoD#4) + canary_fast marker
Some checks failed
continuous-integration/drone/push Build is failing
Four new per-tier RED canaries prove the server catches failure at every
lifecycle tier:

- bad-install: custom-html-tiny @ regression-bad-image (4ae88661)
  nonexistent image → prepull fails → install=fail
  STAGES=install → no prev-version lookup → chaos deploy of HEAD

- bad-upgrade: same branch + SHA, STAGES=install,upgrade
  install uses prev-version (good image) → PASS
  upgrade chaos checks out HEAD (bad image) → prepull fails → FAIL

- bad-backup: custom-html @ regression-bad-backup (e1e3c5fc)
  backupbot.backup.path=/nonexistent-path-cc-ci-canary-bad
  abra app backup create fails → backup=fail

- bad-restore: custom-html @ regression-bad-restore (5a481cc1)
  backup targets .backup-data/ subdir (not where ci-marker.txt lives)
  backup succeeds; restore puts .backup-data back but NOT the marker
  marker stays "mutated" → test_restore_returns_state FAILS → restore=fail

Each test asserts: rc!=0, failing_tier="fail", prior tiers="pass".
Adds @pytest.mark.canary_fast for the fast subset.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-02 01:49:28 +00:00
a2a6eea757 fix(regression): fix relative import (A-reg-1) + consume inbox
Some checks failed
continuous-integration/drone/push Build is failing
- tests/regression/test_canaries.py: replace `from .conftest import ...`
  (relative import fails when not a package) with sys.path + direct import,
  matching the pattern used by all other tests in this repo.
- Delete machine-docs/BUILDER-INBOX.md (Adversary inbox consumed).
- Update STATUS-regression.md + JOURNAL-regression.md with first two
  canary run results (bad-false-green RED confirmed, good-simple GREEN confirmed).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-02 01:37:31 +00:00
fd3db37c49 feat(regression): add tests/regression/ E2E canary suite
Some checks failed
continuous-integration/drone/push Build is failing
Three canaries (@pytest.mark.canary) drive the real cold CI lifecycle:
- good-simple: custom-html-tiny @ main (435df8fc) — fast signal, expects GREEN
- good-significant: lasuite-docs @ main (290a8ad7) — multi-service, expects GREEN
- bad-false-green: custom-html @ v5-stale-docroot (71e7326a) — expects RED

Semantic teeth: beyond exit-code, each test asserts that specific named tests
ran in results.json stages (test_serving, test_serving_and_frontend, test_content_type).
If an assertion is removed, the named test disappears → regression test fails.

Includes conftest (run_recipe_ci helper + stage_has_{passing,failing}_test),
README (cadence policy, how to run, how to add), and phase state files.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-02 01:25:55 +00:00