review(regression): D-final PASS — all 7 canaries cold-verified; PR#5 open; DoD complete
Some checks failed
continuous-integration/drone/push Build is failing
Some checks failed
continuous-integration/drone/push Build is failing
Cold-verified from cc-ci artifact dirs + PR branch collect: - DoD#1: 7 tests collect from regression-canaries branch ✓ - DoD#2: good-simple (install/upgrade=pass, test_serving) ✓; good-significant run-2 (all tiers pass, test_serving_and_frontend) ✓ - DoD#3: bad-false-green RED, rc!=0 false-green guard has teeth ✓ - DoD#4: all 4 per-tier RED canaries at correct tiers (install/upgrade/backup/restore) ✓ - DoD#5: README cadence+canaries+add-instructions ✓ - DoD#6: PR#5 state=open, merged=False ✓ Inbox consumed; no vetoes; phase DONE pending operator PR review.
This commit is contained in:
@ -1,12 +0,0 @@
|
||||
# ADVERSARY-INBOX — final gate update
|
||||
|
||||
good-significant (lasuite-docs) re-run completed GREEN — `regression-good-significant-2` on server:
|
||||
- install=pass, upgrade=pass, backup=pass, restore=pass, custom=pass
|
||||
- clean_teardown=true, no_secret_leak=true
|
||||
- All 5 custom/functional tests pass (auth, create-doc, health, OIDC×2)
|
||||
|
||||
PR#5 opened: https://git.autonomic.zone/recipe-maintainers/cc-ci/pulls/5
|
||||
Branch regression-canaries→main, 10 files, 704 insertions.
|
||||
|
||||
This is the FINAL D-final claim (commit `claim(D-final)`). All 7 canaries verified.
|
||||
All DoD items complete. Awaiting your PASS verdict in REVIEW-regression.md.
|
||||
@ -10,6 +10,53 @@
|
||||
|
||||
## D-gate verdicts
|
||||
|
||||
### D-final: PASS @2026-06-02T03:36Z — all 7 canaries cold-verified; PR#5 open; all DoD items met
|
||||
|
||||
**Cold verification result: PASS**
|
||||
|
||||
All DoD items independently verified (cold shell, Adversary clone, no cached state):
|
||||
|
||||
**DoD#1 — tests/regression/ committed:**
|
||||
- `cc-ci-run -m pytest tests/regression/ --collect-only -q` on cc-ci from PR branch: 7 tests collected ✓
|
||||
- Files present on `regression-canaries` branch: `conftest.py`, `test_canaries.py`, `README.md`, plus `tests/custom-html-bkp-bad/` and `tests/custom-html-rst-bad/` ✓
|
||||
|
||||
**DoD#2 — both good canaries GREEN with semantic assertion teeth:**
|
||||
- `good-simple` (regression-good-simple-1, SHA `435df8fc`): `install=pass, upgrade=pass`, `test_serving` PASS in install stage ✓
|
||||
- Teeth: if `test_serving` removed → `stage_has_passing_test("install","test_serving")` → False → assert fires ✓
|
||||
- `good-significant` (regression-good-significant-2, SHA `290a8ad7`): `install=pass, upgrade=pass, backup=pass, restore=pass, custom=pass`, `clean_teardown=true`, `no_secret_leak=true` ✓
|
||||
- `test_serving_and_frontend` PASS in install stage ✓
|
||||
- Teeth: if `test_serving_and_frontend` removed → `stage_has_passing_test("install","test_serving_and_frontend")` → False → assert fires ✓
|
||||
- Run 1 had upgrade=fail (convergence race, transient); run 2 fully GREEN. Known plan risk; no action needed unless persistent.
|
||||
|
||||
**DoD#3 — bad-false-green catches false-green:**
|
||||
- `bad-false-green` (regression-bad-canary-1, SHA `71e7326a`): `custom=fail`, `test_content_type_html_and_txt: FAIL` (Content-Type='application/octet-stream') ✓
|
||||
- Teeth: if harness returns rc=0 → `assert rc != 0` fires → false-green caught ✓
|
||||
|
||||
**DoD#4 — 4 per-tier RED canaries (cold-verified from artifacts):**
|
||||
- `bad-install` (regression-bad-install-v2, SHA `4ae8866`): `install=fail, upgrade=na` ✓ — failing_tier=install, passing_before=[] ✓
|
||||
- `bad-upgrade` (regression-bad-upgrade-v2, SHA `4ae8866`): `install=pass, upgrade=fail` ✓ — prior tier PASS verified ✓
|
||||
- `bad-backup` (regression-bad-backup-5, SHA `b6fe99de`, recipe `custom-html-bkp-bad`): `install=pass, backup=fail` ✓ — `test_backup_captures_state` FAIL ✓
|
||||
- `bad-restore` (regression-bad-restore-3, SHA `9a73a184`, recipe `custom-html-rst-bad`): `install=pass, backup=pass, restore=fail` ✓ — `test_restore_returns_state` FAIL ✓
|
||||
- All 4: if harness wrongly returned rc=0 → `assert rc != 0` fires ✓; if wrong tier failed → tier check assertion fires ✓
|
||||
|
||||
**DoD#5 — README.md:**
|
||||
- `tests/regression/README.md` present on regression-canaries branch ✓
|
||||
- Contains: cadence policy ("Do NOT run on every commit"), canary table, per-tier teeth explanation, how to add a canary ✓
|
||||
|
||||
**DoD#6 — NOT merged, PR opened for operator review:**
|
||||
- PR#5: `https://git.autonomic.zone/recipe-maintainers/cc-ci/pulls/5` — state=open, merged=False ✓
|
||||
- Branch: `regression-canaries` → `main`. 10 files, 704 insertions ✓
|
||||
- PR body says "Do not merge — loops never merge" ✓
|
||||
|
||||
**Observations (non-blocking, not DoD blockers):**
|
||||
- good-significant run 1's upgrade=fail was a convergence race; transient (run 2 passed without retry). No test weakening, no retry added — consistent with plan policy.
|
||||
- Semantic stage_pass_checks only explicitly guard install tier for good-significant. Upgrade/backup/restore tooth coverage is via `_assert_green`'s "no tier failed" check. Limitation noted; acceptable per plan DoD requirements.
|
||||
- A-reg-2 comment in test_canaries.py says "test_backup_artifact fails" for bad-backup; actual behavior is test_backup_artifact passes and test_backup_captures_state fails. Misleading comment, non-blocking.
|
||||
|
||||
**Verdict: D-final PASS.** All 7 canaries verified. All 6 DoD items met. Phase is complete pending operator review of PR#5. No vetoes.
|
||||
|
||||
---
|
||||
|
||||
### D-initial update @2026-06-02T01:46Z — A-reg-1 CLOSED; A-reg-2 still open
|
||||
|
||||
**A-reg-1 RESOLVED.** Cold-verify after fix:
|
||||
|
||||
Reference in New Issue
Block a user