From 464760ebb729a1d275081d33f33d777de1230131 Mon Sep 17 00:00:00 2001 From: autonomic-bot Date: Tue, 2 Jun 2026 01:34:56 +0000 Subject: [PATCH] =?UTF-8?q?review(regression):=20D-initial=20FAIL=20?= =?UTF-8?q?=E2=80=94=20A-reg-1=20relative=20import=20(suite=20won't=20coll?= =?UTF-8?q?ect),=20A-reg-2=20plan=20gap=20(4=20per-tier=20RED=20canaries?= =?UTF-8?q?=20missing)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- machine-docs/BACKLOG-regression.md | 54 +++++++++++++++++++++++++++++- machine-docs/BUILDER-INBOX.md | 51 ++++++++++++++++++++++++++++ machine-docs/REVIEW-regression.md | 40 ++++++++++++++++++++-- 3 files changed, 142 insertions(+), 3 deletions(-) create mode 100644 machine-docs/BUILDER-INBOX.md diff --git a/machine-docs/BACKLOG-regression.md b/machine-docs/BACKLOG-regression.md index cffa37f..56b2564 100644 --- a/machine-docs/BACKLOG-regression.md +++ b/machine-docs/BACKLOG-regression.md @@ -11,4 +11,56 @@ ## Adversary findings -*(None yet — will be appended as findings are discovered)* +### A-reg-1 [adversary] OPEN — CRITICAL: relative import fails, suite won't collect +**Filed:** 2026-06-02T01:37Z +**Severity:** CRITICAL — suite can't run at all until fixed + +Cold-run `cc-ci-run -m pytest tests/regression/ --collect-only` on cc-ci confirms: +``` +ImportError: attempted relative import with no known parent package +tests/regression/test_canaries.py:18: from .conftest import run_recipe_ci, ... +``` +No tests collected. 0 canaries can run. + +**Root cause:** `test_canaries.py` uses a relative import (`from .conftest import ...`) which +requires the directory to be a Python package. Without `tests/regression/__init__.py` (and +`tests/__init__.py`), pytest imports `test_canaries.py` as a top-level module, not a package +member. Relative imports fail. + +**Repro:** +```bash +ssh cc-ci +cd /root/builder-clone +cc-ci-run -m pytest tests/regression/ --collect-only +# → ImportError: attempted relative import with no known parent package +``` + +**Fix (either approach):** +1. Add `tests/__init__.py` and `tests/regression/__init__.py` (makes it a real package) +2. OR replace `from .conftest import ...` with absolute sys.path manipulation (like other test + files do, e.g. `sys.path.insert(0, ...); import conftest`) + +**Adversary closes:** after re-running `--collect-only` confirms 3+ tests collected, no error. + +--- + +### A-reg-2 [adversary] OPEN — Plan gap: 4 per-tier RED canaries required by updated DoD +**Filed:** 2026-06-02T01:37Z +**Severity:** HIGH — DoD#4 unmet; Builder cannot claim DONE without these + +Updated plan (commit 7bdeb74) added DoD#4: four per-tier RED canaries (install/upgrade/backup/ +restore on `custom-html-tiny`) that prove the server reports RED at EACH tier. Each must: +- Assert overall verdict RED at the intended tier +- Assert prior tiers PASSED +- Have teeth: wrongly-green tier would FAIL the test + +Current suite only has 3 canaries (good-simple, good-significant, bad-false-green). The 4 +per-tier RED canaries are MISSING. This is a mandatory DoD item. + +These also require: +- Fixture branches or SHA-pinned commits where custom-html-tiny is broken at exactly one tier +- A `@pytest.mark.canary_fast` sub-marker (plan recommends it for the fast RED subset) +- README update to document the fast subset + +**Adversary closes:** after all 4 canaries exist, run, and the Adversary cold-verifies each +produces RED at the intended tier with prior tiers PASS. diff --git a/machine-docs/BUILDER-INBOX.md b/machine-docs/BUILDER-INBOX.md new file mode 100644 index 0000000..226bb84 --- /dev/null +++ b/machine-docs/BUILDER-INBOX.md @@ -0,0 +1,51 @@ +# BUILDER-INBOX — from Adversary @2026-06-02T01:38Z + +## D-initial: FAIL — two issues to fix before re-claiming + +**Stop any canary runs that are in flight until A-reg-1 is fixed — they can't complete via +the pytest suite as written (the suite won't collect). The harness would still work standalone, +but the regression test wrapper is broken.** + +### A-reg-1 CRITICAL: relative import, suite fails to collect + +Cold verify on cc-ci (your builder-clone): +``` +cc-ci-run -m pytest tests/regression/ --collect-only +``` +Result: +``` +collected 0 items / 1 error +ImportError: attempted relative import with no known parent package +tests/regression/test_canaries.py:18: from .conftest import run_recipe_ci, ... +``` + +Fix: in `tests/regression/test_canaries.py`, replace: +```python +from .conftest import run_recipe_ci, stage_has_failing_test, stage_has_passing_test +``` +with: +```python +import os, sys +sys.path.insert(0, os.path.dirname(__file__)) +import conftest as _reg_conftest +run_recipe_ci = _reg_conftest.run_recipe_ci +stage_has_passing_test = _reg_conftest.stage_has_passing_test +stage_has_failing_test = _reg_conftest.stage_has_failing_test +``` +OR add `tests/__init__.py` and `tests/regression/__init__.py` (empty files) to make it a package. +The sys.path approach matches how other test files in this repo import harness modules. + +### A-reg-2 HIGH: plan updated (7bdeb74) — 4 per-tier RED canaries now mandatory (DoD#4) + +The updated plan requires: +- `bad-install`: install tier fails (image never healthy) +- `bad-upgrade`: install PASS, upgrade tier fails +- `bad-backup`: install+upgrade PASS, backup tier fails +- `bad-restore`: backup PASS, restore tier fails (marker absent — the scariest false-green) + +Each needs a fixture SHA pinned on custom-html-tiny. Each test must assert: overall RED, the +intended tier FAIL, prior tiers PASS. Optional: add `@pytest.mark.canary_fast` for the fast +subset (plan recommends it). + +Let me know when both are fixed and the --collect-only confirms N tests collected — I'll +re-verify then. diff --git a/machine-docs/REVIEW-regression.md b/machine-docs/REVIEW-regression.md index 257a16e..39c9981 100644 --- a/machine-docs/REVIEW-regression.md +++ b/machine-docs/REVIEW-regression.md @@ -10,13 +10,49 @@ ## D-gate verdicts -*(None yet — Builder has not claimed any gate. Watching for STATUS-regression.md → Gate: CLAIMED)* +### D-initial: FAIL @2026-06-02T01:38Z — suite won't collect (A-reg-1); plan gap (A-reg-2) + +Builder claimed: test suite written, initial gate; canaries in-flight. + +**Cold verification result: FAIL — two blocking issues.** + +**A-reg-1 (CRITICAL): Relative import fails, 0 tests collected.** +``` +ssh cc-ci && cd /root/builder-clone +cc-ci-run -m pytest tests/regression/ --collect-only +``` +Output (cold, fresh shell): +``` +collected 0 items / 1 error +ImportError: attempted relative import with no known parent package +tests/regression/test_canaries.py:18: from .conftest import run_recipe_ci, ... +!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!! +``` +Root cause: `tests/regression/__init__.py` and `tests/__init__.py` missing. Fix: add them or +use absolute imports (as other test files in this repo do). + +**A-reg-2 (HIGH): Plan updated (commit 7bdeb74) — 4 per-tier RED canaries now mandatory (DoD#4).** +Updated plan requires RED canaries for install/upgrade/backup/restore tiers on custom-html-tiny, +each asserting RED at the intended tier with prior tiers PASS. Current suite: 3 canaries only +(2 good + 1 bad-custom-assertion). All four are MISSING. Cannot claim DONE without them. + +**Other code quality observations (not blocking):** +- Canary SHAs all verified present on Gitea ✓ + - custom-html-tiny: `435df8fc98ef7598` ✓ (main 2026-06-02 merge commit) + - lasuite-docs: `290a8ad72d06232f` ✓ (v0.3.3+v5.1.0 merge) + - custom-html v5-stale-docroot: `71e7326a99bbb690` ✓ (confirmed RED via build #81) +- `CCCI_RUN_ID` and `CCCI_RUNS_DIR` correctly picked up by `results.py` ✓ +- `_assert_red` / `_assert_green` logic sound ✓ +- README cadence policy complete ✓ + +**Verdict: FAIL. Standing issues: A-reg-1 (critical), A-reg-2 (high). Builder must fix both +before re-claiming this gate.** --- ## Adversary findings -*(None yet)* +*(See BACKLOG-regression.md § Adversary findings: A-reg-1, A-reg-2)* ---