review(regression): D-initial FAIL — A-reg-1 relative import (suite won't collect), A-reg-2 plan gap (4 per-tier RED canaries missing)
continuous-integration/drone/push Build is failing
continuous-integration/drone/push Build is failing
This commit is contained in:
@@ -11,4 +11,56 @@
|
|||||||
|
|
||||||
## Adversary findings
|
## Adversary findings
|
||||||
|
|
||||||
*(None yet — will be appended as findings are discovered)*
|
### A-reg-1 [adversary] OPEN — CRITICAL: relative import fails, suite won't collect
|
||||||
|
**Filed:** 2026-06-02T01:37Z
|
||||||
|
**Severity:** CRITICAL — suite can't run at all until fixed
|
||||||
|
|
||||||
|
Cold-run `cc-ci-run -m pytest tests/regression/ --collect-only` on cc-ci confirms:
|
||||||
|
```
|
||||||
|
ImportError: attempted relative import with no known parent package
|
||||||
|
tests/regression/test_canaries.py:18: from .conftest import run_recipe_ci, ...
|
||||||
|
```
|
||||||
|
No tests collected. 0 canaries can run.
|
||||||
|
|
||||||
|
**Root cause:** `test_canaries.py` uses a relative import (`from .conftest import ...`) which
|
||||||
|
requires the directory to be a Python package. Without `tests/regression/__init__.py` (and
|
||||||
|
`tests/__init__.py`), pytest imports `test_canaries.py` as a top-level module, not a package
|
||||||
|
member. Relative imports fail.
|
||||||
|
|
||||||
|
**Repro:**
|
||||||
|
```bash
|
||||||
|
ssh cc-ci
|
||||||
|
cd /root/builder-clone
|
||||||
|
cc-ci-run -m pytest tests/regression/ --collect-only
|
||||||
|
# → ImportError: attempted relative import with no known parent package
|
||||||
|
```
|
||||||
|
|
||||||
|
**Fix (either approach):**
|
||||||
|
1. Add `tests/__init__.py` and `tests/regression/__init__.py` (makes it a real package)
|
||||||
|
2. OR replace `from .conftest import ...` with absolute sys.path manipulation (like other test
|
||||||
|
files do, e.g. `sys.path.insert(0, ...); import conftest`)
|
||||||
|
|
||||||
|
**Adversary closes:** after re-running `--collect-only` confirms 3+ tests collected, no error.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### A-reg-2 [adversary] OPEN — Plan gap: 4 per-tier RED canaries required by updated DoD
|
||||||
|
**Filed:** 2026-06-02T01:37Z
|
||||||
|
**Severity:** HIGH — DoD#4 unmet; Builder cannot claim DONE without these
|
||||||
|
|
||||||
|
Updated plan (commit 7bdeb74) added DoD#4: four per-tier RED canaries (install/upgrade/backup/
|
||||||
|
restore on `custom-html-tiny`) that prove the server reports RED at EACH tier. Each must:
|
||||||
|
- Assert overall verdict RED at the intended tier
|
||||||
|
- Assert prior tiers PASSED
|
||||||
|
- Have teeth: wrongly-green tier would FAIL the test
|
||||||
|
|
||||||
|
Current suite only has 3 canaries (good-simple, good-significant, bad-false-green). The 4
|
||||||
|
per-tier RED canaries are MISSING. This is a mandatory DoD item.
|
||||||
|
|
||||||
|
These also require:
|
||||||
|
- Fixture branches or SHA-pinned commits where custom-html-tiny is broken at exactly one tier
|
||||||
|
- A `@pytest.mark.canary_fast` sub-marker (plan recommends it for the fast RED subset)
|
||||||
|
- README update to document the fast subset
|
||||||
|
|
||||||
|
**Adversary closes:** after all 4 canaries exist, run, and the Adversary cold-verifies each
|
||||||
|
produces RED at the intended tier with prior tiers PASS.
|
||||||
|
|||||||
@@ -0,0 +1,51 @@
|
|||||||
|
# BUILDER-INBOX — from Adversary @2026-06-02T01:38Z
|
||||||
|
|
||||||
|
## D-initial: FAIL — two issues to fix before re-claiming
|
||||||
|
|
||||||
|
**Stop any canary runs that are in flight until A-reg-1 is fixed — they can't complete via
|
||||||
|
the pytest suite as written (the suite won't collect). The harness would still work standalone,
|
||||||
|
but the regression test wrapper is broken.**
|
||||||
|
|
||||||
|
### A-reg-1 CRITICAL: relative import, suite fails to collect
|
||||||
|
|
||||||
|
Cold verify on cc-ci (your builder-clone):
|
||||||
|
```
|
||||||
|
cc-ci-run -m pytest tests/regression/ --collect-only
|
||||||
|
```
|
||||||
|
Result:
|
||||||
|
```
|
||||||
|
collected 0 items / 1 error
|
||||||
|
ImportError: attempted relative import with no known parent package
|
||||||
|
tests/regression/test_canaries.py:18: from .conftest import run_recipe_ci, ...
|
||||||
|
```
|
||||||
|
|
||||||
|
Fix: in `tests/regression/test_canaries.py`, replace:
|
||||||
|
```python
|
||||||
|
from .conftest import run_recipe_ci, stage_has_failing_test, stage_has_passing_test
|
||||||
|
```
|
||||||
|
with:
|
||||||
|
```python
|
||||||
|
import os, sys
|
||||||
|
sys.path.insert(0, os.path.dirname(__file__))
|
||||||
|
import conftest as _reg_conftest
|
||||||
|
run_recipe_ci = _reg_conftest.run_recipe_ci
|
||||||
|
stage_has_passing_test = _reg_conftest.stage_has_passing_test
|
||||||
|
stage_has_failing_test = _reg_conftest.stage_has_failing_test
|
||||||
|
```
|
||||||
|
OR add `tests/__init__.py` and `tests/regression/__init__.py` (empty files) to make it a package.
|
||||||
|
The sys.path approach matches how other test files in this repo import harness modules.
|
||||||
|
|
||||||
|
### A-reg-2 HIGH: plan updated (7bdeb74) — 4 per-tier RED canaries now mandatory (DoD#4)
|
||||||
|
|
||||||
|
The updated plan requires:
|
||||||
|
- `bad-install`: install tier fails (image never healthy)
|
||||||
|
- `bad-upgrade`: install PASS, upgrade tier fails
|
||||||
|
- `bad-backup`: install+upgrade PASS, backup tier fails
|
||||||
|
- `bad-restore`: backup PASS, restore tier fails (marker absent — the scariest false-green)
|
||||||
|
|
||||||
|
Each needs a fixture SHA pinned on custom-html-tiny. Each test must assert: overall RED, the
|
||||||
|
intended tier FAIL, prior tiers PASS. Optional: add `@pytest.mark.canary_fast` for the fast
|
||||||
|
subset (plan recommends it).
|
||||||
|
|
||||||
|
Let me know when both are fixed and the --collect-only confirms N tests collected — I'll
|
||||||
|
re-verify then.
|
||||||
@@ -10,13 +10,49 @@
|
|||||||
|
|
||||||
## D-gate verdicts
|
## D-gate verdicts
|
||||||
|
|
||||||
*(None yet — Builder has not claimed any gate. Watching for STATUS-regression.md → Gate: CLAIMED)*
|
### D-initial: FAIL @2026-06-02T01:38Z — suite won't collect (A-reg-1); plan gap (A-reg-2)
|
||||||
|
|
||||||
|
Builder claimed: test suite written, initial gate; canaries in-flight.
|
||||||
|
|
||||||
|
**Cold verification result: FAIL — two blocking issues.**
|
||||||
|
|
||||||
|
**A-reg-1 (CRITICAL): Relative import fails, 0 tests collected.**
|
||||||
|
```
|
||||||
|
ssh cc-ci && cd /root/builder-clone
|
||||||
|
cc-ci-run -m pytest tests/regression/ --collect-only
|
||||||
|
```
|
||||||
|
Output (cold, fresh shell):
|
||||||
|
```
|
||||||
|
collected 0 items / 1 error
|
||||||
|
ImportError: attempted relative import with no known parent package
|
||||||
|
tests/regression/test_canaries.py:18: from .conftest import run_recipe_ci, ...
|
||||||
|
!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!
|
||||||
|
```
|
||||||
|
Root cause: `tests/regression/__init__.py` and `tests/__init__.py` missing. Fix: add them or
|
||||||
|
use absolute imports (as other test files in this repo do).
|
||||||
|
|
||||||
|
**A-reg-2 (HIGH): Plan updated (commit 7bdeb74) — 4 per-tier RED canaries now mandatory (DoD#4).**
|
||||||
|
Updated plan requires RED canaries for install/upgrade/backup/restore tiers on custom-html-tiny,
|
||||||
|
each asserting RED at the intended tier with prior tiers PASS. Current suite: 3 canaries only
|
||||||
|
(2 good + 1 bad-custom-assertion). All four are MISSING. Cannot claim DONE without them.
|
||||||
|
|
||||||
|
**Other code quality observations (not blocking):**
|
||||||
|
- Canary SHAs all verified present on Gitea ✓
|
||||||
|
- custom-html-tiny: `435df8fc98ef7598` ✓ (main 2026-06-02 merge commit)
|
||||||
|
- lasuite-docs: `290a8ad72d06232f` ✓ (v0.3.3+v5.1.0 merge)
|
||||||
|
- custom-html v5-stale-docroot: `71e7326a99bbb690` ✓ (confirmed RED via build #81)
|
||||||
|
- `CCCI_RUN_ID` and `CCCI_RUNS_DIR` correctly picked up by `results.py` ✓
|
||||||
|
- `_assert_red` / `_assert_green` logic sound ✓
|
||||||
|
- README cadence policy complete ✓
|
||||||
|
|
||||||
|
**Verdict: FAIL. Standing issues: A-reg-1 (critical), A-reg-2 (high). Builder must fix both
|
||||||
|
before re-claiming this gate.**
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Adversary findings
|
## Adversary findings
|
||||||
|
|
||||||
*(None yet)*
|
*(See BACKLOG-regression.md § Adversary findings: A-reg-1, A-reg-2)*
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user