feat(harness): P4 — custom-test ergonomics (rcust)
All checks were successful
continuous-integration/drone/push Build is passing
All checks were successful
continuous-integration/drone/push Build is passing
Placement RULE: discovery.custom_tests covers ONLY functional/ + playwright/ — the top-level test_*.py glob for recipe dirs is removed (top level is reserved for lifecycle overlays; zero in-repo users of top-level custom tests, verified by sweep). Lifecycle-name exclusion inside the subdirs stays as the double-run safety net. HC2 default-deny unchanged (repo-local custom now pinned via functional/ in the gate test). New conftest fixture op_state: parses $CCCI_OP_STATE_FILE (op context: versions, artifact paths), skipping with a clear reason when unset/absent/unparseable — overlay tests read op facts from the fixture instead of hand-parsing env (zero existing hand-parsers found; the fixture is the documented path forward). deps fixture landed in P2d. Unit tests: placement-rule discovery tests (top-level custom NOT discovered; functional/playwright are; misfiled lifecycle names excluded), op_state fixture contract (reads file / skips without env / skips on missing file), deps fixture attribute sugar. Verified on cc-ci: cc-ci-run -m pytest tests/unit -q -> 184 passed; scripts/lint.sh -> PASS.
This commit is contained in:
@ -11,7 +11,8 @@ hook; the orchestrator decides additive-vs-skip. Sources, in precedence order
|
||||
> cc-ci tests/<recipe>/test_<op>.py
|
||||
(the generic tests/_generic/test_<op>.py is the always-present floor, run separately by default)
|
||||
|
||||
custom (non-lifecycle) test_*.py — ALL run, additively, from BOTH locations (opt-in).
|
||||
custom test_*.py (functional/ + playwright/ ONLY, rcust P4 placement rule) — ALL run,
|
||||
additively, from BOTH locations (opt-in).
|
||||
|
||||
install-steps hook — install_steps.sh: repo-local > cc-ci, or none.
|
||||
|
||||
@ -100,29 +101,22 @@ def resolve_op(recipe: str, op: str, repo_local_dir: str | None) -> tuple[str, s
|
||||
|
||||
|
||||
def custom_tests(recipe: str, repo_local_dir: str | None) -> list[tuple[str, str]]:
|
||||
"""All non-lifecycle test_*.py from cc-ci's tests/<recipe>/ and (if approved) the recipe's
|
||||
repo-local tests/. Discovered locations (Phase 2 §4.1):
|
||||
- the top-level dir tests/<recipe>/test_*.py (legacy + cross-cutting)
|
||||
- functional/ tests/<recipe>/functional/test_*.py (parity ports + recipe-specific)
|
||||
- playwright/ tests/<recipe>/playwright/test_*.py (UI flows P6)
|
||||
Files named `test_<op>.py` (lifecycle ops) are excluded from this list — the orchestrator runs
|
||||
those in their lifecycle tier, not the custom one. Repo-local is consulted only for
|
||||
allowlist-approved recipes (HC2)."""
|
||||
"""All custom-tier test_*.py from cc-ci's tests/<recipe>/ and (if approved) the recipe's
|
||||
repo-local tests/. PLACEMENT RULE (rcust P4): custom tests live ONLY under
|
||||
- functional/ tests/<recipe>/functional/test_*.py (parity ports + recipe-specific)
|
||||
- playwright/ tests/<recipe>/playwright/test_*.py (UI flows)
|
||||
A top-level test_*.py is a LIFECYCLE OVERLAY (test_<op>.py) and nothing else — top-level
|
||||
non-lifecycle files are NOT discovered (zero users at the time of the change; the lifecycle-
|
||||
name exclusion below stays as a safety net so a misfiled test_<op>.py can never double-run).
|
||||
Repo-local is consulted only for allowlist-approved recipes (HC2)."""
|
||||
lifecycle_names = {f"test_{op}.py" for op in LIFECYCLE_OPS}
|
||||
subdirs = ("functional", "playwright")
|
||||
found: list[tuple[str, str]] = []
|
||||
for source, d in (("cc-ci", cc_ci_dir(recipe)), ("repo-local", _gated(recipe, repo_local_dir))):
|
||||
if not d or not os.path.isdir(d):
|
||||
continue
|
||||
# top-level (legacy / cross-cutting tests not under functional/playwright)
|
||||
for p in sorted(glob.glob(os.path.join(d, "test_*.py"))):
|
||||
if os.path.basename(p) not in lifecycle_names:
|
||||
found.append((source, p))
|
||||
# functional/ and playwright/ subdirs (Phase 2 §4.1)
|
||||
for sub in subdirs:
|
||||
for p in sorted(glob.glob(os.path.join(d, sub, "test_*.py"))):
|
||||
# Phase-2 layout: lifecycle ops never live under functional/playwright, but be
|
||||
# explicit so a misfiled file doesn't silently get double-run.
|
||||
if os.path.basename(p) not in lifecycle_names:
|
||||
found.append((source, p))
|
||||
return found
|
||||
|
||||
Reference in New Issue
Block a user