decisions(lvl5): level-semantics de-cap record, N/A classification table, lint mirror-context decision
All checks were successful
continuous-integration/drone/push Build is passing
All checks were successful
continuous-integration/drone/push Build is passing
This commit is contained in:
@ -1295,3 +1295,61 @@ the abra CLI and abra.recipe_dir()). No test assertion, gate, or overlay content
|
||||
phase guardrail's "never touch tests/<recipe>/ content" is read as protecting test/gate SEMANTICS;
|
||||
this is required P3 fallout, equivalent to the harness-side path routing. Flagged here for the
|
||||
Adversary's gate-integrity review.
|
||||
|
||||
## Phase lvl5 — L5 lint rung + level semantics de-cap (SETTLED 2026-06-11, operator-specified)
|
||||
|
||||
**The level formula (replaces the Phase-3 "N/A caps" stance).** Operator decision 2026-06-11
|
||||
(explicit Q&A, recorded verbatim in plan-phase-lvl5-lint-rung.md): with per-rung statuses
|
||||
{pass, fail, skip (intentional), unver (unintentional/not-verified)}:
|
||||
|
||||
level = max i such that rung_i == "pass" and all j < i have status in {"pass","skip"}; else 0.
|
||||
|
||||
A real FAIL blocks. An INTENTIONAL skip (the rung genuinely does not apply, from a declared or
|
||||
structural fact) is climbed past — this is the de-cap: a non-backup-capable recipe is no longer
|
||||
stuck at L2. An UNVERIFIED rung (should have run, wasn't checked) blocks exactly like a fail —
|
||||
this preserves the honest core of the old N/A-caps rule: never claim what wasn't checked. The
|
||||
words cap/capped/cap_reason are deleted from code, schema (results.json schema 2), card,
|
||||
dashboard, badge and docs; the per-rung table (✔/✘/intentional-skip/unverified) is the SOLE
|
||||
carrier of "why isn't the level higher". The big level badges (card corner, dashboard pill,
|
||||
/badge/<recipe>.svg) show ONLY number + colour (operator-specified). Old schema-1 artifacts are
|
||||
rendered as-is (their stored level, their 4-rung ladder) — no retroactive relabeling.
|
||||
|
||||
**The ladder is now five rungs:** install(1) upgrade(2) backup_restore(3) functional(4)
|
||||
**lint(5) = `abra recipe lint` passes against the exact ref under test** (PR head on PR builds).
|
||||
Lint is a LEVEL RUNG, not a run gate: no lint outcome ever changes the run verdict.
|
||||
|
||||
**N/A classification table (derive_rungs, results.py — every N/A source, Adversary-reviewed).
|
||||
Default for anything unclassifiable: UNVER (conservative).**
|
||||
|
||||
| rung | source of non-pass/fail | class | status |
|
||||
|---|---|---|---|
|
||||
| install | tier skipped / missing (any reason — install always applies) | unintentional | unver |
|
||||
| upgrade | tier skipped by orchestrator AND no upgrade target (`prev is None`: only one published version — structural) | intentional | skip |
|
||||
| upgrade | declared `EXPECTED_NA["upgrade"]` (tier not pass/fail) | intentional | skip |
|
||||
| upgrade | tier skipped though a target exists (install failed → downstream abort), or tier missing (CCCI_STAGES dev escape) | unintentional | unver |
|
||||
| backup_restore | not backup-capable (no backupbot labels / `BACKUP_CAPABLE=False` — structural/declared) | intentional | skip |
|
||||
| backup_restore | declared `EXPECTED_NA["backup_restore"]` (tiers not pass/fail) | intentional | skip |
|
||||
| backup_restore | backup-capable but either tier did not produce pass/fail (abort, partial run) | unintentional | unver |
|
||||
| functional | declared `EXPECTED_NA["functional"]` (no custom tests / tier skipped) | intentional | skip |
|
||||
| functional | no custom tests / tier skipped, undeclared — absent functional coverage is a GAP, not a property | unintentional | unver |
|
||||
| lint | executor could not produce pass/fail (timeout, abra/script missing, env FATA, unparseable output) — NO escape hatch, `EXPECTED_NA["lint"]` is ignored | unintentional | unver |
|
||||
|
||||
EXPECTED_NA never overrides an exercised rung: pass/fail always stand.
|
||||
|
||||
**Lint executor mirror-context decision (plan-phase-lvl5 §2.3).** Probed on cc-ci 2026-06-11
|
||||
(JOURNAL-lvl5): (a) abra lint globs every `compose*.yml` in the recipe tree, so the CI's
|
||||
untracked install_steps overlays (e.g. compose.ccci.yml) FATA it — harness artifact; (b) abra
|
||||
lint force-fetches tags from `origin`, so a PR run's private-mirror origin (token never written
|
||||
to .git/config) FATAs "unable to fetch tags" — harness artifact; (c) `abra recipe lint` exits
|
||||
non-zero ONLY on FATA — rule verdicts live in its table (error-severity ❌ rows + a trailing
|
||||
"WARN critical errors present" sentinel, rc still 0). Decision: the executor (harness/lint.py)
|
||||
lints a PRISTINE SCRATCH CLONE of the per-run recipe tree checked out at the exact tested sha —
|
||||
origin becomes a local path (offline tag fetch, no auth) and the run's true tag set rides along
|
||||
(fetch_recipe already fetches the canonical upstream version tags into the per-run tree, so
|
||||
R014 evaluates the recipe's real tags). **No lint rule is filtered or ignored** — the
|
||||
plumbing pollution is solved by context, not by exemptions. Classifier: fail iff an
|
||||
error-severity rule is unsatisfied (or the FATA is content-attributable: "unable to validate
|
||||
recipe"); pass iff the table rendered clean; anything else unver + loud log. Hard 60s budget
|
||||
(observed ~0.7s); executor runs before the tiers (tree at tested ref), double-wrapped, R7
|
||||
verdict-neutral. Full output → run artifact `lint.txt` (dashboard-served); status + failing
|
||||
rule ids → results.json `lint`.
|
||||
|
||||
Reference in New Issue
Block a user