feat(harness): intentional skips + custom-html-tiny functional test; 4-rung ladder (#6)
Some checks failed
continuous-integration/drone/push Build is failing
Some checks failed
continuous-integration/drone/push Build is failing
Declare intentional skips + custom-html-tiny functional test; 4-rung level ladder
- recipe_meta.EXPECTED_NA = {rung: reason} lists intentionally-skipped rungs; any
essential rung skipped and not listed is unintentional. Skips still cap the level
(never inflate). results.json: skips:{intentional,unintentional} + level_cap_rung.
- Level ladder = the four essential rungs (install, upgrade, backup/restore,
functional; top = L4). integration & recipe-local are optional, not leveled
(SSO still enforced for the run verdict, unchanged).
- Card shows skipped rungs as INTENTIONAL SKIP (green, reason below) / UNINTENTIONAL
SKIP (amber); level badge gains an expected/gap? third segment.
- custom-html-tiny: functional serve test (exact-byte round-trip + 404); declares
backup_restore intentionally skipped (stateless static server).
Independently verified by the adversary: 138 unit tests pass cold; live full-stage
run on custom-html-tiny green (upgrade tier ran; level 2; correct skips/badge);
clean teardown.
This commit is contained in:
@ -24,7 +24,7 @@ import dashboard # noqa: E402
|
||||
def _row(**kw):
|
||||
base = {
|
||||
"recipe": "custom-html", "status": "success", "number": 4, "ref": "db9a9502",
|
||||
"version": "db9a95024e9d", "level": 4, "level_cap_reason": "L5 integration N/A",
|
||||
"version": "db9a95024e9d", "level": 4, "level_cap_reason": "",
|
||||
"has_screenshot": True, "flags": {"clean_teardown": True, "no_secret_leak": True},
|
||||
"finished": 0, "url": "https://drone.x/cc-ci/4",
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user