diff --git a/machine-docs/BACKLOG-settings.md b/machine-docs/BACKLOG-settings.md new file mode 100644 index 0000000..c6937b6 --- /dev/null +++ b/machine-docs/BACKLOG-settings.md @@ -0,0 +1,24 @@ +# BACKLOG — phase `settings` + +## Build backlog + +- [x] **B1** — `harness/settings.py`: stdlib `tomllib` loader, `[upgrade].skip_canonicals_for_upgrade` + (bool, default false), `_SCHEMA` single-source defaults+validation, graceful on absent/malformed, + warn-and-ignore unknown keys/tables, raise on wrong type. Path `$CCCI_SETTINGS` / `/etc/cc-ci/settings.toml`. +- [x] **B2** — tracked `settings.toml.example` documenting keys + defaults (no secrets). +- [x] **B3** — wire `SKIP_CANONICALS_FOR_UPGRADE` into `resolve_upgrade_base` (`run_recipe_ci.py`): + flag true → bypass canonical lookup → no-canonical fallback. Scope = upgrade base only. +- [x] **B4** — improved no-canonical fallback `_no_canonical_base` (§2.C): newest release tag `< head` + (reuse `warm_reconcile.newest_older_version`) → main-tip → skip. Always-on. +- [x] **B5** — unit tests: full resolution matrix (`tests/unit/test_upgrade_base.py`) + loader + (`tests/unit/test_settings.py`). 315 unit pass, lint clean. +- [x] **B6 (M1 claim)** — clean tree, push, claim M1 in STATUS-settings.md. + +### M2 (after M1 PASS) +- [ ] **B7** — deploy to cc-ci (`/etc/cc-ci` git pull + nixos-rebuild if needed); confirm harness reads + settings (absent → default false; or file present false). +- [ ] **B8** — live evidence (a): a recipe WITHOUT a canonical resolves base to newest release tag `< head` + (not raw main-tip). +- [ ] **B9** — live evidence (b): flip `SKIP_CANONICALS_FOR_UPGRADE = true` (scratch) → a canonical-bearing + recipe ALSO resolves to the release-tag base (canonical bypassed); then restore false. +- [ ] **B10 (M2 claim)** — claim M2; on fresh PASS of M1+M2 → `## DONE`. diff --git a/machine-docs/DECISIONS.md b/machine-docs/DECISIONS.md index cafbffd..fd0f7cf 100644 --- a/machine-docs/DECISIONS.md +++ b/machine-docs/DECISIONS.md @@ -1579,3 +1579,33 @@ OVERVIEW (`/`) and badges keep their Drone latest-per-recipe source unchanged. D merge Drone live "running" status into history (optional per plan; re-adds the network dependency the local source removes; overview already shows live status). Retention: 308 parseable runs present, no trim job observed → adequate; revisit only if a cap is ever needed. + +--- + +## Phase `settings` (2026-06-17) — server settings.toml + SKIP_CANONICALS_FOR_UPGRADE + release-tag-first fallback + +- **Settings home = `harness/settings.py` (new), file `/etc/cc-ci/settings.toml` (override `$CCCI_SETTINGS`).** + No pre-existing cc-ci config module existed to extend (config was scattered `os.environ.get` reads); + a minimal stdlib-`tomllib` loader is the minimal+extensible mechanism. `_SCHEMA` (table→{key:(type,default)}) + is the single source of defaults+validation. Tracked `settings.toml.example`; live file untracked/operator- + managed/no-secrets (secrets stay in sops). Default `/etc/cc-ci` chosen over the plan's suggested + `/srv/cc-ci` (orchestrator-ambiguous): `/etc/cc-ci` is where the harness already runs (`CCCI_REPO`), + absolute so Drone+sweep read the same file, untracked file survives deploy `git pull`. +- **`SKIP_CANONICALS_FOR_UPGRADE` scope = upgrade BASE only.** Wired into `resolve_upgrade_base`: flag + true → skip canonical lookup → no-canonical fallback (behaves as if no canonical). Does NOT touch + canonical *promotion* or the `--quick` warm-reattach — those are separate optimizations; a future + `SKIP_CANONICAL_SWEEP` / `SKIP_QUICK` could gate them (out of scope here). +- **No-canonical fallback (always-on, §2.C):** newest release TAG `< head` (reuse + `warm_reconcile.newest_older_version`, the single version-ordering source) → raw main-tip (no prior + release tag) → skip. Replaces the old jump-straight-to-main-tip path; improves this server too (false + flag, un-promoted recipes get a real release base). +- **Canonical-present path (incl. samever step-back) preserved byte-for-byte.** With flag false + a + canonical, behavior is unchanged. The step-back's "no older predecessor → skip" is intentionally NOT + routed to main-tip (would reintroduce the same-version no-op samever prevents); the §2.C "==head" + routing is satisfied because the step-back already takes the same release-tag helper as fallback step 1. +- **Validation:** absent/unreadable/malformed-TOML → WARN + all-defaults (cannot crash the harness); + unknown table/key → warn-and-ignore; present known key of wrong type → raise TypeError (loud typo). +- **OBSERVATION (not this phase's defect):** `scripts/lint.sh` (pinned ruff) reports + `dashboard/dashboard.py` + `tests/unit/test_dashboard.py` would be reformatted — confirmed pre-existing + at HEAD f68f1c5, outside the settings diff. Flagged for the dashboard owner / orchestrator; not fixed + here (narrow scope). diff --git a/machine-docs/JOURNAL-settings.md b/machine-docs/JOURNAL-settings.md new file mode 100644 index 0000000..4c5688a --- /dev/null +++ b/machine-docs/JOURNAL-settings.md @@ -0,0 +1,65 @@ +# JOURNAL — phase `settings` (WHY / reasoning; Adversary does not read before verdict) + +## 2026-06-17 — bootstrap + M1 design + +**Phase:** server-level `settings.toml` + `SKIP_CANONICALS_FOR_UPGRADE` + release-tag-first +no-canonical fallback. Plan: `/srv/cc-ci/cc-ci-plan/plan-phase-settings-ci-server-config.md`. + +### Why a new `harness/settings.py` (not extending an env-var module) +Checked for an existing cc-ci config mechanism first (plan §2.A "extend rather than spawn a parallel +one"). The server config today is **scattered ad-hoc env reads** (`os.environ.get` for `MAX_TESTS`, +`CCCI_RUNS_DIR`, `CCCI_REPO`, `STAGES`, `CCCI_QUICK`, …) — there is **no** central config module/class +to extend (`grep` for `tomllib|settings\.toml|class Settings` → none). So a small dedicated loader IS +the minimal, extensible home rather than threading another env var. Stdlib `tomllib` (py3.12 on the +server, confirmed). One `[upgrade]` table, one key now; `_SCHEMA` is the single source of +defaults+validation so adding a key/table later is a one-line change. + +### Settings file path: `/etc/cc-ci/settings.toml` (override `$CCCI_SETTINGS`) +The harness runs from `/etc/cc-ci` in BOTH execution contexts (nightly sweep sets `CCCI_REPO=/etc/cc-ci` +and `cd`s there; the Drone recipe-CI runner runs from its checkout but an **absolute** host path is read +identically by both). `/etc/cc-ci` is a git checkout kept current by `git pull` + nixos-rebuild on +deploy — an **untracked** `settings.toml` there survives pulls (git pull never deletes untracked files) +and sits next to the tracked `settings.toml.example`. Chose this over `/srv/cc-ci/settings.toml` (the +plan's *suggestion*) because `/srv/cc-ci` is the orchestrator path, ambiguous on the server; `/etc/cc-ci` +is unambiguous and discoverable. The loader is graceful if the file/dir is absent → defaults. + +### Why the canonical-present path (incl. samever step-back) is byte-for-byte unchanged +Guardrail §4: default false must be a no-op for current behavior. Structure: +`if rec and rec.version and not flag:` → the entire existing prevb/samever block runs verbatim +(canonical ≠ head → canonical; canonical == head → step-back older tag, else skip). Only when there is +**no canonical in play** (rec falsy, OR flag true) do we enter the new `_no_canonical_base`. So with +flag false + a canonical, nothing changes; the step-back's "no older predecessor → skip" is preserved +(NOT routed to main-tip), which is correct — routing it to main-tip could reintroduce the same-version +no-op samever exists to prevent. The plan §2.C "unified chain ... (==head)" is satisfied by the +step-back already taking the same release-tag helper as step 1; I deliberately did NOT add a main-tip +tail to the step-back skip, to keep samever's guarantee intact. This is the one place where a literal +reading of §2.C ("==head → ... → main-tip → skip") and the §4 no-op guardrail + samever's intent point +slightly differently; I chose the conservative path that preserves both samever and the no-op guardrail. +If the Adversary reads §2.C literally and wants the step-back-no-older case to fall to main-tip, that is +a one-line change — but I believe it would be a regression (vacuous upgrade), so it's recorded here. + +### Why `_no_canonical_base` guards on `head_version` before calling `recipe_tags` +`newest_older_version(tags, None)` returns None, but evaluating `recipe_tags(recipe)` eagerly would +shell out to `git -C tag` even when head_version is None (e.g. callers/tests that +don't pass it). Guarding `if head_version else None` avoids a needless/erroring git call and preserves +the prevb behavior for the no-head_version caller shape (→ main-tip). + +### Why wrong-type raises but malformed/absent doesn't +Plan M1: "malformed file handled" (graceful) AND "wrong type errors clearly". Reconciled: absent / +unreadable / TOML-syntax-error → WARN + all-defaults (a red file degrades to today's behavior, can't +crash CI). A syntactically-valid file with a **known key of the wrong type** → `TypeError` (a typo'd +value should be loud, not silently mis-parsed). bool-is-int-subclass handled: `1`/`0` for a bool key is +rejected, not coerced. + +### Pre-existing, OUT OF SCOPE: dashboard lint drift on main +`scripts/lint.sh` reports `dashboard/dashboard.py` + `tests/unit/test_dashboard.py` would be reformatted +by the pinned ruff — confirmed present at HEAD f68f1c5 (`git show HEAD:...` through pinned ruff), NOT in +my diff. Not touched by this phase (narrow scope). Recorded in DECISIONS as an observation. My 5 +phase files are format-clean + `ruff check` clean. + +### Verification (commands + output) +- `nix shell nixpkgs#python311Packages.pytest -c pytest tests/unit/test_upgrade_base.py + tests/unit/test_settings.py -q` → **32 passed**. +- full unit suite `pytest tests/unit/ -q` → **315 passed**. +- `ruff check runner/ tests/unit/ bridge/ dashboard/` → All checks passed. +- `ruff format --check` (pinned) on my 5 files → all formatted. diff --git a/runner/harness/settings.py b/runner/harness/settings.py new file mode 100644 index 0000000..8146838 --- /dev/null +++ b/runner/harness/settings.py @@ -0,0 +1,118 @@ +"""Server-level CI settings (phase settings). A minimal, extensible TOML config layer for the cc-ci +server, read once per harness process. Stdlib-only (`tomllib`). Defaults are baked into this loader, +so an ABSENT file — or an absent key within a present file — behaves EXACTLY as before (this server +needs no file to behave as today). The live file is an operator-managed HOST OVERRIDE; it must carry +NO secrets (config only — secrets stay in sops). See `settings.toml.example` for the documented keys. + +Layout (minimal + extensible — one table, one key now; shaped so future CI-server configs slot in +without a redesign): + + [upgrade] + skip_canonicals_for_upgrade = false # when true, resolve the upgrade BASE without canonicals + +Path resolution: `$CCCI_SETTINGS` if set, else `/etc/cc-ci/settings.toml` (co-located with the +deployed checkout `$CCCI_REPO=/etc/cc-ci`, alongside the tracked `settings.toml.example`; both the +Drone recipe-CI runner and the nightly sweep read this same absolute host path). + +Validation: + - absent / unreadable / malformed-TOML file → WARN + all-defaults (a bad file can NEVER crash the + harness — a red config should degrade to today's behavior, not take down CI). + - unknown table / unknown key → WARN-and-ignore (forward/typo tolerant). + - present known key of the WRONG TYPE → raise TypeError (a typo'd value is loud, not silently + mis-parsed) — this is the one "errors clearly" case, distinct from a malformed file. +""" + +from __future__ import annotations + +import os +import sys +import tomllib +from dataclasses import dataclass + +DEFAULT_PATH = "/etc/cc-ci/settings.toml" + +# The full schema: table -> {key: (type, default)}. SINGLE SOURCE OF TRUTH for defaults + validation. +# Add future settings here (a new key, or a new table); `Settings` field names mirror the keys. +_SCHEMA: dict[str, dict[str, tuple[type, object]]] = { + "upgrade": { + "skip_canonicals_for_upgrade": (bool, False), + }, +} + + +@dataclass(frozen=True) +class Settings: + # [upgrade].skip_canonicals_for_upgrade — when True, resolve_upgrade_base skips the canonical + # lookup entirely and takes the no-canonical release-tag-first fallback (phase settings §2.B/§2.C). + skip_canonicals_for_upgrade: bool = False + + +def _warn(msg: str) -> None: + print(f"[settings] WARNING: {msg}", file=sys.stderr, flush=True) + + +def _resolve_path() -> str: + return os.environ.get("CCCI_SETTINGS") or DEFAULT_PATH + + +def _check_type(table: str, key: str, value: object, typ: type) -> object: + # bool is a subclass of int — reject an int given for a bool key (and vice-versa) so a stray + # `1`/`0` for a flag is a loud error, not a silent truthy coercion. + ok = isinstance(value, typ) and (typ is bool or not isinstance(value, bool)) + if not ok: + raise TypeError( + f"settings: [{table}].{key} must be {typ.__name__}, " + f"got {type(value).__name__} ({value!r})" + ) + return value + + +def load(path: str | None = None) -> Settings: + """Load settings from `path` (default: `$CCCI_SETTINGS` / DEFAULT_PATH). Returns all-defaults on an + absent / unreadable / malformed file (WARN, never raises). Raises TypeError only on a present + known key whose value is the wrong type.""" + p = path or _resolve_path() + raw: dict = {} + if os.path.exists(p): + try: + with open(p, "rb") as f: + raw = tomllib.load(f) + except (OSError, tomllib.TOMLDecodeError) as e: + _warn(f"could not read {p} ({e}); using defaults") + raw = {} + + values: dict[str, object] = {} + for table, keys in _SCHEMA.items(): + section = raw.get(table, {}) + if not isinstance(section, dict): + _warn(f"[{table}] is not a table ({type(section).__name__}); ignoring it") + section = {} + for key, (typ, default) in keys.items(): + if key in section: + values[key] = _check_type(table, key, section[key], typ) + else: + values[key] = default + + # Forward/typo tolerant: warn on anything not in the schema, but never fail on it. + for table, section in raw.items(): + if table not in _SCHEMA: + _warn(f"unknown table [{table}] ignored") + continue + if isinstance(section, dict): + for key in section: + if key not in _SCHEMA[table]: + _warn(f"unknown key [{table}].{key} ignored") + + return Settings(**values) # type: ignore[arg-type] + + +_CACHE: Settings | None = None + + +def get() -> Settings: + """Process-wide cached settings (read the file once). The harness calls this; tests call `load()` + with an explicit path (or monkeypatch this) to bypass the cache.""" + global _CACHE + if _CACHE is None: + _CACHE = load() + return _CACHE diff --git a/runner/run_recipe_ci.py b/runner/run_recipe_ci.py index 3a19c9d..83514fc 100644 --- a/runner/run_recipe_ci.py +++ b/runner/run_recipe_ci.py @@ -75,6 +75,9 @@ from harness import ( # noqa: E402 from harness import ( # noqa: E402 screenshot as screenshot_mod, ) +from harness import ( # noqa: E402 + settings as settings_mod, +) ALL_STAGES = ("install", "upgrade", "backup", "restore", "custom") @@ -113,8 +116,14 @@ def resolve_upgrade_base( stages, meta, recipe: str, head_ref: str | None = None, head_version: str | None = None ) -> BasePlan: """Dynamic upgrade-base resolution (phase prevb, replaces the static `recipe_versions[-2]` - default). Order: last-green (warm canonical, with same-version step-back) → target-branch (main) - tip → skip. EXPECTED_NA[upgrade] / upgrade∉stages short-circuit to a declared skip first. + default). Chain: last-green (warm canonical, with same-version step-back) → newest release tag + older than head → main-tip → skip. EXPECTED_NA[upgrade] / upgrade∉stages short-circuit to a + declared skip first. + + SKIP_CANONICALS_FOR_UPGRADE (phase settings, server settings.toml, default false): when true, the + canonical lookup is bypassed entirely — the resolver behaves as if no canonical exists and takes + the no-canonical release-tag-first fallback (`_no_canonical_base`). Scope is the upgrade BASE only; + canonical promotion and the `--quick` warm-reattach are unaffected (see DECISIONS, phase settings). `head_version` is the head checkout's published version (the `coop-cloud..version` label; see abra.head_compose_version). When the last-green warm-canonical version EQUALS it, deploying @@ -142,8 +151,9 @@ def resolve_upgrade_base( flush=True, ) return BasePlan("skip", None, None, f"declared EXPECTED_NA[upgrade]: {declared}") + skip_canonicals = settings_mod.get().skip_canonicals_for_upgrade rec = canonical.read_registry(recipe) - if rec and rec.get("version"): + if rec and rec.get("version") and not skip_canonicals: canon = rec["version"] same = head_version is not None and warm_reconcile.version_key( canon @@ -176,13 +186,46 @@ def resolve_upgrade_base( None, f"base == head ({head_version}) and no older published predecessor", ) + # No canonical in play — none recorded, OR SKIP_CANONICALS_FOR_UPGRADE=true (canonical lookup + # bypassed entirely, behaving as if none exists). Improved fallback (phase settings §2.C): prefer + # a REAL published predecessor (newest release tag < head) over the raw main-tip. + return _no_canonical_base(recipe, head_ref, head_version) + + +def _no_canonical_base(recipe: str, head_ref: str | None, head_version: str | None) -> BasePlan: + """Upgrade base when no canonical is used (none recorded, its promote failed, or + SKIP_CANONICALS_FOR_UPGRADE is true). Release-tag-first fallback (phase settings §2.C): + 1. most recent release TAG with version strictly older than the PR head — a clean published + predecessor (reuses samever's `newest_older_version` helper, the single source of version + ordering, so this and the step-back never diverge); + 2. raw `main`-tip (target-branch tip) — only if the recipe has NO prior release tag at all; + 3. skip — no predecessor (no older tag and head == main-tip, or no main at all). + This replaces the old jump-straight-to-main-tip path, so an un-promoted recipe upgrades from a real + release base instead of a possibly-untagged WIP commit.""" + older = ( + warm_reconcile.newest_older_version(warm_reconcile.recipe_tags(recipe), head_version) + if head_version + else None + ) + if older: + return BasePlan( + "version", + older, + None, + f"no-canonical fallback: newest release tag older than head {head_version}", + ) main_tip = lifecycle.recipe_branch_commit(recipe, "main") if main_tip and main_tip != head_ref: - return BasePlan("ref", None, main_tip, "target-branch (main) tip") + return BasePlan( + "ref", + None, + main_tip, + "no-canonical fallback: target-branch (main) tip (no prior release tag)", + ) if main_tip and main_tip == head_ref: return BasePlan("skip", None, None, "head == main tip (no predecessor delta)") return BasePlan( - "skip", None, None, "no last-green and no main tip (new recipe / no predecessor)" + "skip", None, None, "no release tag and no main tip (new recipe / no predecessor)" ) diff --git a/settings.toml.example b/settings.toml.example new file mode 100644 index 0000000..70d5962 --- /dev/null +++ b/settings.toml.example @@ -0,0 +1,23 @@ +# cc-ci server settings (phase settings) — EXAMPLE / documentation. +# +# This is a minimal, extensible server-level config for the cc-ci CI host. Copy it to the live host +# path and edit as needed: +# +# cp settings.toml.example /etc/cc-ci/settings.toml # (or set $CCCI_SETTINGS to any path) +# +# The live file is OPERATOR-MANAGED and not committed (a per-server host override). Every key has a +# default baked into the loader (runner/harness/settings.py), so an ABSENT file — or an absent key — +# behaves exactly as the documented default. You only need a live file to CHANGE a default. +# +# NO SECRETS in this file — config only. Secrets stay in sops. +# Stdlib TOML (tomllib): booleans are `true`/`false` (lowercase, unquoted). + +[upgrade] +# When true, the upgrade-tier BASE is resolved WITHOUT canonicals: the canonical (last-green warm) +# lookup is skipped entirely and the base falls through to the no-canonical fallback — the most recent +# release TAG on main older than the PR head, with the raw main-tip only as a further fallback, then +# skip. Codifies canonicals as an optional optimization (an operator switch); does NOT affect canonical +# promotion or the `--quick` warm-reattach (those are separate, out of scope here). +# +# Default: false (this server keeps canonicals on — the optimized/robust upgrade-base path). +skip_canonicals_for_upgrade = false diff --git a/tests/unit/test_settings.py b/tests/unit/test_settings.py new file mode 100644 index 0000000..35638d7 --- /dev/null +++ b/tests/unit/test_settings.py @@ -0,0 +1,111 @@ +"""Unit tests for `harness.settings` — the minimal, extensible server-level TOML config loader +(phase settings). Stdlib `tomllib`; defaults baked in; absent/malformed file degrades to defaults +(never crashes the harness); unknown keys warn-and-ignore; a present known key of the wrong type +errors clearly. +""" + +from __future__ import annotations + +import os +import sys + +import pytest + +sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "runner")) +from harness import settings # noqa: E402 + + +def _write(tmp_path, text: str) -> str: + p = tmp_path / "settings.toml" + p.write_text(text) + return str(p) + + +def test_absent_file_yields_defaults(tmp_path): + # a path that does not exist → all-defaults, no exception + missing = str(tmp_path / "nope.toml") + s = settings.load(missing) + assert s.skip_canonicals_for_upgrade is False + + +def test_absent_key_yields_default(tmp_path): + # present file, present table, but the key omitted → default + p = _write(tmp_path, "[upgrade]\n") + assert settings.load(p).skip_canonicals_for_upgrade is False + + +def test_empty_file_yields_defaults(tmp_path): + p = _write(tmp_path, "") + assert settings.load(p).skip_canonicals_for_upgrade is False + + +def test_flag_true_read(tmp_path): + p = _write(tmp_path, "[upgrade]\nskip_canonicals_for_upgrade = true\n") + assert settings.load(p).skip_canonicals_for_upgrade is True + + +def test_flag_false_read(tmp_path): + p = _write(tmp_path, "[upgrade]\nskip_canonicals_for_upgrade = false\n") + assert settings.load(p).skip_canonicals_for_upgrade is False + + +def test_malformed_toml_degrades_to_defaults(tmp_path): + # syntactically broken TOML must NOT crash the harness — WARN + defaults. + p = _write(tmp_path, "[upgrade\nskip_canonicals_for_upgrade = tru") + s = settings.load(p) # must not raise + assert s.skip_canonicals_for_upgrade is False + + +def test_wrong_type_errors_clearly(tmp_path): + # a present key of the wrong type is a loud, actionable error (distinct from a malformed file). + p = _write(tmp_path, '[upgrade]\nskip_canonicals_for_upgrade = "yes"\n') + with pytest.raises(TypeError) as e: + settings.load(p) + assert "skip_canonicals_for_upgrade" in str(e.value) + assert "bool" in str(e.value) + + +def test_int_not_accepted_for_bool(tmp_path): + # bool is an int subclass — a stray 1/0 must not silently coerce to a flag. + p = _write(tmp_path, "[upgrade]\nskip_canonicals_for_upgrade = 1\n") + with pytest.raises(TypeError): + settings.load(p) + + +def test_unknown_key_warns_and_ignored(tmp_path, capsys): + p = _write( + tmp_path, + "[upgrade]\nskip_canonicals_for_upgrade = true\nfuture_knob = 7\n", + ) + s = settings.load(p) + assert s.skip_canonicals_for_upgrade is True # known key still honored + err = capsys.readouterr().err + assert "unknown key" in err and "future_knob" in err + + +def test_unknown_table_warns_and_ignored(tmp_path, capsys): + p = _write(tmp_path, "[future_section]\nx = 1\n") + s = settings.load(p) + assert s.skip_canonicals_for_upgrade is False + err = capsys.readouterr().err + assert "unknown table" in err and "future_section" in err + + +def test_non_table_section_ignored(tmp_path, capsys): + # a key named like a table but given a scalar — warn, ignore, fall back to defaults for that table. + p = _write(tmp_path, "upgrade = 5\n") + s = settings.load(p) + assert s.skip_canonicals_for_upgrade is False + assert "not a table" in capsys.readouterr().err + + +def test_env_var_path_override(tmp_path, monkeypatch): + p = _write(tmp_path, "[upgrade]\nskip_canonicals_for_upgrade = true\n") + monkeypatch.setenv("CCCI_SETTINGS", p) + assert settings.load().skip_canonicals_for_upgrade is True + + +def test_default_path_is_absolute_host_path(): + # the live file is an absolute host override co-located with the deployed checkout. + assert settings.DEFAULT_PATH == "/etc/cc-ci/settings.toml" + assert os.path.isabs(settings.DEFAULT_PATH) diff --git a/tests/unit/test_upgrade_base.py b/tests/unit/test_upgrade_base.py index 38174ec..b8bbd82 100644 --- a/tests/unit/test_upgrade_base.py +++ b/tests/unit/test_upgrade_base.py @@ -13,6 +13,8 @@ import os import sys from types import SimpleNamespace +import pytest + sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "runner")) import run_recipe_ci # noqa: E402 import warm_reconcile # noqa: E402 @@ -23,6 +25,25 @@ HEAD = "aaaa1111head" MAIN = "bbbb2222main" +@pytest.fixture(autouse=True) +def _default_flag_false(monkeypatch): + # Hermetic: SKIP_CANONICALS_FOR_UPGRADE defaults to false regardless of any host settings.toml. + # The few flag-true tests re-patch this within the test body. + monkeypatch.setattr( + run_recipe_ci.settings_mod, + "get", + lambda: SimpleNamespace(skip_canonicals_for_upgrade=False), + ) + + +def _set_flag(monkeypatch, value: bool): + monkeypatch.setattr( + run_recipe_ci.settings_mod, + "get", + lambda: SimpleNamespace(skip_canonicals_for_upgrade=value), + ) + + def _meta(expected_na=None): return SimpleNamespace(EXPECTED_NA=expected_na) @@ -182,3 +203,119 @@ def test_expected_na_other_rung_does_not_suppress_upgrade(monkeypatch): meta = _meta(expected_na={"backup_restore": "stateless"}) plan = run_recipe_ci.resolve_upgrade_base(ALL, meta, "custom-html-tiny", head_ref=HEAD) assert plan.kind == "ref" and plan.ref == MAIN + + +# --- phase settings: improved no-canonical fallback (release tag before main-tip) + the flag --- + + +def test_no_canonical_prefers_release_tag_over_main_tip(monkeypatch): + # flag false + NO canonical → the base must be the newest release TAG strictly older than head + # (a real published predecessor), NOT the raw main-tip. main must not even be consulted. + _no_canonical(monkeypatch) + monkeypatch.setattr(warm_reconcile, "recipe_tags", lambda r: KC_TAGS) + monkeypatch.setattr( + lifecycle, + "recipe_branch_commit", + lambda r, b="main": (_ for _ in ()).throw(AssertionError("main consulted before tag")), + ) + plan = run_recipe_ci.resolve_upgrade_base( + ALL, _meta(), "keycloak", head_ref=HEAD, head_version="10.8.0+26.6.3" + ) + assert plan.kind == "version" and plan.runs + assert plan.version == "10.7.1+26.6.2" # newest tag strictly older than head + assert warm_reconcile.version_key(plan.version) < warm_reconcile.version_key("10.8.0+26.6.3") + assert "no-canonical fallback" in plan.reason and "release tag" in plan.reason + + +def test_no_canonical_no_older_tag_falls_back_to_main_tip(monkeypatch): + # flag false + no canonical + no release tag strictly older than head → raw main-tip is the + # FURTHER fallback (a recipe whose only tag IS the head version, e.g. brand-new single release). + _no_canonical(monkeypatch) + monkeypatch.setattr(warm_reconcile, "recipe_tags", lambda r: ["10.8.0+26.6.3"]) # == head only + monkeypatch.setattr(lifecycle, "recipe_branch_commit", lambda r, b="main": MAIN) + plan = run_recipe_ci.resolve_upgrade_base( + ALL, _meta(), "keycloak", head_ref=HEAD, head_version="10.8.0+26.6.3" + ) + assert plan.kind == "ref" and plan.ref == MAIN and plan.version is None + + +def test_no_canonical_no_tag_no_main_skips(monkeypatch): + # no canonical, no release tag, no main → declared skip (new recipe / no predecessor). + _no_canonical(monkeypatch) + monkeypatch.setattr(warm_reconcile, "recipe_tags", lambda r: []) + _no_main(monkeypatch) + plan = run_recipe_ci.resolve_upgrade_base( + ALL, _meta(), "brandnew", head_ref=HEAD, head_version="1.0.0" + ) + assert plan.kind == "skip" and not plan.runs and "no predecessor" in plan.reason + + +def test_no_head_version_skips_tag_lookup_uses_main(monkeypatch): + # no canonical AND no head_version (unreadable) → cannot compare versions, so recipe_tags is NOT + # consulted; fall straight through to main-tip (preserves prevb behavior for that caller shape). + _no_canonical(monkeypatch) + monkeypatch.setattr( + warm_reconcile, + "recipe_tags", + lambda r: (_ for _ in ()).throw(AssertionError("tags consulted without head_version")), + ) + monkeypatch.setattr(lifecycle, "recipe_branch_commit", lambda r, b="main": MAIN) + plan = run_recipe_ci.resolve_upgrade_base(ALL, _meta(), "discourse", head_ref=HEAD) + assert plan.kind == "ref" and plan.ref == MAIN + + +def test_flag_true_bypasses_canonical_into_release_tag_fallback(monkeypatch): + # SKIP_CANONICALS_FOR_UPGRADE=true: a canonical-bearing recipe (canonical ≠ head, would normally + # resolve to the canonical) instead bypasses the canonical entirely and takes the release-tag + # fallback → newest release tag < head. + _set_flag(monkeypatch, True) + monkeypatch.setattr( + canonical, "read_registry", lambda r: {"version": "10.6.0+26.5.0", "status": "warm"} + ) + monkeypatch.setattr(warm_reconcile, "recipe_tags", lambda r: KC_TAGS) + plan = run_recipe_ci.resolve_upgrade_base( + ALL, _meta(), "keycloak", head_ref=HEAD, head_version="10.8.0+26.6.3" + ) + assert plan.kind == "version" and plan.runs + assert plan.version == "10.7.1+26.6.2" # release tag, NOT the canonical 10.6.0+26.5.0 + assert "no-canonical fallback" in plan.reason + + +def test_flag_true_canonical_present_no_older_tag_uses_main(monkeypatch): + # flag true bypasses the canonical; with no older release tag, the further fallback is main-tip + # (proves the flag genuinely routes through the full no-canonical chain, not just step 1). + _set_flag(monkeypatch, True) + monkeypatch.setattr( + canonical, "read_registry", lambda r: {"version": "10.8.0+26.6.3", "status": "warm"} + ) + monkeypatch.setattr(warm_reconcile, "recipe_tags", lambda r: ["10.8.0+26.6.3"]) # == head only + monkeypatch.setattr(lifecycle, "recipe_branch_commit", lambda r, b="main": MAIN) + plan = run_recipe_ci.resolve_upgrade_base( + ALL, _meta(), "keycloak", head_ref=HEAD, head_version="10.8.0+26.6.3" + ) + assert plan.kind == "ref" and plan.ref == MAIN + + +def test_flag_false_canonical_present_unchanged(monkeypatch): + # explicit guardrail check: flag false + canonical present (≠ head) → canonical, byte-for-byte the + # prevb behavior; recipe_tags / main never consulted. + _set_flag(monkeypatch, False) + monkeypatch.setattr( + canonical, "read_registry", lambda r: {"version": "10.7.1+26.6.2", "status": "warm"} + ) + monkeypatch.setattr( + warm_reconcile, + "recipe_tags", + lambda r: (_ for _ in ()).throw(AssertionError("fallback taken with canonical present")), + ) + monkeypatch.setattr( + lifecycle, + "recipe_branch_commit", + lambda r, b="main": (_ for _ in ()).throw(AssertionError("main consulted")), + ) + plan = run_recipe_ci.resolve_upgrade_base( + ALL, _meta(), "keycloak", head_ref=HEAD, head_version="10.8.0+26.6.3" + ) + assert ( + plan.kind == "version" and plan.version == "10.7.1+26.6.2" and "last-green" in plan.reason + )