feat(settings): server settings.toml loader + SKIP_CANONICALS_FOR_UPGRADE + release-tag-first no-canonical fallback
Some checks failed
continuous-integration/drone/push Build is failing
Some checks failed
continuous-integration/drone/push Build is failing
- harness/settings.py: stdlib tomllib loader, [upgrade].skip_canonicals_for_upgrade (bool, default false), _SCHEMA single-source defaults+validation; graceful on absent/malformed (WARN+defaults), warn-and-ignore unknown keys/tables, TypeError on wrong type. Path $CCCI_SETTINGS / /etc/cc-ci/settings.toml. + tracked settings.toml.example. - resolve_upgrade_base: flag true bypasses the canonical lookup -> no-canonical fallback; canonical-present path (incl. samever step-back) unchanged when false. - _no_canonical_base (always-on, §2.C): newest release tag < head (reuse warm_reconcile.newest_older_version) -> main-tip -> skip; replaces jump-to-main-tip. - unit: full resolution matrix + loader tests; 315 unit pass, ruff clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
24
machine-docs/BACKLOG-settings.md
Normal file
24
machine-docs/BACKLOG-settings.md
Normal file
@ -0,0 +1,24 @@
|
||||
# BACKLOG — phase `settings`
|
||||
|
||||
## Build backlog
|
||||
|
||||
- [x] **B1** — `harness/settings.py`: stdlib `tomllib` loader, `[upgrade].skip_canonicals_for_upgrade`
|
||||
(bool, default false), `_SCHEMA` single-source defaults+validation, graceful on absent/malformed,
|
||||
warn-and-ignore unknown keys/tables, raise on wrong type. Path `$CCCI_SETTINGS` / `/etc/cc-ci/settings.toml`.
|
||||
- [x] **B2** — tracked `settings.toml.example` documenting keys + defaults (no secrets).
|
||||
- [x] **B3** — wire `SKIP_CANONICALS_FOR_UPGRADE` into `resolve_upgrade_base` (`run_recipe_ci.py`):
|
||||
flag true → bypass canonical lookup → no-canonical fallback. Scope = upgrade base only.
|
||||
- [x] **B4** — improved no-canonical fallback `_no_canonical_base` (§2.C): newest release tag `< head`
|
||||
(reuse `warm_reconcile.newest_older_version`) → main-tip → skip. Always-on.
|
||||
- [x] **B5** — unit tests: full resolution matrix (`tests/unit/test_upgrade_base.py`) + loader
|
||||
(`tests/unit/test_settings.py`). 315 unit pass, lint clean.
|
||||
- [x] **B6 (M1 claim)** — clean tree, push, claim M1 in STATUS-settings.md.
|
||||
|
||||
### M2 (after M1 PASS)
|
||||
- [ ] **B7** — deploy to cc-ci (`/etc/cc-ci` git pull + nixos-rebuild if needed); confirm harness reads
|
||||
settings (absent → default false; or file present false).
|
||||
- [ ] **B8** — live evidence (a): a recipe WITHOUT a canonical resolves base to newest release tag `< head`
|
||||
(not raw main-tip).
|
||||
- [ ] **B9** — live evidence (b): flip `SKIP_CANONICALS_FOR_UPGRADE = true` (scratch) → a canonical-bearing
|
||||
recipe ALSO resolves to the release-tag base (canonical bypassed); then restore false.
|
||||
- [ ] **B10 (M2 claim)** — claim M2; on fresh PASS of M1+M2 → `## DONE`.
|
||||
@ -1579,3 +1579,33 @@ OVERVIEW (`/`) and badges keep their Drone latest-per-recipe source unchanged. D
|
||||
merge Drone live "running" status into history (optional per plan; re-adds the network dependency the
|
||||
local source removes; overview already shows live status). Retention: 308 parseable runs present, no
|
||||
trim job observed → adequate; revisit only if a cap is ever needed.
|
||||
|
||||
---
|
||||
|
||||
## Phase `settings` (2026-06-17) — server settings.toml + SKIP_CANONICALS_FOR_UPGRADE + release-tag-first fallback
|
||||
|
||||
- **Settings home = `harness/settings.py` (new), file `/etc/cc-ci/settings.toml` (override `$CCCI_SETTINGS`).**
|
||||
No pre-existing cc-ci config module existed to extend (config was scattered `os.environ.get` reads);
|
||||
a minimal stdlib-`tomllib` loader is the minimal+extensible mechanism. `_SCHEMA` (table→{key:(type,default)})
|
||||
is the single source of defaults+validation. Tracked `settings.toml.example`; live file untracked/operator-
|
||||
managed/no-secrets (secrets stay in sops). Default `/etc/cc-ci` chosen over the plan's suggested
|
||||
`/srv/cc-ci` (orchestrator-ambiguous): `/etc/cc-ci` is where the harness already runs (`CCCI_REPO`),
|
||||
absolute so Drone+sweep read the same file, untracked file survives deploy `git pull`.
|
||||
- **`SKIP_CANONICALS_FOR_UPGRADE` scope = upgrade BASE only.** Wired into `resolve_upgrade_base`: flag
|
||||
true → skip canonical lookup → no-canonical fallback (behaves as if no canonical). Does NOT touch
|
||||
canonical *promotion* or the `--quick` warm-reattach — those are separate optimizations; a future
|
||||
`SKIP_CANONICAL_SWEEP` / `SKIP_QUICK` could gate them (out of scope here).
|
||||
- **No-canonical fallback (always-on, §2.C):** newest release TAG `< head` (reuse
|
||||
`warm_reconcile.newest_older_version`, the single version-ordering source) → raw main-tip (no prior
|
||||
release tag) → skip. Replaces the old jump-straight-to-main-tip path; improves this server too (false
|
||||
flag, un-promoted recipes get a real release base).
|
||||
- **Canonical-present path (incl. samever step-back) preserved byte-for-byte.** With flag false + a
|
||||
canonical, behavior is unchanged. The step-back's "no older predecessor → skip" is intentionally NOT
|
||||
routed to main-tip (would reintroduce the same-version no-op samever prevents); the §2.C "==head"
|
||||
routing is satisfied because the step-back already takes the same release-tag helper as fallback step 1.
|
||||
- **Validation:** absent/unreadable/malformed-TOML → WARN + all-defaults (cannot crash the harness);
|
||||
unknown table/key → warn-and-ignore; present known key of wrong type → raise TypeError (loud typo).
|
||||
- **OBSERVATION (not this phase's defect):** `scripts/lint.sh` (pinned ruff) reports
|
||||
`dashboard/dashboard.py` + `tests/unit/test_dashboard.py` would be reformatted — confirmed pre-existing
|
||||
at HEAD f68f1c5, outside the settings diff. Flagged for the dashboard owner / orchestrator; not fixed
|
||||
here (narrow scope).
|
||||
|
||||
65
machine-docs/JOURNAL-settings.md
Normal file
65
machine-docs/JOURNAL-settings.md
Normal file
@ -0,0 +1,65 @@
|
||||
# JOURNAL — phase `settings` (WHY / reasoning; Adversary does not read before verdict)
|
||||
|
||||
## 2026-06-17 — bootstrap + M1 design
|
||||
|
||||
**Phase:** server-level `settings.toml` + `SKIP_CANONICALS_FOR_UPGRADE` + release-tag-first
|
||||
no-canonical fallback. Plan: `/srv/cc-ci/cc-ci-plan/plan-phase-settings-ci-server-config.md`.
|
||||
|
||||
### Why a new `harness/settings.py` (not extending an env-var module)
|
||||
Checked for an existing cc-ci config mechanism first (plan §2.A "extend rather than spawn a parallel
|
||||
one"). The server config today is **scattered ad-hoc env reads** (`os.environ.get` for `MAX_TESTS`,
|
||||
`CCCI_RUNS_DIR`, `CCCI_REPO`, `STAGES`, `CCCI_QUICK`, …) — there is **no** central config module/class
|
||||
to extend (`grep` for `tomllib|settings\.toml|class Settings` → none). So a small dedicated loader IS
|
||||
the minimal, extensible home rather than threading another env var. Stdlib `tomllib` (py3.12 on the
|
||||
server, confirmed). One `[upgrade]` table, one key now; `_SCHEMA` is the single source of
|
||||
defaults+validation so adding a key/table later is a one-line change.
|
||||
|
||||
### Settings file path: `/etc/cc-ci/settings.toml` (override `$CCCI_SETTINGS`)
|
||||
The harness runs from `/etc/cc-ci` in BOTH execution contexts (nightly sweep sets `CCCI_REPO=/etc/cc-ci`
|
||||
and `cd`s there; the Drone recipe-CI runner runs from its checkout but an **absolute** host path is read
|
||||
identically by both). `/etc/cc-ci` is a git checkout kept current by `git pull` + nixos-rebuild on
|
||||
deploy — an **untracked** `settings.toml` there survives pulls (git pull never deletes untracked files)
|
||||
and sits next to the tracked `settings.toml.example`. Chose this over `/srv/cc-ci/settings.toml` (the
|
||||
plan's *suggestion*) because `/srv/cc-ci` is the orchestrator path, ambiguous on the server; `/etc/cc-ci`
|
||||
is unambiguous and discoverable. The loader is graceful if the file/dir is absent → defaults.
|
||||
|
||||
### Why the canonical-present path (incl. samever step-back) is byte-for-byte unchanged
|
||||
Guardrail §4: default false must be a no-op for current behavior. Structure:
|
||||
`if rec and rec.version and not flag:` → the entire existing prevb/samever block runs verbatim
|
||||
(canonical ≠ head → canonical; canonical == head → step-back older tag, else skip). Only when there is
|
||||
**no canonical in play** (rec falsy, OR flag true) do we enter the new `_no_canonical_base`. So with
|
||||
flag false + a canonical, nothing changes; the step-back's "no older predecessor → skip" is preserved
|
||||
(NOT routed to main-tip), which is correct — routing it to main-tip could reintroduce the same-version
|
||||
no-op samever exists to prevent. The plan §2.C "unified chain ... (==head)" is satisfied by the
|
||||
step-back already taking the same release-tag helper as step 1; I deliberately did NOT add a main-tip
|
||||
tail to the step-back skip, to keep samever's guarantee intact. This is the one place where a literal
|
||||
reading of §2.C ("==head → ... → main-tip → skip") and the §4 no-op guardrail + samever's intent point
|
||||
slightly differently; I chose the conservative path that preserves both samever and the no-op guardrail.
|
||||
If the Adversary reads §2.C literally and wants the step-back-no-older case to fall to main-tip, that is
|
||||
a one-line change — but I believe it would be a regression (vacuous upgrade), so it's recorded here.
|
||||
|
||||
### Why `_no_canonical_base` guards on `head_version` before calling `recipe_tags`
|
||||
`newest_older_version(tags, None)` returns None, but evaluating `recipe_tags(recipe)` eagerly would
|
||||
shell out to `git -C <per-run recipe dir> tag` even when head_version is None (e.g. callers/tests that
|
||||
don't pass it). Guarding `if head_version else None` avoids a needless/erroring git call and preserves
|
||||
the prevb behavior for the no-head_version caller shape (→ main-tip).
|
||||
|
||||
### Why wrong-type raises but malformed/absent doesn't
|
||||
Plan M1: "malformed file handled" (graceful) AND "wrong type errors clearly". Reconciled: absent /
|
||||
unreadable / TOML-syntax-error → WARN + all-defaults (a red file degrades to today's behavior, can't
|
||||
crash CI). A syntactically-valid file with a **known key of the wrong type** → `TypeError` (a typo'd
|
||||
value should be loud, not silently mis-parsed). bool-is-int-subclass handled: `1`/`0` for a bool key is
|
||||
rejected, not coerced.
|
||||
|
||||
### Pre-existing, OUT OF SCOPE: dashboard lint drift on main
|
||||
`scripts/lint.sh` reports `dashboard/dashboard.py` + `tests/unit/test_dashboard.py` would be reformatted
|
||||
by the pinned ruff — confirmed present at HEAD f68f1c5 (`git show HEAD:...` through pinned ruff), NOT in
|
||||
my diff. Not touched by this phase (narrow scope). Recorded in DECISIONS as an observation. My 5
|
||||
phase files are format-clean + `ruff check` clean.
|
||||
|
||||
### Verification (commands + output)
|
||||
- `nix shell nixpkgs#python311Packages.pytest -c pytest tests/unit/test_upgrade_base.py
|
||||
tests/unit/test_settings.py -q` → **32 passed**.
|
||||
- full unit suite `pytest tests/unit/ -q` → **315 passed**.
|
||||
- `ruff check runner/ tests/unit/ bridge/ dashboard/` → All checks passed.
|
||||
- `ruff format --check` (pinned) on my 5 files → all formatted.
|
||||
118
runner/harness/settings.py
Normal file
118
runner/harness/settings.py
Normal file
@ -0,0 +1,118 @@
|
||||
"""Server-level CI settings (phase settings). A minimal, extensible TOML config layer for the cc-ci
|
||||
server, read once per harness process. Stdlib-only (`tomllib`). Defaults are baked into this loader,
|
||||
so an ABSENT file — or an absent key within a present file — behaves EXACTLY as before (this server
|
||||
needs no file to behave as today). The live file is an operator-managed HOST OVERRIDE; it must carry
|
||||
NO secrets (config only — secrets stay in sops). See `settings.toml.example` for the documented keys.
|
||||
|
||||
Layout (minimal + extensible — one table, one key now; shaped so future CI-server configs slot in
|
||||
without a redesign):
|
||||
|
||||
[upgrade]
|
||||
skip_canonicals_for_upgrade = false # when true, resolve the upgrade BASE without canonicals
|
||||
|
||||
Path resolution: `$CCCI_SETTINGS` if set, else `/etc/cc-ci/settings.toml` (co-located with the
|
||||
deployed checkout `$CCCI_REPO=/etc/cc-ci`, alongside the tracked `settings.toml.example`; both the
|
||||
Drone recipe-CI runner and the nightly sweep read this same absolute host path).
|
||||
|
||||
Validation:
|
||||
- absent / unreadable / malformed-TOML file → WARN + all-defaults (a bad file can NEVER crash the
|
||||
harness — a red config should degrade to today's behavior, not take down CI).
|
||||
- unknown table / unknown key → WARN-and-ignore (forward/typo tolerant).
|
||||
- present known key of the WRONG TYPE → raise TypeError (a typo'd value is loud, not silently
|
||||
mis-parsed) — this is the one "errors clearly" case, distinct from a malformed file.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
import tomllib
|
||||
from dataclasses import dataclass
|
||||
|
||||
DEFAULT_PATH = "/etc/cc-ci/settings.toml"
|
||||
|
||||
# The full schema: table -> {key: (type, default)}. SINGLE SOURCE OF TRUTH for defaults + validation.
|
||||
# Add future settings here (a new key, or a new table); `Settings` field names mirror the keys.
|
||||
_SCHEMA: dict[str, dict[str, tuple[type, object]]] = {
|
||||
"upgrade": {
|
||||
"skip_canonicals_for_upgrade": (bool, False),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Settings:
|
||||
# [upgrade].skip_canonicals_for_upgrade — when True, resolve_upgrade_base skips the canonical
|
||||
# lookup entirely and takes the no-canonical release-tag-first fallback (phase settings §2.B/§2.C).
|
||||
skip_canonicals_for_upgrade: bool = False
|
||||
|
||||
|
||||
def _warn(msg: str) -> None:
|
||||
print(f"[settings] WARNING: {msg}", file=sys.stderr, flush=True)
|
||||
|
||||
|
||||
def _resolve_path() -> str:
|
||||
return os.environ.get("CCCI_SETTINGS") or DEFAULT_PATH
|
||||
|
||||
|
||||
def _check_type(table: str, key: str, value: object, typ: type) -> object:
|
||||
# bool is a subclass of int — reject an int given for a bool key (and vice-versa) so a stray
|
||||
# `1`/`0` for a flag is a loud error, not a silent truthy coercion.
|
||||
ok = isinstance(value, typ) and (typ is bool or not isinstance(value, bool))
|
||||
if not ok:
|
||||
raise TypeError(
|
||||
f"settings: [{table}].{key} must be {typ.__name__}, "
|
||||
f"got {type(value).__name__} ({value!r})"
|
||||
)
|
||||
return value
|
||||
|
||||
|
||||
def load(path: str | None = None) -> Settings:
|
||||
"""Load settings from `path` (default: `$CCCI_SETTINGS` / DEFAULT_PATH). Returns all-defaults on an
|
||||
absent / unreadable / malformed file (WARN, never raises). Raises TypeError only on a present
|
||||
known key whose value is the wrong type."""
|
||||
p = path or _resolve_path()
|
||||
raw: dict = {}
|
||||
if os.path.exists(p):
|
||||
try:
|
||||
with open(p, "rb") as f:
|
||||
raw = tomllib.load(f)
|
||||
except (OSError, tomllib.TOMLDecodeError) as e:
|
||||
_warn(f"could not read {p} ({e}); using defaults")
|
||||
raw = {}
|
||||
|
||||
values: dict[str, object] = {}
|
||||
for table, keys in _SCHEMA.items():
|
||||
section = raw.get(table, {})
|
||||
if not isinstance(section, dict):
|
||||
_warn(f"[{table}] is not a table ({type(section).__name__}); ignoring it")
|
||||
section = {}
|
||||
for key, (typ, default) in keys.items():
|
||||
if key in section:
|
||||
values[key] = _check_type(table, key, section[key], typ)
|
||||
else:
|
||||
values[key] = default
|
||||
|
||||
# Forward/typo tolerant: warn on anything not in the schema, but never fail on it.
|
||||
for table, section in raw.items():
|
||||
if table not in _SCHEMA:
|
||||
_warn(f"unknown table [{table}] ignored")
|
||||
continue
|
||||
if isinstance(section, dict):
|
||||
for key in section:
|
||||
if key not in _SCHEMA[table]:
|
||||
_warn(f"unknown key [{table}].{key} ignored")
|
||||
|
||||
return Settings(**values) # type: ignore[arg-type]
|
||||
|
||||
|
||||
_CACHE: Settings | None = None
|
||||
|
||||
|
||||
def get() -> Settings:
|
||||
"""Process-wide cached settings (read the file once). The harness calls this; tests call `load()`
|
||||
with an explicit path (or monkeypatch this) to bypass the cache."""
|
||||
global _CACHE
|
||||
if _CACHE is None:
|
||||
_CACHE = load()
|
||||
return _CACHE
|
||||
@ -75,6 +75,9 @@ from harness import ( # noqa: E402
|
||||
from harness import ( # noqa: E402
|
||||
screenshot as screenshot_mod,
|
||||
)
|
||||
from harness import ( # noqa: E402
|
||||
settings as settings_mod,
|
||||
)
|
||||
|
||||
ALL_STAGES = ("install", "upgrade", "backup", "restore", "custom")
|
||||
|
||||
@ -113,8 +116,14 @@ def resolve_upgrade_base(
|
||||
stages, meta, recipe: str, head_ref: str | None = None, head_version: str | None = None
|
||||
) -> BasePlan:
|
||||
"""Dynamic upgrade-base resolution (phase prevb, replaces the static `recipe_versions[-2]`
|
||||
default). Order: last-green (warm canonical, with same-version step-back) → target-branch (main)
|
||||
tip → skip. EXPECTED_NA[upgrade] / upgrade∉stages short-circuit to a declared skip first.
|
||||
default). Chain: last-green (warm canonical, with same-version step-back) → newest release tag
|
||||
older than head → main-tip → skip. EXPECTED_NA[upgrade] / upgrade∉stages short-circuit to a
|
||||
declared skip first.
|
||||
|
||||
SKIP_CANONICALS_FOR_UPGRADE (phase settings, server settings.toml, default false): when true, the
|
||||
canonical lookup is bypassed entirely — the resolver behaves as if no canonical exists and takes
|
||||
the no-canonical release-tag-first fallback (`_no_canonical_base`). Scope is the upgrade BASE only;
|
||||
canonical promotion and the `--quick` warm-reattach are unaffected (see DECISIONS, phase settings).
|
||||
|
||||
`head_version` is the head checkout's published version (the `coop-cloud.<stack>.version` label;
|
||||
see abra.head_compose_version). When the last-green warm-canonical version EQUALS it, deploying
|
||||
@ -142,8 +151,9 @@ def resolve_upgrade_base(
|
||||
flush=True,
|
||||
)
|
||||
return BasePlan("skip", None, None, f"declared EXPECTED_NA[upgrade]: {declared}")
|
||||
skip_canonicals = settings_mod.get().skip_canonicals_for_upgrade
|
||||
rec = canonical.read_registry(recipe)
|
||||
if rec and rec.get("version"):
|
||||
if rec and rec.get("version") and not skip_canonicals:
|
||||
canon = rec["version"]
|
||||
same = head_version is not None and warm_reconcile.version_key(
|
||||
canon
|
||||
@ -176,13 +186,46 @@ def resolve_upgrade_base(
|
||||
None,
|
||||
f"base == head ({head_version}) and no older published predecessor",
|
||||
)
|
||||
# No canonical in play — none recorded, OR SKIP_CANONICALS_FOR_UPGRADE=true (canonical lookup
|
||||
# bypassed entirely, behaving as if none exists). Improved fallback (phase settings §2.C): prefer
|
||||
# a REAL published predecessor (newest release tag < head) over the raw main-tip.
|
||||
return _no_canonical_base(recipe, head_ref, head_version)
|
||||
|
||||
|
||||
def _no_canonical_base(recipe: str, head_ref: str | None, head_version: str | None) -> BasePlan:
|
||||
"""Upgrade base when no canonical is used (none recorded, its promote failed, or
|
||||
SKIP_CANONICALS_FOR_UPGRADE is true). Release-tag-first fallback (phase settings §2.C):
|
||||
1. most recent release TAG with version strictly older than the PR head — a clean published
|
||||
predecessor (reuses samever's `newest_older_version` helper, the single source of version
|
||||
ordering, so this and the step-back never diverge);
|
||||
2. raw `main`-tip (target-branch tip) — only if the recipe has NO prior release tag at all;
|
||||
3. skip — no predecessor (no older tag and head == main-tip, or no main at all).
|
||||
This replaces the old jump-straight-to-main-tip path, so an un-promoted recipe upgrades from a real
|
||||
release base instead of a possibly-untagged WIP commit."""
|
||||
older = (
|
||||
warm_reconcile.newest_older_version(warm_reconcile.recipe_tags(recipe), head_version)
|
||||
if head_version
|
||||
else None
|
||||
)
|
||||
if older:
|
||||
return BasePlan(
|
||||
"version",
|
||||
older,
|
||||
None,
|
||||
f"no-canonical fallback: newest release tag older than head {head_version}",
|
||||
)
|
||||
main_tip = lifecycle.recipe_branch_commit(recipe, "main")
|
||||
if main_tip and main_tip != head_ref:
|
||||
return BasePlan("ref", None, main_tip, "target-branch (main) tip")
|
||||
return BasePlan(
|
||||
"ref",
|
||||
None,
|
||||
main_tip,
|
||||
"no-canonical fallback: target-branch (main) tip (no prior release tag)",
|
||||
)
|
||||
if main_tip and main_tip == head_ref:
|
||||
return BasePlan("skip", None, None, "head == main tip (no predecessor delta)")
|
||||
return BasePlan(
|
||||
"skip", None, None, "no last-green and no main tip (new recipe / no predecessor)"
|
||||
"skip", None, None, "no release tag and no main tip (new recipe / no predecessor)"
|
||||
)
|
||||
|
||||
|
||||
|
||||
23
settings.toml.example
Normal file
23
settings.toml.example
Normal file
@ -0,0 +1,23 @@
|
||||
# cc-ci server settings (phase settings) — EXAMPLE / documentation.
|
||||
#
|
||||
# This is a minimal, extensible server-level config for the cc-ci CI host. Copy it to the live host
|
||||
# path and edit as needed:
|
||||
#
|
||||
# cp settings.toml.example /etc/cc-ci/settings.toml # (or set $CCCI_SETTINGS to any path)
|
||||
#
|
||||
# The live file is OPERATOR-MANAGED and not committed (a per-server host override). Every key has a
|
||||
# default baked into the loader (runner/harness/settings.py), so an ABSENT file — or an absent key —
|
||||
# behaves exactly as the documented default. You only need a live file to CHANGE a default.
|
||||
#
|
||||
# NO SECRETS in this file — config only. Secrets stay in sops.
|
||||
# Stdlib TOML (tomllib): booleans are `true`/`false` (lowercase, unquoted).
|
||||
|
||||
[upgrade]
|
||||
# When true, the upgrade-tier BASE is resolved WITHOUT canonicals: the canonical (last-green warm)
|
||||
# lookup is skipped entirely and the base falls through to the no-canonical fallback — the most recent
|
||||
# release TAG on main older than the PR head, with the raw main-tip only as a further fallback, then
|
||||
# skip. Codifies canonicals as an optional optimization (an operator switch); does NOT affect canonical
|
||||
# promotion or the `--quick` warm-reattach (those are separate, out of scope here).
|
||||
#
|
||||
# Default: false (this server keeps canonicals on — the optimized/robust upgrade-base path).
|
||||
skip_canonicals_for_upgrade = false
|
||||
111
tests/unit/test_settings.py
Normal file
111
tests/unit/test_settings.py
Normal file
@ -0,0 +1,111 @@
|
||||
"""Unit tests for `harness.settings` — the minimal, extensible server-level TOML config loader
|
||||
(phase settings). Stdlib `tomllib`; defaults baked in; absent/malformed file degrades to defaults
|
||||
(never crashes the harness); unknown keys warn-and-ignore; a present known key of the wrong type
|
||||
errors clearly.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "runner"))
|
||||
from harness import settings # noqa: E402
|
||||
|
||||
|
||||
def _write(tmp_path, text: str) -> str:
|
||||
p = tmp_path / "settings.toml"
|
||||
p.write_text(text)
|
||||
return str(p)
|
||||
|
||||
|
||||
def test_absent_file_yields_defaults(tmp_path):
|
||||
# a path that does not exist → all-defaults, no exception
|
||||
missing = str(tmp_path / "nope.toml")
|
||||
s = settings.load(missing)
|
||||
assert s.skip_canonicals_for_upgrade is False
|
||||
|
||||
|
||||
def test_absent_key_yields_default(tmp_path):
|
||||
# present file, present table, but the key omitted → default
|
||||
p = _write(tmp_path, "[upgrade]\n")
|
||||
assert settings.load(p).skip_canonicals_for_upgrade is False
|
||||
|
||||
|
||||
def test_empty_file_yields_defaults(tmp_path):
|
||||
p = _write(tmp_path, "")
|
||||
assert settings.load(p).skip_canonicals_for_upgrade is False
|
||||
|
||||
|
||||
def test_flag_true_read(tmp_path):
|
||||
p = _write(tmp_path, "[upgrade]\nskip_canonicals_for_upgrade = true\n")
|
||||
assert settings.load(p).skip_canonicals_for_upgrade is True
|
||||
|
||||
|
||||
def test_flag_false_read(tmp_path):
|
||||
p = _write(tmp_path, "[upgrade]\nskip_canonicals_for_upgrade = false\n")
|
||||
assert settings.load(p).skip_canonicals_for_upgrade is False
|
||||
|
||||
|
||||
def test_malformed_toml_degrades_to_defaults(tmp_path):
|
||||
# syntactically broken TOML must NOT crash the harness — WARN + defaults.
|
||||
p = _write(tmp_path, "[upgrade\nskip_canonicals_for_upgrade = tru")
|
||||
s = settings.load(p) # must not raise
|
||||
assert s.skip_canonicals_for_upgrade is False
|
||||
|
||||
|
||||
def test_wrong_type_errors_clearly(tmp_path):
|
||||
# a present key of the wrong type is a loud, actionable error (distinct from a malformed file).
|
||||
p = _write(tmp_path, '[upgrade]\nskip_canonicals_for_upgrade = "yes"\n')
|
||||
with pytest.raises(TypeError) as e:
|
||||
settings.load(p)
|
||||
assert "skip_canonicals_for_upgrade" in str(e.value)
|
||||
assert "bool" in str(e.value)
|
||||
|
||||
|
||||
def test_int_not_accepted_for_bool(tmp_path):
|
||||
# bool is an int subclass — a stray 1/0 must not silently coerce to a flag.
|
||||
p = _write(tmp_path, "[upgrade]\nskip_canonicals_for_upgrade = 1\n")
|
||||
with pytest.raises(TypeError):
|
||||
settings.load(p)
|
||||
|
||||
|
||||
def test_unknown_key_warns_and_ignored(tmp_path, capsys):
|
||||
p = _write(
|
||||
tmp_path,
|
||||
"[upgrade]\nskip_canonicals_for_upgrade = true\nfuture_knob = 7\n",
|
||||
)
|
||||
s = settings.load(p)
|
||||
assert s.skip_canonicals_for_upgrade is True # known key still honored
|
||||
err = capsys.readouterr().err
|
||||
assert "unknown key" in err and "future_knob" in err
|
||||
|
||||
|
||||
def test_unknown_table_warns_and_ignored(tmp_path, capsys):
|
||||
p = _write(tmp_path, "[future_section]\nx = 1\n")
|
||||
s = settings.load(p)
|
||||
assert s.skip_canonicals_for_upgrade is False
|
||||
err = capsys.readouterr().err
|
||||
assert "unknown table" in err and "future_section" in err
|
||||
|
||||
|
||||
def test_non_table_section_ignored(tmp_path, capsys):
|
||||
# a key named like a table but given a scalar — warn, ignore, fall back to defaults for that table.
|
||||
p = _write(tmp_path, "upgrade = 5\n")
|
||||
s = settings.load(p)
|
||||
assert s.skip_canonicals_for_upgrade is False
|
||||
assert "not a table" in capsys.readouterr().err
|
||||
|
||||
|
||||
def test_env_var_path_override(tmp_path, monkeypatch):
|
||||
p = _write(tmp_path, "[upgrade]\nskip_canonicals_for_upgrade = true\n")
|
||||
monkeypatch.setenv("CCCI_SETTINGS", p)
|
||||
assert settings.load().skip_canonicals_for_upgrade is True
|
||||
|
||||
|
||||
def test_default_path_is_absolute_host_path():
|
||||
# the live file is an absolute host override co-located with the deployed checkout.
|
||||
assert settings.DEFAULT_PATH == "/etc/cc-ci/settings.toml"
|
||||
assert os.path.isabs(settings.DEFAULT_PATH)
|
||||
@ -13,6 +13,8 @@ import os
|
||||
import sys
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "runner"))
|
||||
import run_recipe_ci # noqa: E402
|
||||
import warm_reconcile # noqa: E402
|
||||
@ -23,6 +25,25 @@ HEAD = "aaaa1111head"
|
||||
MAIN = "bbbb2222main"
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _default_flag_false(monkeypatch):
|
||||
# Hermetic: SKIP_CANONICALS_FOR_UPGRADE defaults to false regardless of any host settings.toml.
|
||||
# The few flag-true tests re-patch this within the test body.
|
||||
monkeypatch.setattr(
|
||||
run_recipe_ci.settings_mod,
|
||||
"get",
|
||||
lambda: SimpleNamespace(skip_canonicals_for_upgrade=False),
|
||||
)
|
||||
|
||||
|
||||
def _set_flag(monkeypatch, value: bool):
|
||||
monkeypatch.setattr(
|
||||
run_recipe_ci.settings_mod,
|
||||
"get",
|
||||
lambda: SimpleNamespace(skip_canonicals_for_upgrade=value),
|
||||
)
|
||||
|
||||
|
||||
def _meta(expected_na=None):
|
||||
return SimpleNamespace(EXPECTED_NA=expected_na)
|
||||
|
||||
@ -182,3 +203,119 @@ def test_expected_na_other_rung_does_not_suppress_upgrade(monkeypatch):
|
||||
meta = _meta(expected_na={"backup_restore": "stateless"})
|
||||
plan = run_recipe_ci.resolve_upgrade_base(ALL, meta, "custom-html-tiny", head_ref=HEAD)
|
||||
assert plan.kind == "ref" and plan.ref == MAIN
|
||||
|
||||
|
||||
# --- phase settings: improved no-canonical fallback (release tag before main-tip) + the flag ---
|
||||
|
||||
|
||||
def test_no_canonical_prefers_release_tag_over_main_tip(monkeypatch):
|
||||
# flag false + NO canonical → the base must be the newest release TAG strictly older than head
|
||||
# (a real published predecessor), NOT the raw main-tip. main must not even be consulted.
|
||||
_no_canonical(monkeypatch)
|
||||
monkeypatch.setattr(warm_reconcile, "recipe_tags", lambda r: KC_TAGS)
|
||||
monkeypatch.setattr(
|
||||
lifecycle,
|
||||
"recipe_branch_commit",
|
||||
lambda r, b="main": (_ for _ in ()).throw(AssertionError("main consulted before tag")),
|
||||
)
|
||||
plan = run_recipe_ci.resolve_upgrade_base(
|
||||
ALL, _meta(), "keycloak", head_ref=HEAD, head_version="10.8.0+26.6.3"
|
||||
)
|
||||
assert plan.kind == "version" and plan.runs
|
||||
assert plan.version == "10.7.1+26.6.2" # newest tag strictly older than head
|
||||
assert warm_reconcile.version_key(plan.version) < warm_reconcile.version_key("10.8.0+26.6.3")
|
||||
assert "no-canonical fallback" in plan.reason and "release tag" in plan.reason
|
||||
|
||||
|
||||
def test_no_canonical_no_older_tag_falls_back_to_main_tip(monkeypatch):
|
||||
# flag false + no canonical + no release tag strictly older than head → raw main-tip is the
|
||||
# FURTHER fallback (a recipe whose only tag IS the head version, e.g. brand-new single release).
|
||||
_no_canonical(monkeypatch)
|
||||
monkeypatch.setattr(warm_reconcile, "recipe_tags", lambda r: ["10.8.0+26.6.3"]) # == head only
|
||||
monkeypatch.setattr(lifecycle, "recipe_branch_commit", lambda r, b="main": MAIN)
|
||||
plan = run_recipe_ci.resolve_upgrade_base(
|
||||
ALL, _meta(), "keycloak", head_ref=HEAD, head_version="10.8.0+26.6.3"
|
||||
)
|
||||
assert plan.kind == "ref" and plan.ref == MAIN and plan.version is None
|
||||
|
||||
|
||||
def test_no_canonical_no_tag_no_main_skips(monkeypatch):
|
||||
# no canonical, no release tag, no main → declared skip (new recipe / no predecessor).
|
||||
_no_canonical(monkeypatch)
|
||||
monkeypatch.setattr(warm_reconcile, "recipe_tags", lambda r: [])
|
||||
_no_main(monkeypatch)
|
||||
plan = run_recipe_ci.resolve_upgrade_base(
|
||||
ALL, _meta(), "brandnew", head_ref=HEAD, head_version="1.0.0"
|
||||
)
|
||||
assert plan.kind == "skip" and not plan.runs and "no predecessor" in plan.reason
|
||||
|
||||
|
||||
def test_no_head_version_skips_tag_lookup_uses_main(monkeypatch):
|
||||
# no canonical AND no head_version (unreadable) → cannot compare versions, so recipe_tags is NOT
|
||||
# consulted; fall straight through to main-tip (preserves prevb behavior for that caller shape).
|
||||
_no_canonical(monkeypatch)
|
||||
monkeypatch.setattr(
|
||||
warm_reconcile,
|
||||
"recipe_tags",
|
||||
lambda r: (_ for _ in ()).throw(AssertionError("tags consulted without head_version")),
|
||||
)
|
||||
monkeypatch.setattr(lifecycle, "recipe_branch_commit", lambda r, b="main": MAIN)
|
||||
plan = run_recipe_ci.resolve_upgrade_base(ALL, _meta(), "discourse", head_ref=HEAD)
|
||||
assert plan.kind == "ref" and plan.ref == MAIN
|
||||
|
||||
|
||||
def test_flag_true_bypasses_canonical_into_release_tag_fallback(monkeypatch):
|
||||
# SKIP_CANONICALS_FOR_UPGRADE=true: a canonical-bearing recipe (canonical ≠ head, would normally
|
||||
# resolve to the canonical) instead bypasses the canonical entirely and takes the release-tag
|
||||
# fallback → newest release tag < head.
|
||||
_set_flag(monkeypatch, True)
|
||||
monkeypatch.setattr(
|
||||
canonical, "read_registry", lambda r: {"version": "10.6.0+26.5.0", "status": "warm"}
|
||||
)
|
||||
monkeypatch.setattr(warm_reconcile, "recipe_tags", lambda r: KC_TAGS)
|
||||
plan = run_recipe_ci.resolve_upgrade_base(
|
||||
ALL, _meta(), "keycloak", head_ref=HEAD, head_version="10.8.0+26.6.3"
|
||||
)
|
||||
assert plan.kind == "version" and plan.runs
|
||||
assert plan.version == "10.7.1+26.6.2" # release tag, NOT the canonical 10.6.0+26.5.0
|
||||
assert "no-canonical fallback" in plan.reason
|
||||
|
||||
|
||||
def test_flag_true_canonical_present_no_older_tag_uses_main(monkeypatch):
|
||||
# flag true bypasses the canonical; with no older release tag, the further fallback is main-tip
|
||||
# (proves the flag genuinely routes through the full no-canonical chain, not just step 1).
|
||||
_set_flag(monkeypatch, True)
|
||||
monkeypatch.setattr(
|
||||
canonical, "read_registry", lambda r: {"version": "10.8.0+26.6.3", "status": "warm"}
|
||||
)
|
||||
monkeypatch.setattr(warm_reconcile, "recipe_tags", lambda r: ["10.8.0+26.6.3"]) # == head only
|
||||
monkeypatch.setattr(lifecycle, "recipe_branch_commit", lambda r, b="main": MAIN)
|
||||
plan = run_recipe_ci.resolve_upgrade_base(
|
||||
ALL, _meta(), "keycloak", head_ref=HEAD, head_version="10.8.0+26.6.3"
|
||||
)
|
||||
assert plan.kind == "ref" and plan.ref == MAIN
|
||||
|
||||
|
||||
def test_flag_false_canonical_present_unchanged(monkeypatch):
|
||||
# explicit guardrail check: flag false + canonical present (≠ head) → canonical, byte-for-byte the
|
||||
# prevb behavior; recipe_tags / main never consulted.
|
||||
_set_flag(monkeypatch, False)
|
||||
monkeypatch.setattr(
|
||||
canonical, "read_registry", lambda r: {"version": "10.7.1+26.6.2", "status": "warm"}
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
warm_reconcile,
|
||||
"recipe_tags",
|
||||
lambda r: (_ for _ in ()).throw(AssertionError("fallback taken with canonical present")),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
lifecycle,
|
||||
"recipe_branch_commit",
|
||||
lambda r, b="main": (_ for _ in ()).throw(AssertionError("main consulted")),
|
||||
)
|
||||
plan = run_recipe_ci.resolve_upgrade_base(
|
||||
ALL, _meta(), "keycloak", head_ref=HEAD, head_version="10.8.0+26.6.3"
|
||||
)
|
||||
assert (
|
||||
plan.kind == "version" and plan.version == "10.7.1+26.6.2" and "last-green" in plan.reason
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user