feat(settings): server settings.toml loader + SKIP_CANONICALS_FOR_UPGRADE + release-tag-first no-canonical fallback
Some checks failed
continuous-integration/drone/push Build is failing
Some checks failed
continuous-integration/drone/push Build is failing
- harness/settings.py: stdlib tomllib loader, [upgrade].skip_canonicals_for_upgrade (bool, default false), _SCHEMA single-source defaults+validation; graceful on absent/malformed (WARN+defaults), warn-and-ignore unknown keys/tables, TypeError on wrong type. Path $CCCI_SETTINGS / /etc/cc-ci/settings.toml. + tracked settings.toml.example. - resolve_upgrade_base: flag true bypasses the canonical lookup -> no-canonical fallback; canonical-present path (incl. samever step-back) unchanged when false. - _no_canonical_base (always-on, §2.C): newest release tag < head (reuse warm_reconcile.newest_older_version) -> main-tip -> skip; replaces jump-to-main-tip. - unit: full resolution matrix + loader tests; 315 unit pass, ruff clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
24
machine-docs/BACKLOG-settings.md
Normal file
24
machine-docs/BACKLOG-settings.md
Normal file
@ -0,0 +1,24 @@
|
||||
# BACKLOG — phase `settings`
|
||||
|
||||
## Build backlog
|
||||
|
||||
- [x] **B1** — `harness/settings.py`: stdlib `tomllib` loader, `[upgrade].skip_canonicals_for_upgrade`
|
||||
(bool, default false), `_SCHEMA` single-source defaults+validation, graceful on absent/malformed,
|
||||
warn-and-ignore unknown keys/tables, raise on wrong type. Path `$CCCI_SETTINGS` / `/etc/cc-ci/settings.toml`.
|
||||
- [x] **B2** — tracked `settings.toml.example` documenting keys + defaults (no secrets).
|
||||
- [x] **B3** — wire `SKIP_CANONICALS_FOR_UPGRADE` into `resolve_upgrade_base` (`run_recipe_ci.py`):
|
||||
flag true → bypass canonical lookup → no-canonical fallback. Scope = upgrade base only.
|
||||
- [x] **B4** — improved no-canonical fallback `_no_canonical_base` (§2.C): newest release tag `< head`
|
||||
(reuse `warm_reconcile.newest_older_version`) → main-tip → skip. Always-on.
|
||||
- [x] **B5** — unit tests: full resolution matrix (`tests/unit/test_upgrade_base.py`) + loader
|
||||
(`tests/unit/test_settings.py`). 315 unit pass, lint clean.
|
||||
- [x] **B6 (M1 claim)** — clean tree, push, claim M1 in STATUS-settings.md.
|
||||
|
||||
### M2 (after M1 PASS)
|
||||
- [ ] **B7** — deploy to cc-ci (`/etc/cc-ci` git pull + nixos-rebuild if needed); confirm harness reads
|
||||
settings (absent → default false; or file present false).
|
||||
- [ ] **B8** — live evidence (a): a recipe WITHOUT a canonical resolves base to newest release tag `< head`
|
||||
(not raw main-tip).
|
||||
- [ ] **B9** — live evidence (b): flip `SKIP_CANONICALS_FOR_UPGRADE = true` (scratch) → a canonical-bearing
|
||||
recipe ALSO resolves to the release-tag base (canonical bypassed); then restore false.
|
||||
- [ ] **B10 (M2 claim)** — claim M2; on fresh PASS of M1+M2 → `## DONE`.
|
||||
@ -1579,3 +1579,33 @@ OVERVIEW (`/`) and badges keep their Drone latest-per-recipe source unchanged. D
|
||||
merge Drone live "running" status into history (optional per plan; re-adds the network dependency the
|
||||
local source removes; overview already shows live status). Retention: 308 parseable runs present, no
|
||||
trim job observed → adequate; revisit only if a cap is ever needed.
|
||||
|
||||
---
|
||||
|
||||
## Phase `settings` (2026-06-17) — server settings.toml + SKIP_CANONICALS_FOR_UPGRADE + release-tag-first fallback
|
||||
|
||||
- **Settings home = `harness/settings.py` (new), file `/etc/cc-ci/settings.toml` (override `$CCCI_SETTINGS`).**
|
||||
No pre-existing cc-ci config module existed to extend (config was scattered `os.environ.get` reads);
|
||||
a minimal stdlib-`tomllib` loader is the minimal+extensible mechanism. `_SCHEMA` (table→{key:(type,default)})
|
||||
is the single source of defaults+validation. Tracked `settings.toml.example`; live file untracked/operator-
|
||||
managed/no-secrets (secrets stay in sops). Default `/etc/cc-ci` chosen over the plan's suggested
|
||||
`/srv/cc-ci` (orchestrator-ambiguous): `/etc/cc-ci` is where the harness already runs (`CCCI_REPO`),
|
||||
absolute so Drone+sweep read the same file, untracked file survives deploy `git pull`.
|
||||
- **`SKIP_CANONICALS_FOR_UPGRADE` scope = upgrade BASE only.** Wired into `resolve_upgrade_base`: flag
|
||||
true → skip canonical lookup → no-canonical fallback (behaves as if no canonical). Does NOT touch
|
||||
canonical *promotion* or the `--quick` warm-reattach — those are separate optimizations; a future
|
||||
`SKIP_CANONICAL_SWEEP` / `SKIP_QUICK` could gate them (out of scope here).
|
||||
- **No-canonical fallback (always-on, §2.C):** newest release TAG `< head` (reuse
|
||||
`warm_reconcile.newest_older_version`, the single version-ordering source) → raw main-tip (no prior
|
||||
release tag) → skip. Replaces the old jump-straight-to-main-tip path; improves this server too (false
|
||||
flag, un-promoted recipes get a real release base).
|
||||
- **Canonical-present path (incl. samever step-back) preserved byte-for-byte.** With flag false + a
|
||||
canonical, behavior is unchanged. The step-back's "no older predecessor → skip" is intentionally NOT
|
||||
routed to main-tip (would reintroduce the same-version no-op samever prevents); the §2.C "==head"
|
||||
routing is satisfied because the step-back already takes the same release-tag helper as fallback step 1.
|
||||
- **Validation:** absent/unreadable/malformed-TOML → WARN + all-defaults (cannot crash the harness);
|
||||
unknown table/key → warn-and-ignore; present known key of wrong type → raise TypeError (loud typo).
|
||||
- **OBSERVATION (not this phase's defect):** `scripts/lint.sh` (pinned ruff) reports
|
||||
`dashboard/dashboard.py` + `tests/unit/test_dashboard.py` would be reformatted — confirmed pre-existing
|
||||
at HEAD f68f1c5, outside the settings diff. Flagged for the dashboard owner / orchestrator; not fixed
|
||||
here (narrow scope).
|
||||
|
||||
65
machine-docs/JOURNAL-settings.md
Normal file
65
machine-docs/JOURNAL-settings.md
Normal file
@ -0,0 +1,65 @@
|
||||
# JOURNAL — phase `settings` (WHY / reasoning; Adversary does not read before verdict)
|
||||
|
||||
## 2026-06-17 — bootstrap + M1 design
|
||||
|
||||
**Phase:** server-level `settings.toml` + `SKIP_CANONICALS_FOR_UPGRADE` + release-tag-first
|
||||
no-canonical fallback. Plan: `/srv/cc-ci/cc-ci-plan/plan-phase-settings-ci-server-config.md`.
|
||||
|
||||
### Why a new `harness/settings.py` (not extending an env-var module)
|
||||
Checked for an existing cc-ci config mechanism first (plan §2.A "extend rather than spawn a parallel
|
||||
one"). The server config today is **scattered ad-hoc env reads** (`os.environ.get` for `MAX_TESTS`,
|
||||
`CCCI_RUNS_DIR`, `CCCI_REPO`, `STAGES`, `CCCI_QUICK`, …) — there is **no** central config module/class
|
||||
to extend (`grep` for `tomllib|settings\.toml|class Settings` → none). So a small dedicated loader IS
|
||||
the minimal, extensible home rather than threading another env var. Stdlib `tomllib` (py3.12 on the
|
||||
server, confirmed). One `[upgrade]` table, one key now; `_SCHEMA` is the single source of
|
||||
defaults+validation so adding a key/table later is a one-line change.
|
||||
|
||||
### Settings file path: `/etc/cc-ci/settings.toml` (override `$CCCI_SETTINGS`)
|
||||
The harness runs from `/etc/cc-ci` in BOTH execution contexts (nightly sweep sets `CCCI_REPO=/etc/cc-ci`
|
||||
and `cd`s there; the Drone recipe-CI runner runs from its checkout but an **absolute** host path is read
|
||||
identically by both). `/etc/cc-ci` is a git checkout kept current by `git pull` + nixos-rebuild on
|
||||
deploy — an **untracked** `settings.toml` there survives pulls (git pull never deletes untracked files)
|
||||
and sits next to the tracked `settings.toml.example`. Chose this over `/srv/cc-ci/settings.toml` (the
|
||||
plan's *suggestion*) because `/srv/cc-ci` is the orchestrator path, ambiguous on the server; `/etc/cc-ci`
|
||||
is unambiguous and discoverable. The loader is graceful if the file/dir is absent → defaults.
|
||||
|
||||
### Why the canonical-present path (incl. samever step-back) is byte-for-byte unchanged
|
||||
Guardrail §4: default false must be a no-op for current behavior. Structure:
|
||||
`if rec and rec.version and not flag:` → the entire existing prevb/samever block runs verbatim
|
||||
(canonical ≠ head → canonical; canonical == head → step-back older tag, else skip). Only when there is
|
||||
**no canonical in play** (rec falsy, OR flag true) do we enter the new `_no_canonical_base`. So with
|
||||
flag false + a canonical, nothing changes; the step-back's "no older predecessor → skip" is preserved
|
||||
(NOT routed to main-tip), which is correct — routing it to main-tip could reintroduce the same-version
|
||||
no-op samever exists to prevent. The plan §2.C "unified chain ... (==head)" is satisfied by the
|
||||
step-back already taking the same release-tag helper as step 1; I deliberately did NOT add a main-tip
|
||||
tail to the step-back skip, to keep samever's guarantee intact. This is the one place where a literal
|
||||
reading of §2.C ("==head → ... → main-tip → skip") and the §4 no-op guardrail + samever's intent point
|
||||
slightly differently; I chose the conservative path that preserves both samever and the no-op guardrail.
|
||||
If the Adversary reads §2.C literally and wants the step-back-no-older case to fall to main-tip, that is
|
||||
a one-line change — but I believe it would be a regression (vacuous upgrade), so it's recorded here.
|
||||
|
||||
### Why `_no_canonical_base` guards on `head_version` before calling `recipe_tags`
|
||||
`newest_older_version(tags, None)` returns None, but evaluating `recipe_tags(recipe)` eagerly would
|
||||
shell out to `git -C <per-run recipe dir> tag` even when head_version is None (e.g. callers/tests that
|
||||
don't pass it). Guarding `if head_version else None` avoids a needless/erroring git call and preserves
|
||||
the prevb behavior for the no-head_version caller shape (→ main-tip).
|
||||
|
||||
### Why wrong-type raises but malformed/absent doesn't
|
||||
Plan M1: "malformed file handled" (graceful) AND "wrong type errors clearly". Reconciled: absent /
|
||||
unreadable / TOML-syntax-error → WARN + all-defaults (a red file degrades to today's behavior, can't
|
||||
crash CI). A syntactically-valid file with a **known key of the wrong type** → `TypeError` (a typo'd
|
||||
value should be loud, not silently mis-parsed). bool-is-int-subclass handled: `1`/`0` for a bool key is
|
||||
rejected, not coerced.
|
||||
|
||||
### Pre-existing, OUT OF SCOPE: dashboard lint drift on main
|
||||
`scripts/lint.sh` reports `dashboard/dashboard.py` + `tests/unit/test_dashboard.py` would be reformatted
|
||||
by the pinned ruff — confirmed present at HEAD f68f1c5 (`git show HEAD:...` through pinned ruff), NOT in
|
||||
my diff. Not touched by this phase (narrow scope). Recorded in DECISIONS as an observation. My 5
|
||||
phase files are format-clean + `ruff check` clean.
|
||||
|
||||
### Verification (commands + output)
|
||||
- `nix shell nixpkgs#python311Packages.pytest -c pytest tests/unit/test_upgrade_base.py
|
||||
tests/unit/test_settings.py -q` → **32 passed**.
|
||||
- full unit suite `pytest tests/unit/ -q` → **315 passed**.
|
||||
- `ruff check runner/ tests/unit/ bridge/ dashboard/` → All checks passed.
|
||||
- `ruff format --check` (pinned) on my 5 files → all formatted.
|
||||
Reference in New Issue
Block a user