claim(M1-nixenv): single-source harness runtime env — ccciPyEnv+ccciRuntimeTools+cc-ci-run in packages.nix, referenced by harness/sweep/both hosts; sweep execs cc-ci-run (no dup pyEnv, no DEFECT-3 PATH patch); cc-ci host gains git-lfs+openssl; both #cc-ci and #cc-ci-hetzner build; awaiting Adversary
Some checks failed
continuous-integration/drone/push Build is failing

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
autonomic-bot
2026-06-17 17:23:28 +00:00
parent 706583bee3
commit 8b8fc1ff8e
8 changed files with 242 additions and 64 deletions

View File

@ -0,0 +1,19 @@
# BACKLOG — phase `nixenv`
## Build backlog
- [x] M1: define shared harness/recipe-test runtime env once (overlay in `packages.nix`):
`ccciPyEnv` + `ccciRuntimeTools` (the union tool set) + `cc-ci-run`.
- [x] M1: `harness.nix` references `pkgs.cc-ci-run` (no local pyEnv/runtimeInputs).
- [x] M1: `nightly-sweep.nix` invokes `cc-ci-run` (no duplicate pyEnv, no own tool list, DEFECT-3 patch gone).
- [x] M1: both host `configuration.nix` `systemPackages` reference `pkgs.ccciRuntimeTools` (+ openssh); end identical.
- [x] M1: grep proof — exactly one `withPackages`/`pytest playwright` in nix/ (packages.nix); no module declares its own harness tool list.
- [x] M1: `nixos-rebuild build` succeeds for both `#cc-ci` and `#cc-ci-hetzner`.
- [x] M1: CLAIM, await Adversary PASS.
- [ ] M2: deploy via `nixos-rebuild switch`; verify host health (systemctl --failed, oneshots, timer, endpoints).
- [ ] M2: live parity — gitea `test_lfs_roundtrip` green under BOTH Drone path and a real timer fire from the unified env.
- [ ] M2: canon-style sweep still promotes/SKIPs correctly (no regression).
- [ ] M2: CLAIM, await Adversary PASS → `## DONE`.
## Adversary findings
<!-- Adversary-owned section. Builder does not edit. -->

View File

@ -0,0 +1,59 @@
# JOURNAL — phase `nixenv` (Builder)
## 2026-06-17 — M1: single-source the harness runtime env
### Why this design
The phase plan §2 wants ONE definition of "what's needed to run a recipe test", referenced from
three places, so DEFECT-3 (a dep present for one path, missing for another) becomes structurally
impossible. I put the single source in `nix/modules/packages.nix` because it is the existing
"shared pkgs" overlay module already imported by both host configs — so `pkgs.ccciRuntimeTools`
and `pkgs.cc-ci-run` are reachable from every module/host without a fragile cross-module `let`.
Three overlay defs:
- `ccciPyEnv` (let-bound, internal) — `python3.withPackages [pytest playwright]`, the ONLY pyEnv now.
- `ccciRuntimeTools` (overlay attr) — the union tool set.
- `cc-ci-run` (overlay attr) — `writeShellApplication` with `runtimeInputs = [ccciPyEnv] ++ ccciRuntimeTools`.
Consumers:
- `harness.nix``environment.systemPackages = [ pkgs.cc-ci-run ]` (installs the entrypoint).
- `nightly-sweep.nix` → wrapper execs `cc-ci-run` (same binary the Drone pipeline runs), so pyEnv +
tooling + PLAYWRIGHT env are identical to the Drone path by construction. Dropped: the duplicate
pyEnv, the parallel `runtimeInputs` tool list, and the DEFECT-3 `export PATH=/run/current-system/sw/bin…`
prepend — git-lfs/bash/util-linux/openssl now come from cc-ci-run's runtimeInputs.
- both host `configuration.nix``systemPackages = pkgs.ccciRuntimeTools ++ [ pkgs.openssh ]`.
### Why the union is a superset (nothing dropped)
- old cc-ci-run: `abra docker git coreutils util-linux` ⊂ set.
- old sweep: `bash abra docker git curl jq gnused gnugrep gnutar coreutils util-linux procps` ⊂ set;
its host-PATH-derived git-lfs/openssl are now EXPLICIT in the set.
- old host PATH: `curl git jq` (+ git-lfs on hetzner only) ⊂ set; `openssh` kept as host-only add.
- pyEnv (python3+pytest+playwright) + playwright browsers (via PLAYWRIGHT_BROWSERS_PATH) preserved.
Additions vs any single prior list: `git-lfs`, `openssl` (plan §2). The `cc-ci` host GAINS git-lfs,
killing the one-off hetzner-only divergence — both host configs now byte-identical.
### Why writeShellApplication makes this work
`writeShellApplication` emits `export PATH="<runtimeInputs>:$PATH"` (confirmed on the live wrapper).
So cc-ci-run's full tool set is the PATH *prefix* regardless of caller. Under Drone the inherited
suffix is `/run/current-system/sw/bin:/run/wrappers/bin`; under the sweep it's the systemd-minimal
PATH — but the harness tools all resolve from the shared prefix either way, which is the parity the
plan wants. The host `systemPackages` reference is the belt-and-suspenders path for direct
`.drone.yml` shell-outs (`abra --version`, `docker info`) that don't go through cc-ci-run.
### buildEnv collision watch (resolved)
Worry: adding coreutils/util-linux/procps/bash/gnu* to host `systemPackages` could collide with the
NixOS base `requiredPackages`. It did not — base requiredPackages are `lowPrio`, so the normal-prio
additions override cleanly. Both `#cc-ci` and `#cc-ci-hetzner` built with no collision error.
### Note on other modules' tool lists
`backupbot/docker-prune/drone/proxy/warm-keycloak.nix` still list gnused/gnugrep/etc. in their OWN
`runtimeInputs` — those are independent reconcile-service scripts, never part of the harness/recipe
-test env, never part of the DEFECT-3 divergence. Single-sourcing is scoped to the harness env
(pyEnv + recipe-test tooling consumed by cc-ci-run / sweep / host PATH), which is now packages.nix only.
### Verification (local, dirty tree needs `?submodules=1` — `secrets/` is a submodule)
- `nixos-rebuild build --flake '.?submodules=1#cc-ci-hetzner'` → built `nixos-system-…dhmpm232…`.
- `nixos-rebuild build --flake '.?submodules=1#cc-ci'` → built OK.
- cc-ci-run store `zxlx9jnylh7la5m48bsqb1wfm5l9r0bd`; PATH carries all 15 tools incl git-lfs-3.6.1 + openssl-3.3.3.
- sweep wrapper `gh02w1kc…` execs the SAME `zxlx9j…/bin/cc-ci-run`.
- cc-ci host sw/bin now lists git-lfs + openssl (was missing git-lfs pre-refactor).
- `grep -rn withPackages nix/` → 1 hit (packages.nix:17).

View File

@ -0,0 +1,65 @@
# STATUS — phase `nixenv` (Builder)
Phase plan: `/srv/cc-ci/cc-ci-plan/plan-phase-nixenv-shared-runtime-env.md`
## Phase
Single-source the harness/recipe-test runtime env so the Drone runner, the nightly/weekly sweep
timer, and host `systemPackages` share ONE declaration (no duplicate `pyEnv`, no divergent
`runtimeInputs`, DEFECT-3 host-PATH patch removed/subsumed).
## Gate: M1 — CLAIMED, awaiting Adversary
**WHAT (M1 DoD).** The harness/recipe-test runtime env is declared ONCE and referenced by all
consumers; `nixos-rebuild build` succeeds for both hosts; the shared set is superset-or-equal of
every prior list (nothing dropped); the sweep and the Drone runner resolve the same tooling; a
future dep added to the shared set reaches all consumers.
**WHERE (inputs).** All changes at the tip of `main` (commit pushed with this claim).
- Single source: `nix/modules/packages.nix` — overlay defines `ccciPyEnv` (let), `ccciRuntimeTools`
(overlay attr), `cc-ci-run` (overlay attr, `runtimeInputs = [ccciPyEnv] ++ ccciRuntimeTools`).
- Consumers: `nix/modules/harness.nix` (`systemPackages = [ pkgs.cc-ci-run ]`),
`nix/modules/nightly-sweep.nix` (wrapper execs `cc-ci-run`),
`nix/hosts/cc-ci/configuration.nix` + `nix/hosts/cc-ci-hetzner/configuration.nix`
(`systemPackages = pkgs.ccciRuntimeTools ++ [ pkgs.openssh ]`).
- `nix/modules/drone-runner.nix` unchanged (still `PATH=/run/current-system/sw/bin:/run/wrappers/bin`;
it consumes the host PATH, which now references the shared set).
**HOW + EXPECTED (cold-verifiable; `secrets/` is a git submodule → use `?submodules=1` for a dirty
tree, or build from a `git clone --recursive`).**
1. Builds succeed (both hosts):
- `nixos-rebuild build --flake '.?submodules=1#cc-ci-hetzner'` → builds
`nixos-system-nixos-24.11.…` (locally: `/nix/store/dhmpm232r6m0sq3s7y5r5jpyv5kxgzwi-nixos-system-nixos-24.11.20250630.50ab793`;
store hash may differ on a fresh clone if paths differ, but it MUST build with no collision error).
- `nixos-rebuild build --flake '.?submodules=1#cc-ci'` → builds OK (no collision error).
2. Single source (grep proofs):
- `grep -rn withPackages nix/` → EXACTLY 1 hit: `nix/modules/packages.nix` (`ccciPyEnv`).
- `grep -rn "pytest playwright" nix/` → EXACTLY 1 hit: same line. (No duplicate pyEnv.)
- `grep -rn ccciRuntimeTools nix/` → defined once (packages.nix), referenced by both host configs.
- `nightly-sweep.nix` contains NO `withPackages`, NO `python3`, NO `/run/current-system/sw/bin`
PATH prepend, and its `runtimeInputs = [ pkgs.cc-ci-run ]` only; it `exec cc-ci-run …`.
3. Superset-or-equal — `cc-ci-run` carries every tool (inspect the built wrapper's PATH):
- `CCRUN=$(nix eval --raw '.?submodules=1#nixosConfigurations.cc-ci-hetzner.pkgs.cc-ci-run'); grep '^export PATH' "$CCRUN/bin/cc-ci-run"`
- EXPECTED store dirs on PATH (15): python3-3.12.8-env, abra-0.13.0-beta, docker-27.5.1,
git-2.47.2, **git-lfs-3.6.1**, bash-5.2p37, coreutils-9.5, util-linux-2.39.4, curl-8.12.1,
jq-1.7.1, gnused-4.9, gnugrep-3.11, gnutar-1.35, **openssl-3.3.3**, procps-4.0.4.
- git-lfs + openssl are the additions vs prior lists; nothing from any prior list is dropped.
4. Sweep ≡ Drone entrypoint (parity by construction):
- The built `cc-ci-nightly-sweep` wrapper `exec cc-ci-run …` resolves the BYTE-IDENTICAL
cc-ci-run store path that the `.drone.yml` `cc-ci-run runner/run_recipe_ci.py` step runs
(locally `/nix/store/zxlx9jnylh7la5m48bsqb1wfm5l9r0bd-cc-ci-run`). Same store path ⇒ same
pyEnv, same tooling, same PLAYWRIGHT_BROWSERS_PATH.
5. Host divergence removed:
- Both host `configuration.nix` `systemPackages` lines are textually identical
(`pkgs.ccciRuntimeTools ++ [ pkgs.openssh ]`). The `cc-ci` host now GAINS `git-lfs`+`openssl`
on its system PATH (`ls $(nix eval --raw '.?submodules=1#nixosConfigurations.cc-ci.config.system.build.toplevel')/sw/bin/ | grep -E '^(git-lfs|openssl)$'` → both present; pre-refactor cc-ci lacked git-lfs).
6. Future-dep propagation: adding a pkg to `ccciRuntimeTools` in packages.nix lands in cc-ci-run's
runtimeInputs (Drone + sweep) AND both hosts' systemPackages from the single edit.
## Build backlog
See `BACKLOG-nixenv.md`. M2 (deploy + live parity witness) is gated behind the M1 PASS.