cc-ci/machine-docs/REVIEW-nixenv.md

REVIEW — phase nixenv (Adversary)

Phase plan: /srv/cc-ci/cc-ci-plan/plan-phase-nixenv-shared-runtime-env.md SSOT for verification. Verdicts below; cold-runs only.

Status: M1 PASS @ 17:40Z (8b8fc1f) + M2 PASS @ 18:20Z (f7b6f26). Both milestones fresh Adversary PASS, no VETO → Builder cleared to write ## DONE.

---

M2 — PASS @ 2026-06-17T18:20Z — claim f7b6f26 (deployed /etc/cc-ci@d11f8f5 = M1-reviewed tree)

Deploy + live parity proven — cold-verified. Verdict from the plan (SSOT), the code, the claim's verification info, and my OWN live re-runs (Drone API, journald, host probes). JOURNAL-nixenv.md NOT read before this verdict (anti-anchoring preserved).

(1) Deploy clean + host healthy (re-verified live post-sweep @18:16–18:18Z).

• Deployed system dhmpm232r6m0sq3s7y5r5jpyv5kxgzwi-nixos-system-… BYTE-IDENTICAL to my M1 build.
• systemctl --failed EMPTY; nightly-sweep.timer active+enabled; drone-runner-exec / deploy-proxy / warm-keycloak / swarm-init all active; nightly-sweep.service finished Result=success ExecMainStatus=0. drone /healthz→200, ci.commoninternet.net→200.
• Live cc-ci-run = zxlx9jnylh7la5m48bsqb1wfm5l9r0bd (M1-reviewed path). git-lfs/openssl/script/bash resolve on host PATH AND inside cc-ci-run (git-lfs→33ikv…-git-lfs-3.6.1, openssl→48p8b…-openssl-3.3.3 from runtimeInputs, NOT host PATH). openssl was MISSING on this host pre-deploy.
• NO orphan ephemeral test stacks left by the sweep (no gite-/matt-/disc- per-run stacks); only the expected warm canonicals (bluesky-pds, gitea, keycloak) remain — clean teardown.

(2) Live LFS parity — GREEN on BOTH paths (the DEFECT-3 witness).

• Real timer fire: systemctl start nightly-sweep.service @17:35:38Z; gitea RUN-eligible (canonical 3.5.3 < tag 3.6.0) → tests/gitea/custom/test_lfs_roundtrip.py::test_lfs_roundtrip PASSED @17:57:54Z (+ install/upgrade/backup/restore all PASS). The systemd unit PATH carries NO git-lfs and NO /run/current-system/sw/bin, so git-lfs MUST have resolved from cc-ci-run's runtimeInputs — exactly the old DEFECT-3 condition, now satisfied by the shared env.
• Drone path: independently inspected build #871 via Drone API (status=success): stage recipe-ci → step ci runs cc-ci-run runner/run_recipe_ci.py (.drone.yml:83). Log shows LFS RAN not skipped: test_lfs_roundtrip PASSED; RUN SUMMARY install/upgrade/backup/restore/custom all pass, level=5 of 5.
• Both paths exec the SAME zxlx9jn cc-ci-run ⇒ git-lfs resolves identically. DEFECT-3 class structurally eliminated, demonstrated live.

(3) No regression — sweep SKIPs/promotes correct; the 3 non-green results ALL pre-existing.

• Regression canary: scanned the ENTIRE post-deploy sweep journal for missing-tool signatures (command not found / not found / executable file not found / No such file) → ZERO. Nothing got dropped from the env (consistent with the M1 superset proof). No recipe went GREEN→RED.
• SKIPs all correct (cryptpad/ghost/drone/hedgedoc/immich/lasuite-*/mailu/matrix-synapse/n8n/ plausible/uptime-kuma — no-new-version); promotes correct (custom-html, mumble).
• gitea GREEN-BUT-PROMOTE-FAILED: tests green; WC5 promote abra app deploy warm-gitea… -o -n fails FATA … is already deployed — abra idempotency on the persistent warm canonical (warm-gitea confirmed still up). canonical.json unchanged (3.5.3, ts 08:39Z). Promote path = nightly_sweep.py @canon f94de22, UNCHANGED by nixenv (diff dd6712c..d11f8f5 is nix/+machine-docs only, zero runner/tests) → behaviour identical to canon by construction.
• discourse rc=1 / mattermost-lts rc=1: recipe-level reds, env-independent — discourse test_head_runs_official_image_not_bitnamilegacy + test_sidekiq_service_dropped_by_head (HEAD-image/service assertions); mattermost test_restore_returns_state → docker exec … postgres … relation "ci_marker" does not exist (docker RESOLVED and ran — a restore-data failure, not a missing tool). Corroborated pre-existing: the SAME reds occur in BOTH OLD-env pre-deploy fires today (PID 2149231@14:xx, PID 2248547@15:xx) — mattermost byte-identical postgres error; discourse red in all fires (never green). Not caused by the env change.

No defects, no VETO. M2 DoD fully met live. The harness runtime env is single-sourced and proven identical across the Drone runner, the timer sweep, and host systemPackages, with git-lfs/openssl now guaranteed from one declaration — the DEFECT-3 divergence class is structurally impossible.

M1 + M2 fresh Adversary PASS → DONE is cleared. (Consulted JOURNAL-nixenv.md? No — verdict stands on plan + code + my own live re-runs.)

---

M1 — PASS @ 2026-06-17T17:40Z — claim 8b8fc1f

Single-source harness runtime env — cold-verified, all 6 DoD items. Verdict formed from the phase plan (SSOT), the code, and my OWN cold builds/evals — JOURNAL-nixenv.md NOT consulted (anti-anchoring preserved).

1. Builds succeed, both hosts (no collision). nix build .?submodules=1#…cc-ci-hetzner…toplevel → EXIT 0; …#…cc-ci…toplevel → EXIT 0. (A transient SQLite eval-cache "busy" from running both in parallel was error (ignored), not a build failure.)
2. Single source (greps). withPackages → 1 hit (packages.nix:17 ccciPyEnv); pytest playwright → 1 hit (same line); ccciRuntimeTools defined once (packages.nix:45), referenced by cc-ci-run (:68) + both host configs. nightly-sweep.nix has NO withPackages, NO python3, NO /run/current-system/sw/bin PATH prepend — runtimeInputs = [ pkgs.cc-ci-run ] and exec cc-ci-run …. The DEFECT-3 host-PATH patch is GONE.
3. Superset-or-equal — inspected the BUILT wrapper PATH. cc-ci-run store zxlx9jnylh7la5m48bsqb1wfm5l9r0bd export PATH carries all 15 store dirs: python3-3.12.8-env, abra-0.13.0-beta, docker-27.5.1, git-2.47.2, git-lfs-3.6.1, bash-5.2p37, coreutils-9.5, util-linux-2.39.4, curl-8.12.1, jq-1.7.1, gnused-4.9, gnugrep-3.11, gnutar-1.35, openssl-3.3.3, procps-4.0.4 — and ends :$PATH (PREPEND, inherited PATH retained → nothing from any path lost). Covers the full union of all 3 prior lists; git-lfs+openssl are the only additions. Nothing dropped.
4. Sweep ≡ Drone entrypoint (parity by construction). Built cc-ci-nightly-sweep references the BYTE-IDENTICAL zxlx9jnylh7la5m48bsqb1wfm5l9r0bd-cc-ci-run; both hosts' pkgs.cc-ci-run resolve that SAME store path; .drone.yml:83 runs cc-ci-run runner/run_recipe_ci.py (host systemPackages wrapper = same path). Same store path ⇒ identical pyEnv + tooling + PLAYWRIGHT_BROWSERS_PATH on Drone path AND timer sweep.
5. Host divergence removed. Both configuration.nix systemPackages lines are textually identical (pkgs.ccciRuntimeTools ++ [ pkgs.openssh ]). The pre-refactor cc-ci-vs-hetzner git-lfs one-off divergence (my prep flag #1) is ELIMINATED: built cc-ci toplevel sw/bin now contains git-lfs, openssl, script (util-linux) — tools it previously lacked. openssh correctly kept host-only (ssh client, not a recipe tool); it remains on both hosts so the Drone path's inherited PATH is unchanged for it.
6. Future-dep propagation (by construction). ccciRuntimeTools is the lone definition; it feeds cc-ci-run.runtimeInputs (→ Drone path via .drone.yml, → sweep via exec cc-ci-run) AND both hosts' systemPackages (→ Drone runner host PATH). One edit to that list reaches every consumer. Proven structurally via the reference graph; no working-tree mutation needed.

No defects, no VETO. Faithful refactor — one shared definition, three references, DEFECT-3 class structurally eliminated. M2 (deploy via nixos-rebuild switch + live parity witness: gitea LFS roundtrip green under BOTH Drone path and a real timer fire) remains to be claimed/verified.

---

(prior) Cold-prep notes

---

Cold-prep — enumeration of the CURRENT (pre-refactor) declarations @ HEAD dd6712c

The M1 superset-or-equal proof must show the new shared set ⊇ the union of all of these. Captured from the code (SSOT), independent of any Builder narrative:

(A) nix/modules/harness.nix — cc-ci-run (Drone entrypoint) runtimeInputs: pyEnv abra docker git coreutils util-linux

• pyEnv = python3.withPackages [ pytest playwright ]
• env: PLAYWRIGHT_BROWSERS_PATH=${playwright-driver.browsers}, PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1

(B) nix/modules/nightly-sweep.nix — sweep runtimeInputs: bash abra docker git curl jq gnused gnugrep gnutar coreutils util-linux procps

• DUPLICATE pyEnv = python3.withPackages [ pytest playwright ]
• same PLAYWRIGHT env
• DEFECT-3 patch: export PATH="/run/current-system/sw/bin:/run/wrappers/bin:$PATH" (host-PATH prepend)

(C) Drone runner path — nix/modules/drone-runner.nix: PATH = mkForce "/run/current-system/sw/bin:/run/wrappers/bin" → recipe shell-outs resolve from host environment.systemPackages, NOT a runtimeInputs list.

(D) Host systemPackages (feeds C):

• nix/hosts/cc-ci/configuration.nix: curl git jq openssh ← NO git-lfs
• nix/hosts/cc-ci-hetzner/configuration.nix: curl git git-lfs jq openssh

UNION the shared set must cover (≥):

python3+pytest+playwright (pyEnv) · playwright browsers · abra docker git git-lfs coreutils util-linux bash curl jq gnused gnugrep gnutar procps openssh Plan §2 also names openssl as a recipe shell-out → expect it present too.

Pre-noted suspicions to break on M1/M2 (cold, not yet verdicts):

1. Host divergence: cc-ci config lacks git-lfs but hetzner has it. Which config is the LIVE ssh cc-ci server running, and does git-lfs actually resolve there today? If the shared set is applied to both host configs, cc-ci should GAIN git-lfs. Verify both configs end identical.
2. Nothing dropped: any token in the union missing from the shared set = blast-radius break.
3. Sweep parity by construction: plan wants sweep to invoke cc-ci-run (same entrypoint) — if it instead keeps a parallel list, "single source" is not actually achieved; grep must prove no module declares its own harness dep list.
4. DEFECT-3 patch removal: the host-PATH prepend should be gone/subsumed; if removed, git-lfs etc. must now come from the shared runtimeInputs, else the sweep regresses.
5. Live witness: gitea test_lfs_roundtrip must stay GREEN under BOTH Drone path and a real timer fire from the unified env.