tangled_pr: verify against the pulls list, not a response header; and a tools test suite
THE BUG THAT PROMPTED THIS. tangled_pr.py judged success ONLY by an HX-Redirect header on the POST. A create that SUCCEEDED but answered without that header read as a failure, so the caller retried and Tangled grew duplicates — that is exactly how #397, #398 and #399 were filed for one branch. A response header describes what the server meant to say; it is not the artifact. Now it checks the artifact, in both directions: * BEFORE posting, refuse if an open pull already exists for this source branch, naming it. A retry cannot duplicate, whatever the response said. (--allow-duplicate to override.) * AFTER posting, confirm against the pulls list: a new pull number that did not exist before, whose page names this source branch, IS the success — with or without a redirect header. * Failure is reported only when no such pull appeared. A false failure is worse than a loud error here, because the caller's remedy is to retry. Verified live: a dry-run against a branch that already has a pull refuses with rc=3, naming #417. TWO REAL DEFECTS FOUND BY WRITING THE TESTS. agents.py shelled out to `pgrep -P` and `ps -o comm=`. Neither is on the agent PATH on this host, and a missing binary under shell=True returns rc=127 with EMPTY stdout — indistinguishable from "this process has no children" and "no build is running". So _build_running was ALWAYS False and the stall detector could reboot an agent mid-build. Both now read /proc directly: no PATH dependency, and it cannot fail silently in that direction. That shipped because the unit tests MOCKED pgrep and ps. The fakes stood in for the broken dependency, so the suite passed on a host where neither tool was reachable and never exercised the real path. The tests now patch _proc_descendants and _comms — the seams this repo owns. A test that mocks a dependency proves the mock works. Also fixed a monkeypatch leak those tests had: restoration used a name derivation that silently matched nothing, so the patch escaped into another test class and failed an unrelated test — only in a full run, never when that test ran alone. Now addCleanup, which cannot be ordered wrong. NEW: tests/test_tools.py, 24 tests over tangled_pr, tangled_pr_close and gateway-domain, with every HTTP boundary injected so they run offline. Mutation-checked: breaking classify(), the pull-number regex, the branch match, or the scan bound each turns the suite red. Suite is 93 tests, green, and order-stable across repeated runs. README: a "PATH on a NixOS host" section. Every one of ps, pgrep, free, cmp, awk, curl, diff, strings, nm, getent and ping is INSTALLED here and simply not on the agent PATH, so each reports "command not found" and reads as a missing package. Documents how to check before concluding a tool is absent, how to add the system profile, `nix shell` for what is genuinely missing, and the rule that harness code should not shell out for what the kernel already exposes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V3LdmEL7CvCYTNpoBq1kce
This commit is contained in:
@@ -480,12 +480,72 @@ The agent CLIs themselves (`claude`, `opencode`) are **external, non-Nix tools**
|
||||
per their own docs and make sure they are on `PATH` before launching live agents. The devShell
|
||||
documents this in its banner.
|
||||
|
||||
### PATH on a NixOS host: why a tool that IS installed says "command not found"
|
||||
|
||||
**An agent's `PATH` does not include the system profile.** On `notplants-orchestrator` the agent
|
||||
shell gets a pinned list of individual store paths (bash, git, python, coreutils, findutils, grep,
|
||||
sed, systemd, tmux, openssh, net-tools) and **not** `/run/current-system/sw/bin`. Everything the
|
||||
host declares in `environment.systemPackages` therefore exists and is unreachable.
|
||||
|
||||
Measured 2026-08-21: `ps`, `pgrep`, `free`, `cmp`, `awk`, `curl`, `diff`, `strings`, `nm`, `getent`
|
||||
and `ping` were ALL installed (notplants-nix `hosts/notplants-orchestrator/configuration.nix`,
|
||||
"give agents a real toolbox") and all reported `command not found`.
|
||||
|
||||
**Why this is worse than an inconvenience.** A missing binary run through `shell=True` returns
|
||||
**rc=127 with empty stdout**, and empty stdout is indistinguishable from a real answer of "none":
|
||||
|
||||
- `agents.py` shelled out to `pgrep -P` to find a pane's child processes. With `pgrep` unreachable
|
||||
it returned *no children*, so `_build_running` was **always False** and the stall detector could
|
||||
reboot an agent in the middle of a build. It shipped that way and no test caught it, because the
|
||||
unit tests **mocked `pgrep`** — the fake stood in for the broken dependency.
|
||||
- The same session read an empty `ps` as "0 agent processes running", and an empty `strings` as
|
||||
proof that a binary had its features stripped.
|
||||
|
||||
**Check before concluding a tool is absent:**
|
||||
|
||||
```bash
|
||||
ls /run/current-system/sw/bin/<tool> # installed but unreachable?
|
||||
command -v <tool> # reachable?
|
||||
```
|
||||
|
||||
**Put the system profile on PATH** (appended, so the sandbox's pinned store paths keep priority):
|
||||
|
||||
```bash
|
||||
case ":$PATH:" in *":/run/current-system/sw/bin:"*) ;;
|
||||
*) export PATH="$PATH:/run/current-system/sw/bin" ;; esac
|
||||
```
|
||||
|
||||
For something genuinely not installed, fetch it without changing the host:
|
||||
|
||||
```bash
|
||||
nix shell nixpkgs#tcpdump -c tcpdump ... # one command, nothing persisted
|
||||
nix run nixpkgs#git-filter-repo -- --help
|
||||
```
|
||||
|
||||
`nix` itself may also be off `PATH`; it lives at `/nix/var/nix/profiles/default/bin/nix`.
|
||||
|
||||
**Rule for harness code: do not shell out for something the kernel already exposes.** `agents.py`
|
||||
now reads `/proc/<pid>/stat` and `/proc/<pid>/comm` directly instead of calling `pgrep` and `ps`.
|
||||
That has no PATH dependency and cannot fail silently in the direction that matters. If you must
|
||||
call an external tool, check the return code — never treat empty output as an answer.
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
The `tests/` directory holds the harness's own test suite. One runner drives everything:
|
||||
|
||||
- `tests/test_unit.py` — the harness: config load, kickoff, the phase machine, limit parsing,
|
||||
waiting-until, the build-process detector.
|
||||
- `tests/test_tools.py` — the standalone tools (`tangled_pr.py`, `tangled_pr_close.py`,
|
||||
`tools/gateway-domain.py`). **No network**: every HTTP boundary is injected as a fake `_fetch`.
|
||||
|
||||
**Mock the seam you own, not the tool you depend on.** The build-detector tests used to fake
|
||||
`pgrep` and `ps` subprocess calls, so they passed on a host where neither was reachable and the
|
||||
real defect — empty output read as "no build running" — was invisible to every test. They now
|
||||
patch `_proc_descendants` and `_comms`, the functions this repo owns. A test that mocks a
|
||||
dependency proves the mock works.
|
||||
|
||||
```bash
|
||||
nix develop -c ./tests/run.sh # unit tests always; live backend smokes when available
|
||||
# or just: ./tests/run.sh # (python3 + tmux must be on PATH)
|
||||
|
||||
Reference in New Issue
Block a user