tangled_pr: verify against the pulls list, not a response header; and a tools test suite

THE BUG THAT PROMPTED THIS. tangled_pr.py judged success ONLY by an HX-Redirect header on the POST.
A create that SUCCEEDED but answered without that header read as a failure, so the caller retried
and Tangled grew duplicates — that is exactly how #397, #398 and #399 were filed for one branch. A
response header describes what the server meant to say; it is not the artifact.

Now it checks the artifact, in both directions:
  * BEFORE posting, refuse if an open pull already exists for this source branch, naming it. A
    retry cannot duplicate, whatever the response said. (--allow-duplicate to override.)
  * AFTER posting, confirm against the pulls list: a new pull number that did not exist before,
    whose page names this source branch, IS the success — with or without a redirect header.
  * Failure is reported only when no such pull appeared. A false failure is worse than a loud
    error here, because the caller's remedy is to retry.
Verified live: a dry-run against a branch that already has a pull refuses with rc=3, naming #417.

TWO REAL DEFECTS FOUND BY WRITING THE TESTS.

agents.py shelled out to `pgrep -P` and `ps -o comm=`. Neither is on the agent PATH on this host,
and a missing binary under shell=True returns rc=127 with EMPTY stdout — indistinguishable from
"this process has no children" and "no build is running". So _build_running was ALWAYS False and
the stall detector could reboot an agent mid-build. Both now read /proc directly: no PATH
dependency, and it cannot fail silently in that direction.

That shipped because the unit tests MOCKED pgrep and ps. The fakes stood in for the broken
dependency, so the suite passed on a host where neither tool was reachable and never exercised the
real path. The tests now patch _proc_descendants and _comms — the seams this repo owns. A test that
mocks a dependency proves the mock works.

Also fixed a monkeypatch leak those tests had: restoration used a name derivation that silently
matched nothing, so the patch escaped into another test class and failed an unrelated test — only
in a full run, never when that test ran alone. Now addCleanup, which cannot be ordered wrong.

NEW: tests/test_tools.py, 24 tests over tangled_pr, tangled_pr_close and gateway-domain, with every
HTTP boundary injected so they run offline. Mutation-checked: breaking classify(), the pull-number
regex, the branch match, or the scan bound each turns the suite red. Suite is 93 tests, green, and
order-stable across repeated runs.

README: a "PATH on a NixOS host" section. Every one of ps, pgrep, free, cmp, awk, curl, diff,
strings, nm, getent and ping is INSTALLED here and simply not on the agent PATH, so each reports
"command not found" and reads as a missing package. Documents how to check before concluding a tool
is absent, how to add the system profile, `nix shell` for what is genuinely missing, and the rule
that harness code should not shell out for what the kernel already exposes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3LdmEL7CvCYTNpoBq1kce
This commit is contained in:
2026-08-21 03:50:11 +00:00
co-authored by Claude Fable 5
parent e1ba9b39be
commit e185cec88c
5 changed files with 427 additions and 29 deletions
+60
View File
@@ -480,12 +480,72 @@ The agent CLIs themselves (`claude`, `opencode`) are **external, non-Nix tools**
per their own docs and make sure they are on `PATH` before launching live agents. The devShell
documents this in its banner.
### PATH on a NixOS host: why a tool that IS installed says "command not found"
**An agent's `PATH` does not include the system profile.** On `notplants-orchestrator` the agent
shell gets a pinned list of individual store paths (bash, git, python, coreutils, findutils, grep,
sed, systemd, tmux, openssh, net-tools) and **not** `/run/current-system/sw/bin`. Everything the
host declares in `environment.systemPackages` therefore exists and is unreachable.
Measured 2026-08-21: `ps`, `pgrep`, `free`, `cmp`, `awk`, `curl`, `diff`, `strings`, `nm`, `getent`
and `ping` were ALL installed (notplants-nix `hosts/notplants-orchestrator/configuration.nix`,
"give agents a real toolbox") and all reported `command not found`.
**Why this is worse than an inconvenience.** A missing binary run through `shell=True` returns
**rc=127 with empty stdout**, and empty stdout is indistinguishable from a real answer of "none":
- `agents.py` shelled out to `pgrep -P` to find a pane's child processes. With `pgrep` unreachable
it returned *no children*, so `_build_running` was **always False** and the stall detector could
reboot an agent in the middle of a build. It shipped that way and no test caught it, because the
unit tests **mocked `pgrep`** — the fake stood in for the broken dependency.
- The same session read an empty `ps` as "0 agent processes running", and an empty `strings` as
proof that a binary had its features stripped.
**Check before concluding a tool is absent:**
```bash
ls /run/current-system/sw/bin/<tool> # installed but unreachable?
command -v <tool> # reachable?
```
**Put the system profile on PATH** (appended, so the sandbox's pinned store paths keep priority):
```bash
case ":$PATH:" in *":/run/current-system/sw/bin:"*) ;;
*) export PATH="$PATH:/run/current-system/sw/bin" ;; esac
```
For something genuinely not installed, fetch it without changing the host:
```bash
nix shell nixpkgs#tcpdump -c tcpdump ... # one command, nothing persisted
nix run nixpkgs#git-filter-repo -- --help
```
`nix` itself may also be off `PATH`; it lives at `/nix/var/nix/profiles/default/bin/nix`.
**Rule for harness code: do not shell out for something the kernel already exposes.** `agents.py`
now reads `/proc/<pid>/stat` and `/proc/<pid>/comm` directly instead of calling `pgrep` and `ps`.
That has no PATH dependency and cannot fail silently in the direction that matters. If you must
call an external tool, check the return code — never treat empty output as an answer.
---
## Testing
The `tests/` directory holds the harness's own test suite. One runner drives everything:
- `tests/test_unit.py` — the harness: config load, kickoff, the phase machine, limit parsing,
waiting-until, the build-process detector.
- `tests/test_tools.py` — the standalone tools (`tangled_pr.py`, `tangled_pr_close.py`,
`tools/gateway-domain.py`). **No network**: every HTTP boundary is injected as a fake `_fetch`.
**Mock the seam you own, not the tool you depend on.** The build-detector tests used to fake
`pgrep` and `ps` subprocess calls, so they passed on a host where neither was reachable and the
real defect — empty output read as "no build running" — was invisible to every test. They now
patch `_proc_descendants` and `_comms`, the functions this repo owns. A test that mocks a
dependency proves the mock works.
```bash
nix develop -c ./tests/run.sh # unit tests always; live backend smokes when available
# or just: ./tests/run.sh # (python3 + tmux must be on PATH)