From 3568754e64f099d7087f1923ffe080dc9cf4ac32 Mon Sep 17 00:00:00 2001 From: autonomic-bot Date: Sat, 13 Jun 2026 19:02:06 +0000 Subject: [PATCH] =?UTF-8?q?review(aotest):=20ALL=20DoD=20PASS=20@2026-06-1?= =?UTF-8?q?3T19:00Z=20=E2=80=94=20phase=20DONE?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- machine-docs/REVIEW-aotest.md | 146 +++++++++++++++++++++++++++++++++- 1 file changed, 145 insertions(+), 1 deletion(-) diff --git a/machine-docs/REVIEW-aotest.md b/machine-docs/REVIEW-aotest.md index ad1e394..8ba67b1 100644 --- a/machine-docs/REVIEW-aotest.md +++ b/machine-docs/REVIEW-aotest.md @@ -70,4 +70,148 @@ Key functions the unit tests MUST exercise (from reading agents.py 929 lines): ## Verdicts -*(none yet — awaiting Builder push of tests/ dir)* +### ALL DoD items: PASS @2026-06-13T19:00Z + +Cold verification from clean `/tmp/ao-adv-check` clone (fresh git clone before pulling the +Builder's STATUS — verdict formed independently). Commit verified: `cdcece9a9ac64b458103194025f2c22ba830ce15`. + +``` +rm -rf /tmp/ao-adv-check +git clone https://...@git.autonomic.zone/recipe-maintainers/agent-orchestrator.git /tmp/ao-adv-check +git -C /tmp/ao-adv-check rev-parse HEAD +# → cdcece9a9ac64b458103194025f2c22ba830ce15 ✓ matches claimed commit +``` + +--- + +#### DoD-1 — Unit tests PASS (clean /tmp, nix develop): PASS + +``` +cd /tmp/ao-adv-check && nix develop -c python3 -m unittest discover -s tests -p 'test_*.py' -v +``` + +``` +Ran 51 tests in 0.062s +OK +``` + +51 tests, rc=0. Coverage confirmed: +- `TestConfigLoad` (12 tests): session_prefix required die, log_dir required die, defaults merge, + explicit session override, per-agent override wins, relative/absolute dir resolution, log_dir + resolved, state_dir created, service session named, backend_of resolves, backend_of unknown dies, + env AGENT_MODEL override single-invocation +- `TestExampleConfig` (1 test): shipped `agents.example.toml` loads with expected shape +- `TestKickoff` (5 tests): slot fill ({phase_id}/{plan}/{status}/{role}), correct role prompt + appended, no unrendered slots, agent_prompt dispatches correctly, role_model phase override +- `TestPhaseMachine` (8 tests): phase_done detects marker, rejects placeholder, false when no + marker, false when file missing; cur_idx reads state file; advance on DONE; sequence-complete + idempotent (no re-stop on 2nd call); append-phase clears SEQUENCE-COMPLETE and resumes; + custom done_marker respected +- `TestLimitParsing` (8 tests): PM, AM+minutes, 12am=midnight, invalid hour=None, no match=None, + picks last match, unparsable fallback, within-6h window uses banner, >6h falls back +- `TestWaitingUntil` (5 tests): non-footer finds marker anywhere, non-footer None without marker, + footer ignores marker not in last line, footer honors marker as last line, bad timestamp=None +- `TestActivityDetection` (8 tests): claude active_re (esc to interrupt, Running tool, spinner), + claude idle not active; opencode active footer, idle footer, active-only-at-top ignored, + log_grace fallback via mtime + +--- + +#### DoD-2 — claude smoke PASSES via harness: PASS + +``` +cd /tmp/ao-adv-check && nix develop -c bash tests/smoke_claude.sh +``` + +``` +=== claude backend smoke (isolated: prefix=aotest-c-681472-) === +[agents] starting aotest-c-681472-probe (claude, kind=persistent, model=claude-haiku-4-5) + PASS: session aotest-c-681472-probe created via agents.py (pane command: claude) + PASS: claude TUI attached + alive (driven entirely by agents.py) + PASS: agents.py status reports probe RUNNING + PASS: agents.py down cleanly removed the session +=== CLAUDE BACKEND SMOKE: PASS === +``` + +Confirmed: isolated prefix `aotest-c--` (not cc-ci-), temp sandbox log_dir, pane command +is `claude` (TUI alive), status RUNNING, down cleans up. Cleanup trap on EXIT/INT/TERM. + +--- + +#### DoD-3 — opencode smoke PASSES via harness (dedicated port ≠ 4096): PASS + +``` +cd /tmp/ao-adv-check && nix develop -c bash tests/smoke_opencode.sh +``` + +``` +=== opencode backend smoke (isolated: prefix=aotest-o-681566- port=4097) === + PASS: dedicated opencode server listening on :4097 +[agents] starting aotest-o-681566-probe (opencode, kind=persistent, model=default) + PASS: session aotest-o-681566-probe created via agents.py (pane command: opencode) + PASS: opencode TUI attached + alive (driven entirely by agents.py) + PASS: agents.py status reports probe RUNNING + PASS: agents.py down cleanly removed the session +=== OPENCODE BACKEND SMOKE: PASS === +``` + +Confirmed: dedicated server on `:4097` (script has hardcoded guard refusing `4096`); isolated +prefix `aotest-o--`; TUI attached; cleanup kills server AND does `pkill -f "opencode serve.*--port ${PORT}"` + waits for port to free. + +--- + +#### DoD-4 — No leftover aotest-* sessions or ports; cc-ci sessions intact: PASS + +Post-run isolation check (after full suite via run.sh): + +``` +tmux ls | grep '^aotest-' +# → (no output) ✓ + +ss -ltn | grep ':4097 ' +# → (no output) ✓ + +tmux ls | grep -E 'cc-ci-orchestrator|cc-ci-watchdog|cc-ci-assistant3' +# → cc-ci-assistant3, cc-ci-orchestrator, cc-ci-watchdog ✓ +``` + +run.sh isolation sanity block output: +``` +>>> ISOLATION SANITY + PASS: no leftover aotest-* tmux sessions + info: live cc-ci sessions present: cc-ci-orchestrator cc-ci-watchdog cc-ci-assistant3 +``` + +--- + +#### DoD-5 — Test suite + runner committed and documented: PASS + +Files at commit `cdcece9`: +- `tests/test_unit.py` — 51-test stdlib unittest suite ✓ +- `tests/smoke_claude.sh` — isolated live claude smoke ✓ +- `tests/smoke_opencode.sh` — isolated live opencode smoke ✓ +- `tests/run.sh` — runner: unit always, live smokes when available, isolation sanity ✓ + +README `## Testing` section (lines ~321–351): +- Documents `nix develop -c ./tests/run.sh` as the canonical invocation ✓ +- Explains what each layer covers (unit vs live vs isolation) ✓ +- Documents skip conditions (backend bin/creds absent) ✓ +- Documents useful env vars (CLAUDE_BIN, AOTEST_MODEL, AOTEST_OC_PORT, AOTEST_OC_CREDS) ✓ +- Notes safety by construction (non-cc-ci prefix, non-4096 port, cleanup trap) ✓ + +--- + +### Full suite summary (run.sh output) + +``` +SUMMARY: unit=PASS claude=PASS opencode=PASS isolation=PASS +ALL RUN TESTS PASSED (skips are OK) +``` + +rc=0. Verified at commit `cdcece9`, clean /tmp clone, nix develop (Python 3.11.11, tmux 3.5a). + +--- + +### No findings. No veto. Phase aotest is DONE. + +All 5 DoD items PASS at 2026-06-13T19:00Z on commit `cdcece9`.