recipe-maintainers/agent-orchestrator-benchmark
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
bb85aa9f1170edbf6c3ab8b1bddfb4e7880f6d1a
agent-orchestrator-benchmark/calculators/builder-adversary-lean/run-03/machine-docs/REVIEW-eval.md

3.3 KiB
Raw Blame History

REVIEW — phase eval (Adversary)

Adversary-owned. Builder: read-only.

Status summary

All 5 gates PASSED. No vetoes. Recommending DONE.

Gate Verdict Timestamp Notes
D1 PASS 2026-06-15T05:18:03Z All plan spot-checks verified cold
D2 PASS 2026-06-15T05:18:03Z True division, EvalError wraps div-by-zero
D3 PASS 2026-06-15T05:18:03Z Whole→int, non-whole→float, all type assertions
D4 PASS 2026-06-15T05:18:03Z Exit 0 valid, exit 1+stderr on error; stdout/stderr clean
D5 PASS 2026-06-15T05:18:03Z 68 tests, 0 failures; all 3 prior phases intact

Verdicts

D1 — arithmetic: PASS @2026-06-15T05:18:03Z

Cold verification (fresh shell, work-adv clone):

calc("2+3*4")   → 14   ✓
calc("(2+3)*4") → 20   ✓
calc("8-3-2")   → 3    ✓
calc("-2+5")    → 3    ✓
calc("2*-3")    → -6   ✓

Break-it probes: negative results (3-7→-4), double unary (--3→3), nested parens. All correct.

CLI: python calc.py "8-3-2" → 3; python calc.py "-2+5" → 3; python calc.py "2*-3" → -6. ✓

D2 — division: PASS @2026-06-15T05:18:03Z

Cold verification:

  • calc("7/2") → 3.5 ✓
  • calc("1/0") raises EvalError("division by zero") ✓ (not ZeroDivisionError)
  • calc("5/(3-3)") raises EvalError ✓ (zero through expression)
  • calc("12/4/3") → 1 (left-associative chain) ✓

CLI: python calc.py "1/0" → stderr error: division by zero, exit 1 ✓

D3 — result type: PASS @2026-06-15T05:18:03Z

Cold verification:

  • calc("4/2") → 2, isinstance(result, int) ✓
  • calc("7/2") → 3.5, isinstance(result, float) ✓
  • calc("1+2") → isinstance(result, int) ✓
  • calc("-3") → -3, isinstance(result, int) ✓
  • calc("1.5+1.5") → 3, isinstance(result, int) ✓ (whole float coerced)
  • calc("2.5*2") → 5, isinstance(result, int) ✓
  • calc("-1.5") → -1.5, isinstance(result, float) ✓

CLI: python calc.py "4/2" → 2 (no trailing .0) ✓; python calc.py "7/2" → 3.5 ✓

D4 — CLI: PASS @2026-06-15T05:18:03Z

Cold verification:

Command stdout stderr exit
python calc.py "2+3*4" 14 (empty) 0 ✓
python calc.py "(2+3)*4" 20 (empty) 0 ✓
python calc.py "7/2" 3.5 (empty) 0 ✓
python calc.py "4/2" 2 (empty) 0 ✓
python calc.py "1/0" (empty) error: division by zero 1 ✓
python calc.py "1 +" (empty) error: unexpected token 'EOF' (None) 1 ✓
python calc.py (no args) (empty) usage msg 1 ✓
python calc.py "1+2" extra (empty) usage msg 1 ✓

stderr/stdout separation confirmed: errors never appear on stdout, results never leak to stderr.

D5 — tests green + end-to-end: PASS @2026-06-15T05:18:03Z

Cold verification:

python -m unittest -q
----------------------------------------------------------------------
Ran 68 tests in 0.001s

OK

Breakdown: 25 lexer + 23 parser + 20 evaluator = 68 tests. 0 failures. All prior phases intact. Each test class verified independently (TestArithmetic: 10, TestDivision: 5, TestResultType: 5).

Adversary findings

(none — all gates pass cleanly)