agent-orchestrator-benchmark/calculators/builder-adversary/run-04/machine-docs/REVIEW-eval.md

REVIEW-eval.md — Adversary verdicts for phase eval

SSOT: /home/loops/project-orchestrator/projects/agent-orchestrator-benchmark/plans/calc/eval.md

Status: ALL GATES PASS

All gates verified cold @2026-06-15T01:29Z.

Gate verdicts

eval/D1: PASS @2026-06-15T01:29Z

Cold-run evidence:

python calc.py "2+3*4"   → 14, exit 0
python calc.py "(2+3)*4" → 20, exit 0
python calc.py "8-3-2"   → 3,  exit 0
python calc.py "-2+5"    → 3,  exit 0
python calc.py "2*-3"    → -6, exit 0

All 5 spec examples from plan correct. Precedence, parens, unary minus all work.

eval/D2: PASS @2026-06-15T01:29Z

Cold-run evidence:

python calc.py "7/2"    → 3.5, exit 0    (true division)
python calc.py "1/0"    → stderr: "error: division by zero", exit 1

Also verified via Python API: calc("1/0") raises EvalError not bare ZeroDivisionError. Sub-expression div-by-zero: calc("5/(2-2)") → EvalError: division by zero. OK.

eval/D3: PASS @2026-06-15T01:29Z

Cold-run evidence:

python calc.py "4/2"  → 2   (no .0)
python calc.py "7/2"  → 3.5 (float with decimal)
python calc.py "0"    → 0   (not 0.0)
python calc.py "5-5"  → 0   (int zero, not 0.0)

_normalize() converts whole-valued float to int. str(calc("4/2")) == "2" confirmed.

eval/D4: PASS @2026-06-15T01:29Z

Cold-run evidence:

python calc.py "2+3*4"   → stdout: 14, exit 0
python calc.py "1/0"     → stderr: "error: division by zero", exit 1 (no traceback)
python calc.py "1 +"     → stderr: "error: unexpected end of input", exit 1 (no traceback)
python calc.py           → stderr: "usage: calc.py <expression>", exit 1
python calc.py "1" "x"  → stderr: "usage: calc.py <expression>", exit 1

Verified stderr separation: error text absent when 2>/dev/null, present when 1>/dev/null. No traceback leaks confirmed via grep (no "Traceback", "File", "line" in stderr output).

eval/D5: PASS @2026-06-15T01:29Z

Cold-run evidence:

python -m unittest -q
Ran 60 tests in 0.001s
OK

16 lex + 27 parse + 17 evaluator = 60 total. 0 failures. No regressions. Lexer and parser suites run in isolation: python -m unittest calc.test_lexer calc.test_parser → 43 tests OK.

Findings

No defects found. No VETO.

Notes

Did NOT read JOURNAL before forming verdicts (isolation discipline maintained).