artifacts: add calculators/ — the 30 built calculators (5/variant) + machine-docs + git logs
This commit is contained in:
@ -0,0 +1,74 @@
|
||||
# REVIEW-eval.md — Adversary verdicts for phase `eval`
|
||||
|
||||
SSOT: /home/loops/project-orchestrator/projects/agent-orchestrator-benchmark/plans/calc/eval.md
|
||||
|
||||
## Status: ALL GATES PASS
|
||||
|
||||
All gates verified cold @2026-06-15T01:29Z.
|
||||
|
||||
## Gate verdicts
|
||||
|
||||
### eval/D1: PASS @2026-06-15T01:29Z
|
||||
|
||||
Cold-run evidence:
|
||||
```
|
||||
python calc.py "2+3*4" → 14, exit 0
|
||||
python calc.py "(2+3)*4" → 20, exit 0
|
||||
python calc.py "8-3-2" → 3, exit 0
|
||||
python calc.py "-2+5" → 3, exit 0
|
||||
python calc.py "2*-3" → -6, exit 0
|
||||
```
|
||||
All 5 spec examples from plan correct. Precedence, parens, unary minus all work.
|
||||
|
||||
### eval/D2: PASS @2026-06-15T01:29Z
|
||||
|
||||
Cold-run evidence:
|
||||
```
|
||||
python calc.py "7/2" → 3.5, exit 0 (true division)
|
||||
python calc.py "1/0" → stderr: "error: division by zero", exit 1
|
||||
```
|
||||
Also verified via Python API: `calc("1/0")` raises `EvalError` not bare `ZeroDivisionError`.
|
||||
Sub-expression div-by-zero: `calc("5/(2-2)")` → `EvalError: division by zero`. OK.
|
||||
|
||||
### eval/D3: PASS @2026-06-15T01:29Z
|
||||
|
||||
Cold-run evidence:
|
||||
```
|
||||
python calc.py "4/2" → 2 (no .0)
|
||||
python calc.py "7/2" → 3.5 (float with decimal)
|
||||
python calc.py "0" → 0 (not 0.0)
|
||||
python calc.py "5-5" → 0 (int zero, not 0.0)
|
||||
```
|
||||
`_normalize()` converts whole-valued float to int. `str(calc("4/2")) == "2"` confirmed.
|
||||
|
||||
### eval/D4: PASS @2026-06-15T01:29Z
|
||||
|
||||
Cold-run evidence:
|
||||
```
|
||||
python calc.py "2+3*4" → stdout: 14, exit 0
|
||||
python calc.py "1/0" → stderr: "error: division by zero", exit 1 (no traceback)
|
||||
python calc.py "1 +" → stderr: "error: unexpected end of input", exit 1 (no traceback)
|
||||
python calc.py → stderr: "usage: calc.py <expression>", exit 1
|
||||
python calc.py "1" "x" → stderr: "usage: calc.py <expression>", exit 1
|
||||
```
|
||||
Verified stderr separation: error text absent when 2>/dev/null, present when 1>/dev/null.
|
||||
No traceback leaks confirmed via grep (no "Traceback", "File", "line" in stderr output).
|
||||
|
||||
### eval/D5: PASS @2026-06-15T01:29Z
|
||||
|
||||
Cold-run evidence:
|
||||
```
|
||||
python -m unittest -q
|
||||
Ran 60 tests in 0.001s
|
||||
OK
|
||||
```
|
||||
16 lex + 27 parse + 17 evaluator = 60 total. 0 failures. No regressions.
|
||||
Lexer and parser suites run in isolation: `python -m unittest calc.test_lexer calc.test_parser` → 43 tests OK.
|
||||
|
||||
## Findings
|
||||
|
||||
No defects found. No VETO.
|
||||
|
||||
## Notes
|
||||
|
||||
Did NOT read JOURNAL before forming verdicts (isolation discipline maintained).
|
||||
Reference in New Issue
Block a user