agent-orchestrator-benchmark/calculators/builder-adversary/run-05/machine-docs/REVIEW-lex.md

REVIEW — phase lex (Adversary)

Status

All 4 gates verified. Phase COMPLETE.

Verdicts

lex/D1: PASS @2026-06-15T01:35:56Z

Cold-run evidence:

tokenize('42')   → [('NUMBER', 42), ('EOF', None)]   — int ✓
tokenize('3.14') → [('NUMBER', 3.14), ('EOF', None)] — float ✓
tokenize('.5')   → [('NUMBER', 0.5), ('EOF', None)]  — float ✓
tokenize('10.')  → [('NUMBER', 10.0), ('EOF', None)] — float ✓
type(tokenize('42')[0].value)   == int   ✓
type(tokenize('3.14')[0].value) == float ✓

Value is int for integers, float for floats. DoD fully met.

lex/D2: PASS @2026-06-15T01:35:56Z

Cold-run evidence:

tokenize('1+2*3')  → ['NUMBER','PLUS','NUMBER','STAR','NUMBER','EOF'] ✓
tokenize('+-*/()')  → ['PLUS','MINUS','STAR','SLASH','LPAREN','RPAREN','EOF'] ✓
tokenize('3.5*(1-2)') → correct kind/value pairs ✓

All 6 operator/paren kinds present and correct.

lex/D3: PASS @2026-06-15T01:35:56Z

Cold-run evidence:

tokenize('  12  +  3 ') → NUMBER(12) PLUS NUMBER(3) EOF — spaces skipped ✓
tokenize('1\t+\t2')     → NUMBER PLUS NUMBER EOF — tabs skipped ✓
tokenize('1 @ 2')  → LexError: unexpected character '@' at position 2 ✓
tokenize('$5')     → LexError: unexpected character '$' at position 0 ✓
tokenize('x')      → LexError: unexpected character 'x' at position 0 ✓

Offending char and position both present in message. DoD fully met.

lex/D4: PASS @2026-06-15T01:35:56Z

python -m unittest -q
Ran 22 tests in 0.001s
OK

Zero failures, zero errors. All 22 tests cover D1–D3 including required cases: " 12 + 3 ", "3.5*(1-2)", and "1 @ 2" raising LexError.

Informational finding (non-blocking)

F1 — lone . raises ValueError not LexError: tokenize('.') crashes with ValueError: could not convert string to float: '.' instead of a LexError. The DoD does not require this case to be handled, so this is NOT a blocker for any gate. Noted for future phases that may extend the lexer.

No VETO. All DoD items independently verified.