REVIEW — Phase `lex`

Adversary: cold-verification log. One entry per gate per pass.

Verdicts

lex/D1: PASS @2026-06-15T03:52Z

Cold-ran from own clone at commit 462ad1f.

tokenize("42")  → [('NUMBER', 42), ('EOF', None)]  ✓ int
tokenize(".5")  → [('NUMBER', 0.5), ('EOF', None)]  ✓ float
tokenize("10.") → [('NUMBER', 10.0), ('EOF', None)] ✓ float
tokenize("3.14")→ [('NUMBER', 3.14), ('EOF', None)] ✓ float

EOF always appended as final token. int/float types correct.

lex/D2: PASS @2026-06-15T03:52Z

tokenize("1+2*3") → ['NUMBER', 'PLUS', 'NUMBER', 'STAR', 'NUMBER', 'EOF']  ✓
tokenize("()+-(*/") → LPAREN RPAREN PLUS MINUS LPAREN STAR SLASH EOF  ✓

All six operator/paren kinds map correctly.

lex/D3: PASS @2026-06-15T03:52Z

tokenize("  12  +  3 ") → ['NUMBER', 'PLUS', 'NUMBER', 'EOF']  ✓ spaces skipped
tokenize("1\t+\t2")     → ['NUMBER', 'PLUS', 'NUMBER', 'EOF']  ✓ tabs skipped
tokenize("1 @ 2") raises LexError: unexpected character '@' at position 2  ✓
tokenize("hello")  raises LexError: unexpected character 'h' at position 0  ✓
tokenize("$10")    raises LexError  ✓

LexError message contains the offending character and position.

lex/D4: PASS @2026-06-15T03:52Z

$ python -m unittest -q
Ran 14 tests in 0.000s
OK

14 tests, 0 failures. Plan's canonical verify commands all produce expected output:

3.5*(1-2) → [('NUMBER', 3.5), ('STAR', '*'), ('LPAREN', '('), ('NUMBER', 1), ('MINUS', '-'), ('NUMBER', 2), ('RPAREN', ')'), ('EOF', None)]
1 @ 2 → raises LexError

Adversary findings

F1 (non-blocking) — malformed number literals leak ValueError instead of LexError

tokenize("..") → ValueError: could not convert string to float: '..'
tokenize("1.2.3") → ValueError: could not convert string to float: '1.2.3'

The number-scanning loop greedily consumes all [0-9.] chars, then calls float() which throws a raw ValueError. The DoD (D3) only specifies invalid characters (@ $ letters) and these cases are not in the test suite, so this does not block DONE. Noted for later phases.

Summary

All four DoD gates PASS. No veto. Builder may write "## DONE" to STATUS-lex.md.

2.2 KiB Raw Blame History

REVIEW — Phase lex