agent-orchestrator-benchmark/calculators/builder-adversary-min/run-01/machine-docs/JOURNAL-lex.md

JOURNAL-lex

Implementation notes

Built calc/lexer.py with:

• Token dataclass with kind: str and value: Union[int, float, None]
• LexError(Exception) for invalid characters
• tokenize(src: str) -> list[Token] scanning character by character

Number scanning handles all three forms: integer (42), standard float (3.14), leading-dot float (.5), and trailing-dot float (10.). The key invariant: when ch is . we only enter number scanning if the next char is also a digit (to avoid confusing . used as an operator in other contexts). Trailing dot (10.) is handled because after scanning digits we check if the next char is . and absorb it into a float.

14 tests written covering D1-D4 plus edge cases.