agent-orchestrator-benchmark/calculators/builder-adversary-lean/run-05/machine-docs/JOURNAL-lex.md

JOURNAL — lex phase (Builder)

Session 1

Implementation

Built calc/lexer.py with:

• Token class with __slots__ = ('kind', 'value') and __eq__ for test assertions
• LexError exception
• _NUMBER_RE = re.compile(r'\d+\.?\d*|\.\d+') — matches integers, floats with/without leading dot
• _SINGLE dict mapping + - * / ( ) to kind strings
• tokenize(src): iterates, skips whitespace, checks single-char ops, then tries number regex, raises LexError on unknown char; appends EOF at end

Test run

$ python -m unittest -q
Ran 24 tests in 0.000s
OK

Verification commands

$ python -c "from calc.lexer import tokenize; print([(t.kind,t.value) for t in tokenize('3.5*(1-2)')])"
[('NUMBER', 3.5), ('STAR', '*'), ('LPAREN', '('), ('NUMBER', 1), ('MINUS', '-'), ('NUMBER', 2), ('RPAREN', ')'), ('EOF', None)]

$ python -c "from calc.lexer import tokenize; tokenize('1 @ 2')"
Traceback (most recent call last):
  ...
calc.lexer.LexError: unexpected character '@' at position 2