agent-orchestrator-benchmark/calculators/builder-adversary-deferred/run-05/machine-docs/JOURNAL-lex.md

JOURNAL — lex phase (Builder)

Implementation

Built calc/lexer.py with:

• Token dataclass with kind: str and value: int | float | str | None
• LexError(Exception) with message including offending character and position
• tokenize(src: str) -> list[Token] scanning left-to-right with index

Number parsing handles three forms: pure int (42), float (3.14), leading-dot float (.5), trailing-dot float (10.). The scanner reads digits first, then checks for a . to switch to float mode.

Test Run

python -m unittest -v
test_float ... ok
test_integer ... ok
test_leading_dot ... ok
test_trailing_dot ... ok
test_all_operators ... ok
test_expression ... ok
test_complex_expr ... ok
test_lex_error_at_sign ... ok
test_lex_error_dollar ... ok
test_lex_error_letter ... ok
test_whitespace_skipped ... ok
Ran 11 tests in 0.000s — OK

Verify Commands Output

$ python -m unittest -q
Ran 11 tests in 0.000s
OK

$ python -c "from calc.lexer import tokenize; print([(t.kind,t.value) for t in tokenize('3.5*(1-2)')])"
[('NUMBER', 3.5), ('STAR', '*'), ('LPAREN', '('), ('NUMBER', 1), ('MINUS', '-'), ('NUMBER', 2), ('RPAREN', ')'), ('EOF', None)]

$ python -c "from calc.lexer import tokenize; tokenize('1 @ 2')"
Traceback (most recent call last):
  ...
calc.lexer.LexError: unexpected character '@' at position 2