1.6 KiB
DONE
Phase: lex — tokenizer
All DoD items self-certified (BUILD phase — deferred Adversary review).
D1 — numbers
WHAT: integers and floats tokenize to NUMBER with int/float value; EOF appended.
HOW: python -c "from calc.lexer import tokenize; print([(t.kind,t.value) for t in tokenize('42')])"
EXPECTED: [('NUMBER', 42), ('EOF', None)]
WHERE: calc/lexer.py
D2 — operators & parens
WHAT: + - * / ( ) each produce correct kind token.
HOW: python -c "from calc.lexer import tokenize; print([t.kind for t in tokenize('1+2*3')])"
EXPECTED: ['NUMBER', 'PLUS', 'NUMBER', 'STAR', 'NUMBER', 'EOF']
WHERE: calc/lexer.py
D3 — whitespace & errors
WHAT: spaces/tabs skipped; invalid char raises LexError with char + position.
HOW: python -c "from calc.lexer import tokenize; tokenize('1 @ 2')"
EXPECTED: raises calc.lexer.LexError: unexpected character '@' at position 2
WHERE: calc/lexer.py
D4 — tests green
WHAT: 14 unittest tests in calc/test_lexer.py, 0 failures.
HOW: python -m unittest -q
EXPECTED: Ran 14 tests in 0.000s\nOK
WHERE: calc/test_lexer.py
Plan verify commands (verbatim from lex.md)
python -m unittest -q
python -c "from calc.lexer import tokenize; print([(t.kind,t.value) for t in tokenize('3.5*(1-2)')])"
python -c "from calc.lexer import tokenize; tokenize('1 @ 2')"
EXPECTED outputs:
Ran 14 tests in 0.000s\nOK[('NUMBER', 3.5), ('STAR', '*'), ('LPAREN', '('), ('NUMBER', 1), ('MINUS', '-'), ('NUMBER', 2), ('RPAREN', ')'), ('EOF', None)]calc.lexer.LexError: unexpected character '@' at position 2