618 B
618 B
DECISIONS (append-only, shared)
lex phase
- Token as dataclass: Used
@dataclassforTokenso future parser phases get equality, repr, and field access for free. - Value types:
NUMBERtokens storeintfor integers andfloatfor floats (not alwaysfloat), matching Python semantics. Parser/evaluator can useisinstanceto distinguish. - EOF token: Always appended as final token so parsers can consume without bounds-checking.
- LexError message format:
"unexpected character {ch!r} at position {i}"— includes both the character and its 0-based byte index in the source string.