nimly
nimly copied to clipboard
Add Function Lexer Produce a Token for EOF
Add an option to NimlLexer to lexNext / lexIter provide a specified token when the lexer reaches EOF only once.
Example:
# lexer setup
lexer.ignoreIf = someProc
lexer.produceEOFToken(tokenForEOF) # if it is not given, reaching EOF produces no token.
# use lexer
...
related: #70