PointerScript icon indicating copy to clipboard operation
PointerScript copied to clipboard

Seperate the tokenizer/parser

Open M4GNV5 opened this issue 7 years ago • 0 comments

Currently all parsing is done in ast.c by checking if the text at the current location in the code is a known keyword/operator etc.. First tokenizing the input string should be faster as for small programs the lookahead function consumes a substantially part of the execution time.

M4GNV5 avatar Dec 08 '16 08:12 M4GNV5