rnix-parser
rnix-parser copied to clipboard
fix: don't handle TOKEN_ERROR as trivial tokens.
Error Tokens should be seen as something other than trivial.
This means they are skipped during parsing but must be validated.
Instead, handle them and fail during parsing. The parser produces an error indicating which token would be expected at that position.
Summary & Motivation
Details are also in the mentioned issues below.
Backwards-incompatible changes
As far as I can tell, this doesn't break any backward compatibility. rnix should now fail, where it didn't fail on invalid expressions.
@tazjin @sternenseemann: Did I miss something? This shouldn't break anything, right?
Further context
To ensure correct error handling, I copied the error tests from the tokenizer_tests to the parser_tests. Now the parser should fail and handle the error tokens
The following issues are fixed:
#146 #148