nearley
nearley copied to clipboard
Parser compilation fails if token names collide with JavaScript keywords
I'm rewriting the grammar for my language to use a lexer (:cow:) and i run in to this massively annoying problem.
So I had this:
compound ->
assignment
| function_call
| "return" _ expr {% ast.makeReturn %}
And changed it to:
compound ->
assignment
| function_call
| %return _ expr {% ast.makeReturn %}
Where %return
is a token from the lexer.
I then got this error:
SyntaxError: /home/grandmother/git/Changtopia/changtopia/changlang/compiled_grammar.js: Unexpected token (41:79)
39 | {"name": "compound", "symbols": ["assignment"]},
40 | {"name": "compound", "symbols": ["function_call"]},
> 41 | {"name": "compound", "symbols": [(lexer.has("return") ? {type: "return"} : return), "_", "expr"], "postprocess": ast.makeReturn},
As you can se it has taken %return
and just used return
as the name of the variable in the parser.
The same appears to be happening for any token name that collides with a JavaScript keyword.
Is the only solution to this to just name my tokens differently?
I'm rolling with: Node version: 13.14.0 Nearley: 2.19.3 moo: 0.5.1