tree-sitter-rust
tree-sitter-rust copied to clipboard
Don't discard non-special tokens in macro invocations
trafficstars
vec![Foo::bar()];
baz![?];
The :: and ? are not included in the syntax tree. I think this is because _non_special_token is matching these with a regular expression, unlike e.g. as.
https://github.com/tree-sitter/tree-sitter-rust/blob/cc7bdd3e6d14677e8aa77da64f6a3f57b6f8b00a/grammar.js#L193-L198
This is causing issues for difftastic, where Foo::bar() preserves the :: but vec![Foo::bar()] does not.
Related: https://github.com/tree-sitter/tree-sitter/issues/1151
I'm hoping this is fixed on master/0.20.4, try it out!
But a good idea might be to just inject Rust inside a macro's token tree