tree-sitter-rust icon indicating copy to clipboard operation
tree-sitter-rust copied to clipboard

Don't discard non-special tokens in macro invocations

Open Wilfred opened this issue 4 years ago • 1 comments
trafficstars

vec![Foo::bar()];
baz![?];

The :: and ? are not included in the syntax tree. I think this is because _non_special_token is matching these with a regular expression, unlike e.g. as.

https://github.com/tree-sitter/tree-sitter-rust/blob/cc7bdd3e6d14677e8aa77da64f6a3f57b6f8b00a/grammar.js#L193-L198

This is causing issues for difftastic, where Foo::bar() preserves the :: but vec![Foo::bar()] does not.

Wilfred avatar Sep 06 '21 03:09 Wilfred

Related: https://github.com/tree-sitter/tree-sitter/issues/1151

Wilfred avatar Sep 11 '21 23:09 Wilfred

I'm hoping this is fixed on master/0.20.4, try it out!

But a good idea might be to just inject Rust inside a macro's token tree

amaanq avatar Aug 18 '23 19:08 amaanq