tree-sitter-rust
tree-sitter-rust copied to clipboard
tree-sitter
Hard to get some symbols inside macro calls and definitions on the user side
Not sure if it's a proper bug, but I got myself unable to classify some symbols on Atom inside macros.
Like, when I define something like:
'"+"' : 'keyword.operator.add'
It will classify all + signs, except inside macro invocations and macro definitions
The same are true for all operator-like symbols and the in, but other keywords work just fine.
Looking at the code, I suspect that this happens inside _non_special_token, where in is not there and the operator-like symbols are defined by a regular expression /[/_\-=->,;:::!=?.@*=/='&=#%=^=+<>|~]+/, not separated strings.
Again, not sure if it can be considered a bug or if it should be handled in some way on the user side.
No, I think it's reasonable to change the definition of non_special_token to treat all of the tokens separately. I can't remember exactly why it's currently defined with one single token.
Maybe because it is allowed any kind of string, for example I could do mymacro!(1 <--> 10) or some kind of haskell like operator, are all valid in most cases.
But I would like to at least the normal Rust operators be able to be captured inside macros.
In the case of Atom, it should be re-parsing the body of the macro with the Rust parser, in order to highlight it correctly. There aren't well-documented APIs for accessing these nested parse trees, but I think you can use this method to access nodes from 'syntax layers' other than the root layer.
I can use this method inside of the Atom tree-sitter grammar definition? And how so?
Oh, I thought you were wanting to access the symbols programatically. For highlighting, I believe that should already work out of the box, because Atom re-parses the contents of token trees as rust source code. This is set up here: https://github.com/atom/atom/blob/0b34be796312a94331acbc4602944cfe3666c76a/packages/language-rust-bundled/lib/main.js#L2-L13.
Yeah, true, doing that it does reparse the token tree, but that I would like to happen on token_tree_pattern as well on macro_definition, that is possible?
This seems to be of similar concern to https://github.com/tree-sitter/tree-sitter-rust/issues/98
