elisp-tree-sitter
elisp-tree-sitter copied to clipboard
emacs-tree-sitter
Support local queries/predicates
The original issue is from #93.
It will be great if this feature is supported!
I assume this means Tree-sitter's Local Variables feature? If so, I would love to see this as well.
Would it need to be implemented in Rust, or can it be done in Lisp? If Lisp, I might be able to take a stab at it.
It needs some Rust code to implement the custom predicates, e.g. (.is-not? local).
You can study Tree-sitter's implementation. It queries the whole buffer in a single pass using a combined query (locals + highlight), It doesn't handle changes, so it cannot be applied as-is.
There's also nvim-treesitter's implementation of locals, which also seems to be single-pass and whole-buffer, but does not support highlighting (locals only).
An MVP prototype could do it in 2 passes: The first pass queries (the whole buffer) locals, and provides data for the custom predicates used by the second pass. From there we can start optimizing things.
We have a implementation of local highlighting (just use is-local?) But it is not used because our current implementation is to slow (mainly wrong data structures and single buffer pass). This is why it isn't used for highlighting at the moment.
Our predicates are custom Lua functions so they can use data from a cached local passes.
EDIT: our local query works like this (is? @variable "parameter") to check whether an identifier is a function parameter (or local variable or function).
We have a implementation of local highlighting (just use is-local?) But it is not used because our current implementation is to slow (mainly wrong data structures and single buffer pass). This is why it isn't used for highlighting at the moment.
I think the reason could be that the query patterns needed to support this require accessing buffer text a lot. Doing that on the whole buffer is slow, unless it's direct access without copying. tree-sitter-hl suffered from the same issue, so we use range restriction query execution (ts_query_cursor_set_byte_range) for highlighting. A similar approach could probably be used for the local pass. (It doesn't work well with sibling patterns, but these are probably uncommon for querying locals.)
Our predicates are custom Lua functions so they can use data from a cached local passes.
How are text edits handled? Are subsequent query executions (after the first full-buffer execution) range-restricted?
Side note: I think eventually it's more desirable for tree-sitter to support some sort of "reactive query execution", maybe by somehow fusing the parser automata and the query automata together, so that incremental parses emit a streams of matches/captures and their retractions. (I don't know if that makes sense. I don't have deep enough knowledge in this area.)
Personally I'm looking at the brute-forcing approach of adding direct no-copy text access to Emacs dynamic modules, which hopefully can eliminate the need for these "clever" optimizations.
Any updates on this issue? I'm working on implementing indentation along the lines of Helix Editor's indent queries and while their custom predicates aren't widely used in the indent queries they actually have in-repo, they are essential in some cases.
