Add region-restricted parsing and highlighting

[ubolonton]

This will allow jupyter-repl to integrate with tree-sitter and tree-sitter-hl. See #78 for more context.

I initially tried to make tree-sitter parse only the narrowed region. However, since tree-sitter's incremental parsing requires precisely tracking all changes to the source code, there are some issues with that approach:

• Emacs's change tracking mechanism are the hooks before-change-functions and after-change-functions, which can be run while narrowing is enabled for a different region than the intended region.
• Tree-sitter needs to be able to track changes of narrowed regions, since they are code changes. This cannot be done automatically by tree-sitter itself.

The design of this new functionality is instead like followed:

• tree-sitter still tracks all changes in the buffer.
• When tree-sitter parses, it asks the major mode for the region to parse, by calling the function tree-sitter-get-parse-regions.
• When tree-sitter-hl highlights the buffer, it does so only for the parsed region, while using the old region-fontification function for other regions.
• When the major mode wants to change the region to be parsed (e.g. moving to the next input cell):
  • It calls tree-sitter-hl-dry-up-region to "persist" the highlighted region (now-frozen input cell).
  • It calls tree-sitter-pause, and then tree-sitter-resume afterwards, to start a new, non-incremental parse.

Tasks:

• [x] Parsing
• [x] Highlighting
• [x] jupyter-repl integration
• [ ] inferior-python-mode integration
• [ ] Tests

[orzechowskid]

looking forward to this PR landing in master. I was just attempting to add tree-sitter support to a CSS-in-JS mode I've been working on, and I think this will be of immense use once it's ready.

thanks!

[ubolonton]

looking forward to this PR landing in master. I was just attempting to add tree-sitter support to a CSS-in-JS mode I've been working on, and I think this will be of immense use once it's ready.

The design in this PR is intended for one-at-a-time parsing of code blocks, so I'm not sure it would help a CSS-in-JS mode (a multi-language use case). How would you use it?