[WIP] Tree-sitter for LaTeX

[Aerijo]

Things to do:

• [ ] Refine the grammar itself to support everything the TextMate one does (where possible; injections may be out of our control for now)
• [ ] Fix up all the scopes to match the existing ones as closely as reasonable (or use this as a chance to start fresh?)
• [ ] Set up CI to prebuild the parser with prebuild and upload to GitHub on each release. Needed for users without access to build tools.
• [ ] Lock down the parser version when the above is settled
• [ ] Remove private: true from package.json and add said dependencies when ready
• [ ] Merge the TS biber support while we're at it

Please comment with any other concerns that should be addressed before merging

[yitzchak]

I have some injection code at main.js

[Aerijo]

@yitzchak AFAIK injections only work TS to TS, so each language we currently have explicit support for would need their own TS parser, and that could take a while.

Still, that looks like a good thing to add. I'll merge a PR you submit

[PgLoLo]

Is there any progress on transferring to the new tree-sitter? It is two years now, and it would be great to get rid of the highlighting line length limit:

[Aerijo]

@PgLoLo No, nothing yet. What you are running into can be solved with the grammar-token-limit package.

The biggest problem I had was that the parser was too brittle, basically failing to parse anything if there was any invalid input, so as you type the text would just be flashing as it parses and errors as you go. So this, the existing grammar being mostly good enough, and other priorities meant no work on the TS grammar.