DanielXMoore
LSP Semantic Token Provider + syntax highlighting
Trying to merge the TypeScript TextMate grammar and an updated CoffeeScript grammar seems like it would be an unspeakable nightmare. So it would be nice if the Civet parser / LSP could provide semantic tokens.
Essentially tokens that have {$loc} would be source tokens styled from the nearest ancestor that has some sort of {sematicToken} property. It might require passing a flag into the parser and skipping normal AST processing instead constructing a stream of semantic tokens.
https://code.visualstudio.com/api/references/vscode-api#DocumentSemanticTokensProvider (looks like it is very similar to how source maps work)
As discussed in Discord: We should aim to solve syntax highlighting at the same time. One approach is for each leaf token (with $loc) to have a group that defines what type of token it is (e.g. keyword), where group could be from this TextMate list. Input tokens that don't appear in the output could still received a group via {$loc, group: <something>, token: ''}. A highlighting pass would take the ASP, visit all tokens with group, highlight those accordingly, maybe merge together adjacent equal groups, and assume everything else isn't highlighted in any way.
Semantic Tokens might need more of a hierarchy (e.g. for foldable sections) so the semanticToken parent structure might be more useful there. Or some combination thereof.
