haskell-language-server icon indicating copy to clipboard operation
haskell-language-server copied to clipboard

[feat] syntactic semantic tokens

Open MangoIV opened this issue 7 months ago • 5 comments

Use the GHC AST and lsp semantic tokens to convince the language server to give highlighting even without any editor highlighting plugins.

Fixes #4501

  • [x] check that this works on all ghc versions
  • [ ] check which AST nodes I might have overlooked that are important. (add a tests that checks that all source ranges are covered (except space)

MangoIV avatar Jul 22 '25 18:07 MangoIV

Some keypoints beyond the tokens retrieval:

  1. UTF8 vs UTF16 differences for the token range between haskell and LSP spec. see focusTokenAt.
  2. Consider position mappings so we have correct highlighting even the file could not even parse, see toCurrentRange.
  3. we need to consider delta between each result, see semanticTokensFullDelta. This is to minimize the bandwith between LSP server and clients.

Post it here as a reminder.

soulomoon avatar Jul 22 '25 18:07 soulomoon

Thank you. I will go through all of these.

MangoIV avatar Jul 22 '25 18:07 MangoIV

@soulomoon I'm currently struggling a bit with the tests - the ASTs are different enough that making the syntax highlighting consistent between ghc versions is probably a big pain. Should I introduce more goldens for all supported ghc versions?

MangoIV avatar Jul 26 '25 10:07 MangoIV

also stylish-haskell's parser seems to be confused about the CPP :/

MangoIV avatar Jul 26 '25 14:07 MangoIV

@soulomoon I'm currently struggling a bit with the tests - the ASTs are different enough that making the syntax highlighting consistent between ghc versions is probably a big pain. Should I introduce more goldens for all supported ghc versions?

The only different should be that module name in the head is included since 9.10?

soulomoon avatar Jul 27 '25 07:07 soulomoon