Semantic coloring

[dvc94ch]

A proposal for LSP is in the works. theia-ide/theia#1845 https://github.com/Microsoft/language-server-protocol/pull/124 Here someone implemented the hack I initially proposed https://github.com/rust-lang-nursery/rls/issues/85

[Eugleo]

@jeapostrophe I think the LSP has already included semantic coloring, see link to VSCode docs. It would be cool to have something like this, regexes aren't very good for things like quasiquotes or macros.

EDIT: Scratch that, it's still in the works. Maybe it's indeed better to wait till the proposal is merged.

[mlavrent]

Once that's added (if it is), here's some resources on how to get the colorer from the #lang. You probably want to get the language info using:

(define lang-info (read-language (open-input-string (send doc-text get-text))))
(lang-info 'color-lexer #f)

The relevant docs can be found here.

Based on this comment it seems the current 3.16 version of the LSP supports semantic coloring. I'm going to take a look into doing this and see if we can add this, since I think it's necessary to do coloring on the server side to support custom #langs.

[6cdh]

I'm interested in implement semantic tokens recently.

For convenience, all tokens in specification.

The meaning of tokens are be found at vscode's doc

These are my rough idea:

Design

Supported Tokens

The following tokens will be supported

token	information source
variable	read/syntax (symbol) and syncheck:add-text-type
comment	scan visible characters that can't read by read/syntax (comment)
string	read/syntax (string)
number	read/syntax (number)
regexp	read/syntax (regexp or pregexp)
??	keyword #:keyword
parameter	manually walk syntax object
function	manually walk syntax object
struct	manually walk syntax object
class	manually walk syntax object

EDITED: use read/syntax instead of lexer

modifiers:

modifiers	information source
definition	drracket syncheck:add-definition-target

Requests

method	support
full	Yes
full/delta	Yes
range	Yes
refresh	Yes

EDITED: should support refresh response

Prototype

I wrote a simple prototype: snippet

Note in this prototype, I used read/syntax not lexer, it don't recognize comments.

usage:

racket pt.rkt filename

Then it will print the tokens and their information in the order that they appear in the file.