Nested multi-line comments not highlighted correctly

[ahelwer]

By "correctly" here I mean "as in the toolbox". Here's the code:

---- MODULE test ----
(*this
is
a
(*
nested
*)
multi
line
comment
*)
====

Here's how this is highlighted in the vs code extension vs. the toolbox: I actually don't think this is theoretically parse-able with regular expressions lol. SANY's debug syntax tree output confirms that it's doing some nest counting:

N_Module          #heirs: 4       kind:   382
| N_BeginModule   #heirs: 3       kind:   333
| | ---- MODULE   #heirs: 0       kind:   3
| | Test          #heirs: 0       kind:   231
| | ----          #heirs: 0       kind:   35
| N_Extends       #heirs: 0       kind:   350
| N_Body          #heirs: 0       kind:   334
| N_EndModule     #heirs: 1       kind:   345
| | ====          #heirs: 0       kind:   36
 preComment: 0 (*this
is
a
(*
             1
nested
*)
             2
multi
line
comment
*)

Anyway this is clearly a minor issue but I thought I'd document it. @hwayne pointed out this feature is quite nice when you want to comment out a large section of the file that already itself has comments.

I'm also tackling this problem in my tree-sitter grammar, see https://github.com/tlaplus-community/tree-sitter-tlaplus/issues/15. It's plausible that highlighting from the tree-sitter grammar could be integrated into the vs code extension once the grammar is completed, if that proves valuable.

[alygin]

@ahelwer, you're right, TextMate grammar is not powerful enough to highlight such things. It's only possible with a full-featured parser.

Maybe your tree-sitter grammar will make it to a TLA+ LSP sometime :) That would make highlighting much more accurate, let alone other cool features.

[ahelwer]

@alygin did you fork https://github.com/agentultra/TLAGrammar (same as what github uses) or write your own textmate grammar for highlighting? I don't know whether the github grammar is really being maintained. You could consider replacing it with yours. See Changing the source of a syntax highlighting grammar.

[ahelwer]

@alygin check it out! Progress https://tlaplus-community.github.io/tree-sitter-tlaplus/