reasonml-editor
Syntax highlight broken in ast_402.ml
The [% in comment is mis-tokenized as String and make all the code below tokenized as Comment type.
I am not sure is this tokenization defined in ocaml.json (which is minified?) or ocaml.ts. I am very new to VSC extension and textmate grammer. With more hints I might be able to fix this 🤔
First, thanks for the report.
The issue of "mistokenization" has come up before. Originally it was a design choice to use the "wrong" scopes in many cases because the major themes (having mostly been designed for languages like javascript, ruby, python, etc.) do not provide enough highlighting to account for all of the different syntactic categories of OCaml or Reason.
However, this has been a point of controversy for many users. I understand why (they want the highlighting to look similar to the other languages they use) although I feel differently about it.
In any case, the current plan, once I have time to work on it again (maybe early August), is to greatly simplify the highlighting and just do what people expect and not try to provide the most syntactically informative highlighting.
Although I really do appreciate the effort, I would suggest not spending too much time trying to decipher the existing highlighting. Once the rewrite/simplification happens, I will be very happy to accept PRs though.
I am not sure is this tokenization defined in ocaml.json (which is minified?) or ocaml.ts. I am very new to VSC extension and textmate grammer. With more hints I might be able to fix this 🤔
Just for clarity, the way the tokenization works here is I create a little DSL for regular expressions and textmate grammars and I use it to define an object in the ocaml.ts file which gets compiled (and minimized) to the ocaml.json.
This isn't the usual way of defining textmate grammars but it has certain advantages, for example the ability to re-use definitions. Some of this convenience would be possible if we were starting from a YAML file too, but this is even more flexible. (Originally I was using inheritance to describe a common subset of OCaml and Reason this way too).