chroma
chroma copied to clipboard
alecthomas
Add lexer for Typst
Is there an existing issue for this?
- [X] I have searched the existing issues
What is the missing lexer?
Typst is a new markup language and I've noticed that chroma doesn't support it.
The syntax documentation can be found here: https://typst.app/docs/reference/syntax/
Links to existing syntax definitions
Github Linguist uses the following: https://github.com/michidk/typst-grammar/blob/master/grammars/tmlanguage.json
I converted the new pygments lexer (credits go to @jvoisin) to the Chroma XML format using python3 _tools/pygments2chroma_xml.py pygments.lexers.typst.TypstLexer > lexers/embedded/typst.xml. See https://github.com/miliog/chroma/commit/d05a092a4851f7a040da5c9cafc1b2dc37851144.
Only the line
<rule pattern="(_|^|+|-|/|*|->|<-|!=|==)\b"><token type="Operator"/></rule>
was broken and an error was shown while running the test. To fix it, I escaped the special characters +, -, /, and * by preceding them with a backslash (\):
<rule pattern="(_|\^|\+|\-|\/|\*|>|<|!=|==)\b"><token type="Operator"/></rule>
However, there are still a few problems remaining, as shown below:
My example works just fine in pygments according to https://pygments.org/demo/:
FYI I have looked at this problem and put some incremental work into the typst lexer at https://github.com/pygments/pygments It will probably be the easiest if we first develop there and then simply copy the lexer over just as you did before. Happy for feedback on my recent PR over there: https://github.com/pygments/pygments/pull/2656
Combining the latest state https://github.com/pygments/pygments/pull/2724 (with Bin instead of Binary) and the conversion fix proposed in #982, the result look good:
If anyone wants to play with it, the converted xml is attached (as TXT to please github...) typst.xml.txt
