Josh Goebel

@UziTech Is there any option to make the parser/lexer spit out more context, such as the position (index) of tokens in the original source string?

> some text is converted right away Are there any options to disable/configure that? (or is it required for the lexer?)  (I...

Right now though I'm unsure which `raw` components we'd even want to add up though... because it's not as simple as "ignore the parent, always add up the children" or...

> Once marked parses the markdown into tokens it should be pretty easy to parse the raw of each token with regexps to change it into highlight.js tokens. Oh yes,...

This is our internal "emitter" API that builds the token tree on our side: https://github.com/highlightjs/highlight.js/blob/main/src/lib/token_tree.js#L104 I had originally hoped we could just walk the lexer tree and then make calls...

> The renderer doesn't get the information about parents. Yeah, I'm thinking just using the lex output directly might be simpler since it has all that information... for example for...

> I know the goal is to get it into tokens but I don't think it would be too difficult to figure out how to change this to accomplish that....

Also running into this exact problem and thinking perhaps this should be configurable?

> Please make sure you run npm test and fix up the failing unit tests. We want to be sure that everything is passing before merging in. If there is...

``` single special character x 5,221,477 ops/sec ±1.84% (178 runs sampled) VS single special character x 4,523,898 ops/sec ±1.03% (186 runs sampled) ``` This test specifically makes me feel the...