marked icon indicating copy to clipboard operation
marked copied to clipboard

[Suggestion] Provide Escaping Options in Lexer

Open Bistard opened this issue 4 months ago • 4 comments

Describe the feature Is it possible to provide an option in the lexer, like enableEscaping: boolean; that controls the lexer should turn any characters (e.g. &, <, > etc) into escaping characters (e.g. &amp;, &lt;, &gt; etc).

const tokens = marked.lexer(text, {
    enableEscaping: false,
});

Why is this feature necessary?

  1. For example, when I get a token like {type: 'codespan', raw: '<h1>', text: '&lt;h1&gt;'}. I wish to display the text normally. I have to use extra calculations like regular expressions to turn &lt;h1&gt; back to <h1>.
  2. But this is essentially just undoing the process during the first time (default tokenizer in marked) which caused meaningless extra calculations.
  3. So my suggestion is to provide an option that prevents the escaping process in the first place inside the default lexer.

Describe alternatives you've considered I saw the issue answered in 2022 said I have to override the tokenizer to achieve this.

Bistard avatar Oct 14 '24 18:10 Bistard