John MacFarlane

Benchmarks for different extensions: extension | mean ------------|---------------------- -xautolinks | 310.8 ms (309.3 ms .. 311.3 ms) -xpipe_tables | 295.2 ms (293.2 ms .. 296.6 ms) -xstrikethrough | 267.9 ms...

One idea to explore: use `ShortText` from `text-short` package instead of `Text` in `Tok`. The public API could still use `Text`. This should reduce the memory used by the tokens.

Are you enabling the pipe tables extension?

The issue can be reproduced with the command line tool thus: ``` % commonmark -xmath -xpipe_tables Examples: * $|x| = 2$ * other example ^D Examples: * \(|x| = 2\)...

Even simpler example: ``` % commonmark -xpipe_tables # Hi | there | ok ^D # Hi | there | ok ```

Another option would be to do some kind of lookahead for the separator lines. Current architecture, though, is for line-by-line parsing.

Simple workaround now would be to make sure that the pipe_tables syntax is AFTER the list syntax. ``` defaultSyntaxSpec pipeTableSpec ```

One idea would be to add an Options data structure ``` data Options m = Options{ optIncludeSourceRanges :: Bool, optWarn :: Warning -> m (), optAddToSourceMap :: Text -> SourceRange...

We could use a reader monad transformer to make these available in block and inline parsers.

The option is there already (as well as the ability to create a source map). This was just an idea about an alternative way to put it there.