CommonMark compatibility (Tracker bug)

[susnux]

Background

Currently there are "commonmark" tests, but they only test markdown-it to output correct HTML for markdown (which is already tested upstream), but they are not testing our pipeline to keep the markdown valid. Meaning validating that the input and output are at least equivalent (both result in same HTML, as the mapping is surjective but not injective).

md ==> markdown-it ==> HTML ==> TipTap ==> prosemirror-markdown ==> md

Problems

I created a branch with adjusted tests, currently there are a lot of failures, which need to be triaged (real bugs, whitespace problems, test issues...), so this is a list of all bugs blocking full CommonMark support:

• [x] :lady_beetle: https://github.com/nextcloud/text/issues/2699
• [x] :lady_beetle: Multiple (allowed) (white)spaces are removed, #2720
• [ ] :up: :timer_clock: https://github.com/ProseMirror/prosemirror-markdown/issues/78 (waiting for release)
• [ ] :up: https://github.com/ProseMirror/prosemirror-markdown/issues/77
• [ ] :up: https://github.com/nextcloud/text/issues/969 (Waiting for release of https://github.com/ProseMirror/prosemirror-markdown/issues/76 )
• [x] :lady_beetle: https://github.com/nextcloud/text/issues/2703
• [ ] :heavy_plus_sign: https://github.com/nextcloud/text/issues/2170 (HTML is required for full CommonMark support)

Opinions

There are already issues about breaking the formatting of markdown files, like https://github.com/nextcloud/text/issues/593 and https://github.com/nextcloud/text/issues/2577 but from my point of view it is hard to provide that kind of compatibility, at least the way text is build. But I do not think it is needed, if I need a markdown editor with that behavior, I would recommend to use https://github.com/icewind1991/files_markdown Of cause you as the developers of this app should decide the scope of application, but for me text is a collaborative text editor.