docusaurus
docusaurus copied to clipboard
facebook
Migrate to MDX 2.0
🚀 Feature
We should migrate to MDX 2.0
Issue to complete for motivations
Related to https://github.com/facebook/docusaurus/issues/3668
Blocking issues
- We need ESM to support MDX OOTB: https://mdxjs.com/migrating/v2/#esm
- #5816
mdx 2.0 introduced breaking change, comments <!-- --> are not longer allowed in code, and thus mdxjs can't be used for simple markdown files
There are also disagreements in the MDX core contributors:
- https://github.com/mdx-js/mdx/pull/1425
- https://github.com/mdx-js/mdx/issues/1385
There's also a "fork" xdm project from Titus himself.
Let's not rush on this migration for now and see how things evolve.
I strongly encourage using xdm, it's a simpler, faster and more stable pipeline, it is always in sync with the remark/rehype ecosystem and its development is moving quickly
Ok, @lex111, probably then add a warning block to the top of https://docusaurus.io/docs/markdown-features/react page, saying that current Docusaurus uses MDX v1? Cause I believe the current docs do not mention it, and the linked MDX site pages do not make it clear that there are two version with some features not supported in v1.
@birdofpreyru alright, although it seems self-evident for now, because MDX v2 is not officially released. However, the migration to MDX v2 will probably not be that easy, so it makes sense to do it (see #5575). FYI, earlier in header of MDX website listed its latest version, but currently they have removed that info.
MDX 2 RC 👍 https://v2.mdxjs.com/blog/v2/
And there's a migration guide: https://v2.mdxjs.com/migrating/v2/
Would really love to get to v2 to have things like: https://github.com/mdx-js/mdx/issues/821
This would be a tremendous help for porting in plain markdown documents that have <pre /> or inline html fragments.
Nextra just upgraded in their 2.0 beta.
Seems like it was not too hard to do this upgrade:
- https://github.com/shuding/nextra/pull/184
- https://github.com/shuding/nextra/pull/251
(except maybe the ESM support 😅 )
The blocking issue is #6520, right?
I'm wondering if migrating to ESM is really needed for MDX v2.
@pomber No, it's not strictly needed. If we can't get the ESM migration done before v3, we will just re-distribute MDX as a CJS bundle. However, we are releasing v2 very soon and it's no longer permitting us to do these big architectural things, so everything has been delayed to v3.
@Josh-Cena does this mean we should make a v3 feature list? mdx@2 would be great to land in 2, but totally understand it being delayed.
I created a theme to add mdx v2 support to Docusaurus until it's officially supported: https://github.com/pomber/docusaurus-mdx-2
Great 👍 thanks for that
So if I understand correctly we mostly need to figure out how to upgrade admonitions
MDX v2's comment syntax change is pretty annoying. I think we should definitely support this pattern, given that I have a mild objection to #3018. I believe we should just implement a compatibility layer to ensure things that can be correctly parsed are being correctly parsed.
So if I understand correctly we mostly need to figure out how to upgrade admonitions
remark-admonitions can be replaced with @theme/Admonition + remark-directive. I tried and couldn't make it work, but it should be doable.
If you are asking how would a migration to mdx v2 look like on the docusaurus repo, I think most of the changes will be on the docusaurus-mdx-loader package, to handle the new AST for the jsx nodes and the export nodes. You'll also need to add the remark-gfm to support some syntax that was removed from v2.
Outside of docusaurus-mdx-loader, I only encountered issues with the MDXPre and MDXCode components.
FWIW: I tried to start from @pomber's theme and integrate remark-directive. Most updates were fairly simple, following this guide https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c.
I ran into a wall when I declared the mdx-loader as a type "module" to be interpreted by webpack. While the loader-runner is already able to load loaders via import(), webpack itself does not seem ready for it: https://github.com/webpack/loader-runner/commit/4f91458eca03c7350971e0c555521f4e6c13c6e0#r40502100
Not sure what you mean, but here's a WIP PR I'm working on to upgrade to MDX 2: https://github.com/facebook/docusaurus/pull/8288
So far it builds and display a site with some broken features and workarounds I had to setup. Will try to get our own full site building on MDX 2 asap.
Using ESM modules with dynamic imports doesn't seem to be the main problem of this migration, but rather some MDX changes that I'll document.
@slorber I haven't checked the code, but what's your decision on HTML comments? I'd rather build a preprocessing layer so those comments continue to be allowed.
@slorber I haven't checked the code, but what's your decision on HTML comments? I'd rather build a preprocessing layer so those comments continue to be allowed.
😄 This is the processing layer I'm thinking of: str.replaceAll('<!--', '\\<!--')
Note that escaping HTML comments this way doesn't seem required if you use format: 'md'. I'll probably add a way to move from "mdx" to "md" format through frontMatter, and maybe use file extension-based format later. I guess it could help some md sites migrate
FWIW: I tried to start from @pomber's theme and integrate remark-directive. Most updates were fairly simple, following this guide https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c. I ran into a wall when I declared the mdx-loader as a type "module" to be interpreted by webpack. While the loader-runner is already able to load loaders via
import(), webpack itself does not seem ready for it: webpack/loader-runner@4f91458#r40502100
@scubamaggo did you ever get pomber's theme working with remark-directive? I am working with the same theme; my Docusaurus site compiles fine, but remark-directive doesn't seem to be doing anything
😄 This is the processing layer I'm thinking of:
str.replaceAll('<!--', '\\<!--')
I know I'm late to the party, @slorber, but are we worried about code samples here? I could see that replacement potentially breaking an HTML snippet in a code fence.
@dprothero We're probably not going to do that kind of naïve replacement. It would be contextual, so any code would be skipped.
😄 This is the processing layer I'm thinking of:
str.replaceAll('<!--', '\\<!--')I know I'm late to the party, @slorber, but are we worried about code samples here? I could see that replacement potentially breaking an HTML snippet in a code fence.
The way I see it is that I'll add a few env variables to turn compatibility options on. To make the upgrade simpler you'll be able to turn all the env variables on, and once you have fully migrated to MDX 2 you could remove the env variables.
Will probably add an option to turn this on: https://github.com/leebyron/remark-comment (or maybe it will simply work if you pass it as a remark plugin, didn't try that yet)
I also think I'll allow you to provide your own preprocessor so that you can handle the most complex cases yourself.
- The MDX 2 PR has been merged
- I documented many breaking changes and provided some compatibility options (see PR)
- A canary release is now available (>= 0.0.0-5556)
Please test it and give me some feedback here! 🙏
Am I to understand this isn't going to land in any of the 2.x releases and is being saved for 3.0.0?
Am I to understand this isn't going to land in any of the
2.xreleases and is being saved for3.0.0?
yes. this is for the 3.0 release
