MDX loader: linkify should process the md AST instead of the md string

[slorber]

Have you read the Contributing Guidelines on issues?

• [X] I have read the Contributing Guidelines on issues.

Motivation

Our current MDX loader code looks like this:

export default function markdownLoader(
  this: LoaderContext<DocsMarkdownOption>,
  source: string,
): void {
  const fileString = source;
  const callback = this.async();
  const options = this.getOptions();
  return callback(null, linkify(fileString, this.resourcePath, options));
}

The linkify process that turns [link](./xyz.md) to [link](/baseUrl/xyz-pathname) is based on string manipulations and regexes.

As a result, linkify has many edge case bugs related to the unability to properly resolve markdown relative file path links such as [](./relative-path.md)

We should instead do this with a Remark plugin after MDX has parsed the string.

See also:

• https://github.com/facebook/docusaurus/pull/8927#issuecomment-1525937829
• https://github.com/facebook/docusaurus/discussions/9041#discussioncomment-6113436
• https://github.com/facebook/docusaurus/pull/9046
• https://github.com/facebook/docusaurus/issues/9720#issuecomment-1883539189

Self-service

• [X] I'd be willing to do some initial work on this proposal myself.

[Josh-Cena]

See previous attempt: https://github.com/facebook/docusaurus/pull/6261

[slorber]

Another case it would fix is a bug when using relative md links on markdown images:

[text](./installation.mdx)

[![alt](/img/slash-introducing.svg)](./installation.mdx)

Only the first link resolves.

But reported here: https://github.com/facebook/docusaurus/issues/9720#issuecomment-1883539189, that should also be solved by this proposed solution.

[slorber]

Another case reported where markdown links resolution fails due to usage of inline code blocks with triple backticks (TIL this exists and is supported by MDX 😅). https://github.com/facebook/docusaurus/issues/9876

```something```

[test](test.md)

[slorber]

Another use-case reported is when the link is multi-line:

<!-- This works -->
A [link to another page](other-page.md)

<!-- This doesn't work -->
A [link to
another page](other-page.md)

https://github.com/facebook/docusaurus/issues/9636 https://github.com/facebook/docusaurus/discussions/10067

[slorber]

Another use-case reported is when the link contains certain combinations of symbols:

[a](./migration/migrate-6.md#hooks) ([b](./migration/migrate-6.md#hooks))

[`Type1`](some_doc.md#type1)\<[`Type2`](some_doc.md#type2)\>

https://github.com/facebook/docusaurus/issues/9518 https://github.com/facebook/docusaurus/issues/9553 https://github.com/facebook/docusaurus/issues/10072

[pellyadolfo]

another case that breaks for me is a link in docusaurus.config.js in the navbar

{ to: '/docs/mypage', label: 'MyPage', position: 'right' }

fixed with

{ type: 'doc', docId: 'mypage', label: 'MyPage', position: 'right' }

[slorber]

@pellyadolfo that's unrelated, here we are only discussing markdown links ending with a .md extension. If you found a bug, please open a separate bug report and provide a docusaurus.new runnable repro