gatsby-plugin-mdx: TOC broken with custom IDs

[LukeShu]

Preliminary Checks

• [X] This issue is not a duplicate. Before opening a new issue, please search existing issues: https://github.com/gatsbyjs/gatsby/issues
• [X] This issue is not a question, feature request, RFC, or anything other than a bug report directly related to Gatsby. Please post those things in GitHub Discussions: https://github.com/gatsbyjs/gatsby/discussions

Description

(This is in fact a duplicate of https://github.com/gatsbyjs/gatsby/issues/30361 which was closed by stalebot.)

I use gatsby-remark-autolink-headers with enableCustomId: true to create headers with custom link targets:

# Header {#header}
## SubHeader {#subheader}

The result is that the html code is produced as it should be: {#header} is removed, the id is applied to the <h1>:

<h1 id="header" style="position: relative;">
    <a href="#header" aria-label="header permalink" class="anchor before"></a>
    Header
</h1>

BUT: The TOC is not using new IDs and headers:

query MyQuery {
  allMdx {
    nodes {
      tableOfContents
    }
  }
}

{
  "data": {
    "allMdx": {
      "nodes": [
        {
          "tableOfContents": {
            "items": [
              {
                "url": "#header-header",                 <----- should be "#header"
                "title": "Header {#header}",             <----- should be "Header"
                "items": [
                  {
                    "url": "#subheader-subheader",       <----- should be "#subheader"
                    "title": "SubHeader {#subheader}"    <----- should be "SubHeader"
                  }
                ]
              }
            ]
          }
        }
      ]
    }
  },
  "extensions": {}
}

Reproduction Link

https://github.com/datawire/gatsby-mdx-autoheader-bug

Steps to Reproduce

1. Install gatsby-plugin-mdx and gatsby-remark-autolink-headers
2. Configure gatsby-remark-autolink-headers: enableCustomId: true
3. Write some markdown with a {#custom-id} suffix
4. Run npx gatsby develop and use /__graphql to query

   query MyQuery {
     allMdx {
       nodes {
         tableOfContents
       }
     }
   }
   

The first 3 steps can be done by cloning the reproducer URL.

Expected Result

I expected the query to return

{
  "data": {
    "allMdx": {
      "nodes": [
        {
          "tableOfContents": {
            "items": [
              {
                "url": "#header",
                "title": "Header",
                "items": [
                  {
                    "url": "#subheader",
                    "title": "SubHeader"
                  }
                ]
              }
            ]
          }
        }
      ]
    }
  },
  "extensions": {}
}

Actual Result

The query actually returned

{
  "data": {
    "allMdx": {
      "nodes": [
        {
          "tableOfContents": {
            "items": [
              {
                "url": "#header-header",                 <----- should be "#header"
                "title": "Header {#header}",             <----- should be "Header"
                "items": [
                  {
                    "url": "#subheader-subheader",       <----- should be "#subheader"
                    "title": "SubHeader {#subheader}"    <----- should be "SubHeader"
                  }
                ]
              }
            ]
          }
        }
      ]
    }
  },
  "extensions": {}
}

Environment

System:
    OS: Linux 5.13 Parabola
    CPU: (4) x64 Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz
    Shell: 5.1.8 - /bin/bash
  Binaries:
    Node: 16.7.0 - /usr/bin/node
    Yarn: 1.22.11 - /usr/bin/yarn
    npm: 7.21.0 - /usr/bin/npm
  Languages:
    Python: 3.9.6 - /usr/bin/python
  Browsers:
    Firefox: 91.0.2
  npmPackages:
    gatsby: ^3.13.0 => 3.13.0
    gatsby-plugin-mdx: ^2.13.0 => 2.13.0
    gatsby-remark-autolink-headers: ^4.10.0 => 4.10.0
    gatsby-source-filesystem: ^3.13.0 => 3.13.0

Config Flags

No response

[LukeShu]

There seem to be 2 pieces to this. Background: The TOC is generated from the mdast returned from genMDX in utils/gen-mdx.js. That genMDX function returns both the MDX mdast and a processed stringified body (as well as a few other things). For whatever reason, it actually instantiates 2 separate MDX compilers, one to spit out the AST and one to spit out the stringified output.

So, the 2 pieces of the issue are:

1. The MDX compiler instance used to spit out the AST doesn't have gatsbyRemarkPlugins applied to it, just the vanilla remarkPlugins.
2. The mdast returned is the raw parse (compiler.parse(file)), before any transformer plugins have had a chance to run on it (compiler.run(compiler.parse(file))).

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here. If we missed this issue or if you want to keep it open, please reply here. As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[LukeShu]

not stale

[david-wb]

I see this too, would love to see a fix.

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here. If we missed this issue or if you want to keep it open, please reply here. As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[LukeShu]

not stale

[iChenLei]

Render Pipeline

.mdx -> gatsby-plugin-mdx -> gatsbyRemarkPlugins[gatsby-remark-autolink-headers] .html <- xxx <- gatsby-plugin-mdx <--------------------------------------------------|

So it works as expected.

GraphQL Query Pipeline

GraphiQL -> gatsby-plugin-mdx GraphiQL <- --------------------|

No special process for gatsby-remark-autolink-headers's enableCustomId feature

https://github.com/gatsbyjs/gatsby/blob/83396a5a91eb2f329d4b5e49ef5544106d0e380f/packages/gatsby-plugin-mdx/gatsby/create-schema-customization.js#L251-L265

Any idea for this bugfix ? cc @wardpeet

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here. If we missed this issue or if you want to keep it open, please reply here. As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[iChenLei]

Not stale

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here. If we missed this issue or if you want to keep it open, please reply here. As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[LukeShu]

not stale

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here. If we missed this issue or if you want to keep it open, please reply here. As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[LukeShu]

not stale

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here. If we missed this issue or if you want to keep it open, please reply here. As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[LukeShu]

not stale

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here. If we missed this issue or if you want to keep it open, please reply here. As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[zachmerrill]

not stale. I'm also interested in seeing this fixed

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here. If we missed this issue or if you want to keep it open, please reply here. As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[vrabe]

Not stale!

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here. If we missed this issue or if you want to keep it open, please reply here. As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[zachmerrill]

Another 20 days, another not stale

[vrabe]

@axe312ger Will the new gatsby-plugin-mdx fix this issue?

[axe312ger]

@vrabe I don't know, but the related dependencies got updated.

You can give it a try! :)

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here. If we missed this issue or if you want to keep it open, please reply here. As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[vrabe]

Not stale!

[LekoArts]

Hi!

We merged https://github.com/gatsbyjs/gatsby/pull/35650 which means that a new major version of gatsby-plugin-mdx will be released on August 16. You can already try it out by installing gatsby-plugin-mdx@next & gatsby@next.

I know that https://github.com/gatsbyjs/gatsby/issues/36265 was created and since then I looked into it. You'll find https://github.com/rehypejs/rehype-slug/issues/10 and subsequently the package https://github.com/unicorn-utterances/rehype-slug-custom-id.

You'll need to use https://github.com/unicorn-utterances/rehype-slug-custom-id to be able to use this feature with MDX v2. A note about this was added to the README of gatsby-remark-autolink-headers and to the migration guide for the MDX plugin.