yarn build error with "Docusaurus found broken links" after set baseUrl to "/docs/"

[xiaosongxiaosong]

Have you read the Contributing Guidelines on issues?

• [X] I have read the Contributing Guidelines on issues.

Prerequisites

• [X] I'm using the latest version of Docusaurus.
• [X] I have tried the npm run clear or yarn clear command.
• [X] I have tried rm -rf node_modules yarn.lock package-lock.json and re-installing packages.
• [X] I have tried creating a repro with https://new.docusaurus.io.
• [X] I have read the console error message carefully (if applicable).

Description

yarn build error with "Docusaurus found broken links" after set baseUrl to "/docs/"

Steps to reproduce

1. init

npx create-docusaurus@latest docs classic

2. build

cd docs 		
yarn build
...
[SUCCESS] Generated static files in build.
[INFO] Use `npm run serve` command to test your build locally.
Done in 69.04s.
PS D:\Document\github\docs>

3. change baseUrl from '/' to '/docs/'

# docusaurus.config.js
-   baseUrl: '/',
+  baseUrl: '/docs/',

4. build

yarn build
...
[ERROR] Unable to build website for locale en.
[ERROR] Error: Docusaurus found broken links!

Please check the pages of your site in the list below, and make sure you don't reference any path that does not exist.
Note: it's possible to ignore broken links with the 'onBrokenLinks' Docusaurus configuration, and let the build pass.

It looks like some of the broken links we found appear in many pages of your site.
Maybe those broken links appear on all pages through your site layout?
We recommend that you check your theme configuration for such links (particularly, theme navbar and footer).
Frequent broken links are linking to:
- /docs/intro


Exhaustive list of all broken links found:

- On source page path = /docs/404.html:
   -> linking to /docs/intro

5. check the links in files generated by build

<a class="button button--secondary button--lg" href="/docs/intro">Docusaurus Tutorial - 5min ⏱️</a>

Expected behavior

the links in files generated by build should be like this

<a class="button button--secondary button--lg" href="/docs/docs/intro">Docusaurus Tutorial - 5min ⏱️</a>

Actual behavior

link in document should be like '/docs/docs/intro', but actually is '/docs/intro'

Your environment

• Public source code: https://codesandbox.io/s/zealous-meadow-i63wc
• Public site URL: https://codesandbox.io/s/zealous-meadow-i63wc
• Docusaurus version used: 2.0.0-beta.14
• Environment name and version (e.g. Chrome 89, Node.js 16.4): Chrome 94, Node.js v14.18.1
• Operating system and version (e.g. Ubuntu 20.04.2 LTS): win10x64

Reproducible demo

No response

Self-service

• [ ] I'd be willing to fix this bug myself.

[Josh-Cena]

Interesting, this is because we don't add baseUrl to links that already seem to have base URL:

https://github.com/facebook/docusaurus/blob/main/packages/docusaurus/src/client/exports/useBaseUrl.ts#L36-L37

I guess that's a bad assumption then😄

[slorber]

Yes, that's a bad assumpting that I made to fix issues we had in the past, where site owners had to use useBaseUrl everywhere including in regular <Link>

This was leading to verbosity (or requiring to update all links when updating the baseurl...).

This "security" can probably be removed now, but we need to ensure that it does not produce any weird side-effect.

Note React-Router has a special attribute to handle that: https://v5.reactrouter.com/web/api/BrowserRouter/basename-string but it has not been used historically.

As part of the v6 upgrade we should probably try to use it

[Josh-Cena]

@slorber I'm tempted to close this as wontfix, because useBaseUrl("/docs/intro") is pretty ambiguous. Sites with baseUrl: '/docs/' should still be able to write [Intro](/docs/intro) and be redirected to https://example.com/docs/intro, because that also conforms to URL semantics. Allowing writing [Intro](/intro), on the other hand, makes things more convenient. So in order to disambiguate in this very rare edge case, we should just recommend writing to="/docs/docs/intro", WDYT?

[slorber]

not sure to understand, but the idea for me is that we should allow users to change baseUrl without rewriting URLs

if we can build our own site with baseUrl = /docs/ today, I guess it's good enough, but I'm not sure it's the case

[Josh-Cena]

The problem is, what is the user's intent when they write a link like [intro](/docs/intro)? Today, if you write <a href="/docs/intro"> and you serve it on any location, you would be taken to https://example.com/docs/intro instead of https://example.com/docs/docs/intro because the browser doesn't have the concept of "base URL". URL semantics tells us to resolve absolute URLs from the domain name instead of from the base URL, and many users write Markdown with that mental model in mind. That's why a link like /docs/intro on a site with base URL /docs/ is at best ambiguous, because there's no way we can tell what the user's intent is. Of course I value the convenience of not having to hardcode base URLs in links, but in this case, there's no good heuristic except asking users to be explicit. If we consistently force-prepend base URL to links, that means we break URL semantics in Markdown, which is still confusing.

[slorber]

the browser doesn't have the concept of "base URL".

Actually there is a <base> tag but everytime I tried to use this I end up with weird browser issues: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/base

That's why a link like /docs/intro on a site with base URL /docs/ is at best ambiguous, because there's no way we can tell what the user's intent is.

I don't think it's ambiguous, the user intent should always be to link to <baseUrl>/docs/intro, no matter what <baseUrl> is. If the user has a different intuition, then we should explain better why his intuition is wrong

[Arhell]

Good afternoon! Updated to latest version 21 beta With yarn start, links behave correctly and work. But with yarn build it says that the links have problems

https://github.com/kubevela/kubevela.io/runs/7039961689?check_suite_focus=true

сс @Josh-Cena

[slorber]

@Arhell I believe this could be related to a subtle changes we made to how the doc slug is computed.

• workflow/workflow.md => URL is now /workflow instead of /workflow/workflow
• addon/addon.md => URL is now /workflow instead of /addon/addon

Broken links checking only happens on prod builds, and you have to carefully check the error message to debug such issues:

Exhaustive list of all broken links found:
- On source page path = /docs/next/end-user/traits/more:
   -> linking to ../../how-to/cli/addon/addon (resolved as: /docs/next/how-to/cli/addon/addon)
- On source page path = /docs/next/platform-engineers/policy/custom-policy:
   -> linking to ../workflow/workflow (resolved as: /docs/next/platform-engineers/workflow/workflow)
- On source page path = /docs/next/platform-engineers/traits/customize-trait:
   -> linking to ../workflow/workflow (resolved as: /docs/next/platform-engineers/workflow/workflow)

If the above message is not clear, can you help us improve it?

For example, this means that this doc page does not link properly to the addon page anymore:

• https://github.com/kubevela/kubevela.io/blob/main/docs/end-user/traits/more.md
• https://kubevela.io/docs/next/end-user/traits/more

The problematic markdown link is [install addons](../../how-to/cli/addon/addon), and should now be `install addons => note we recommend in our doc to link to docs with a relative file path (including .md extension): this makes the link insensitive to doc URL changes.

---

All this is not really related to the current issue. Can we move this to a separate discussion if you need more help?

[Arhell]

@Arhell I believe this could be related to a subtle changes we made to how the doc slug is computed.

• workflow/workflow.md => URL is now /workflow instead of /workflow/workflow
• addon/addon.md => URL is now /workflow instead of /addon/addon

Broken links checking only happens on prod builds, and you have to carefully check the error message to debug such issues:

Exhaustive list of all broken links found:
- On source page path = /docs/next/end-user/traits/more:
   -> linking to ../../how-to/cli/addon/addon (resolved as: /docs/next/how-to/cli/addon/addon)
- On source page path = /docs/next/platform-engineers/policy/custom-policy:
   -> linking to ../workflow/workflow (resolved as: /docs/next/platform-engineers/workflow/workflow)
- On source page path = /docs/next/platform-engineers/traits/customize-trait:
   -> linking to ../workflow/workflow (resolved as: /docs/next/platform-engineers/workflow/workflow)

If the above message is not clear, can you help us improve it?

For example, this means that this doc page does not link properly to the addon page anymore:

• https://github.com/kubevela/kubevela.io/blob/main/docs/end-user/traits/more.md
• https://kubevela.io/docs/next/end-user/traits/more

The problematic markdown link is [install addons](../../how-to/cli/addon/addon), and should now be `install addons => note we recommend in our doc to link to docs with a relative file path (including .md extension): this makes the link insensitive to doc URL changes.

All this is not really related to the current issue. Can we move this to a separate discussion if you need more help?

Thank you, with the help of tests (manual) I did it.

[MaximeKoitsalu]

Hi, I am having the same exact issue. What would be the recommended way forward if we want to deploy the Docusaurus build to /docs/ ?

[slorber]

@MaximeKoitsalu I guess it really depends on the amount of links that are currently reported, and if you want your site to remain "portable" to future baseurl changes (ie not needing to rewrite all the links)

It could be as simple as replacing a single link on your landing page.

<Link to={useBaseUrl("/docs/intro")}/> ➡️ <Link to="/docs/docs/intro"/>

For md files, using relative md file paths is the recommended way. Does it also trigger the problem?

In any case you should be able to prepend the /docs/ prefix yourself: Docusaurus won't remove it

Why do you want /docs/docs/ in the first place? you could as well change the second segment of the URL:

• Site with baseUrl: "docs"
• Docs plugin with routeBasePath: "somethingElse"

Please show a repro or a broken link checker error message to understand better the problem.

[d8niz]

Hi, any suggestions/improvements with this issue? I have the same error with the op as shown below when baseUrl: '/docs/' at gh-pages build pipe at 'yarn build' stage. Thank you in advance for any tips :)

/docs on   main ❯ yarn build                                                   ✘ INT 
yarn run v1.22.19                                                                                         
$ docusaurus build                                                                                        
[INFO] [en] Creating an optimized production build...                                                     
                                                                                                          
✔ Client                                                                                                  
                                                                                                          
                                                                                                          
✔ Server                                                                                                  
  Compiled successfully in 15.90s                                                                         
                                                                                                          
                                                                                                          
✔ Client                                                                                                  
                                                                                                          
                                                                                                          
● Server █████████████████████████ cache (99%) shutdown IdleFileCachePlugin                               
 stored                                                                                                   
                                                                                                          
[ERROR] Unable to build website for locale en.                                                            
[ERROR] Error: Docusaurus found broken links!                                                             
                                                                                                          
Please check the pages of your site in the list below, and make sure you don't reference any path that doe
s not exist.                                                                                              
Note: it's possible to ignore broken links with the 'onBrokenLinks' Docusaurus configuration, and let the 
build pass.                                                                                               
                                                                                                          
It looks like some of the broken links we found appear in many pages of your site.                        
Maybe those broken links appear on all pages through your site layout?                                    We recommend that you check your theme configuration for such links (particularly, theme navbar and footer
).                                                                                                        Frequent broken links are linking to:                                                                     
- /docs/docs/contributing/contributing                                                                                                                                                                              
Exhaustive list of all broken links found:                                                                                                                                                                          
- On source page path = /docs/404.html:                                                                      -> linking to /docs/docs/contributing/contributing                                                     
                                                     
- On source page path = /docs/blog:                
   -> linking to /docs/docs/contributing/contributing                                                     
                                                                                                          
- On source page path = /docs/blog/2019/05/28/hola:                                                       
   -> linking to /docs/docs/contributing/contributing 

[slorber]

@d8niz can't seem to reproduce on your repo here: https://github.com/eclipse-muto/docs https://stackblitz.com/github/eclipse-muto/docs

Please provide a branch with code that fails to build otherwise it's impossible to help

[d8niz]

• I am truly sorry for not withdrawing the issue here after we fixed our issues (there was an actual problem with our links) which i could not figure out by myself at the time

• When I found this issue i thought the problem might be similar (we recently upgraded to v2 that's why I was looking to the problem outside of our repository)

• We got our docs to be published here without any problems

• Thank you for reaching out (and for your time) and again sorry for the inconvenience my comment might have caused.

@slorber

[craftzneko]

Hi I have set blog off and docs only mode as per the documentation. This creates load of broken links as described above when building. I have not even begun customising the site yet this is template pages added when creating the site.

These are the build errors

[ERROR] Error: Docusaurus found broken links!

Please check the pages of your site in the list below, and make sure you don't reference any path that does not exist. Note: it's possible to ignore broken links with the 'onBrokenLinks' Docusaurus configuration, and let the build pass.

It looks like some of the broken links we found appear in many pages of your site. Maybe those broken links appear on all pages through your site layout? We recommend that you check your theme configuration for such links (particularly, theme navbar and footer). Frequent broken links are linking to:

• /docs/intro

• /docs/blog

• On source page path = /404.html: -> linking to /docs/intro -> linking to /blog

I have followed the documentation including setting a intro.md file and removing index.js

sample from config file

// Set the production url of your site here
  url: 'https://your-docusaurus-test-site.com',
  // Set the /<baseUrl>/ pathname under which your site is served
  // For GitHub pages deployment, it is often '/<projectName>/'
  baseUrl: '/',

  // GitHub pages deployment config.
  // If you aren't using GitHub pages, you don't need these.
  organizationName: 'facebook', // Usually your GitHub org/user name.
  projectName: 'docusaurus', // Usually your repo name.

  onBrokenLinks: 'throw',
  onBrokenMarkdownLinks: 'warn',

  // Even if you don't use internalization, you can use this field to set useful
  // metadata like html lang. For example, if your site is Chinese, you may want
  // to replace "en" with "zh-Hans".
  i18n: {
    defaultLocale: 'en',
    locales: ['en'],
  },

  presets: [
    [
      'classic',
      /** @type {import('@docusaurus/preset-classic').Options} */
      ({
        docs: {
          routeBasePath: '/',
          sidebarPath: require.resolve('./sidebars.js'),
          // Please change this to your repo.
          // Remove this to remove the "edit this page" links.
          editUrl:
            'https://github.com/facebook/docusaurus/tree/main/packages/create-docusaurus/templates/shared/',
        },
        blog: false,
        theme: {
          customCss: require.resolve('./src/css/custom.css'),
        },
      }),
    ],
  ],

  themeConfig:
    /** @type {import('@docusaurus/preset-classic').ThemeConfig} */
    ({
      // Replace with your project's social card
      image: 'img/docusaurus-social-card.jpg',
      navbar: {
        title: 'ICT Technotes',
        logo: {
          alt: 'My Site Logo',
          src: 'img/LogoSmall.png',
        },
        items: [
          {
            type: 'doc',
            docId: 'intro',
            position: 'left',
            label: 'Documentation',
          },
          //{to: '/blog', label: 'Blog', position: 'left'},
          {
            href: 'https://github.com/facebook/docusaurus',
            label: 'GitHub',
            position: 'right',
          },
        ],
      },

[craftzneko]

Resolved it by reviewing this other issue 3567

[ctrlaltdylan]

I solved it by using a different basePath other than /docs/, like /help/.

Then the conflict didn't arise, and relative file pathing worked just fine.

[drinkius]

Same issue here. Quite weird that the naming of the repo affects the end result. Also renamed it to /help and it helped

[Josh-Cena]

If your repo is called "docs" and it contains nothing else than docs (i.e. no landing page, no blog, etc.), you should set routeBasePath: "/" for your docs plugin (a la docs only mode). I don't think it makes much sense for your user to see a slug like https://example.github.io/docs/docs.

[maxwowpow]

baseUrl (not docs) + routeBasePath

 Frequent broken links are linking to:
  - /content/blog

  Exhaustive list of all broken links found:
  - Broken link on source page path = /content/404.html:
     -> linking to /content/blog

{
    baseUrl: '/content/',
     blog: {
            // tried /blog too
            routeBasePath: 'blog'
    },
    themeConfig:
      /** @type {import('@docusaurus/preset-classic').ThemeConfig} */
      ({
        navbar: {
          items: [
          {
         // fails for /blog, /content/blog, blog and same for with "to:" instead of "href:"
         href: 'blog',
          label: 'Blog',
          position: 'left'
           }
          ],
        },
}

• sounds similar to docs with the difference not the basePath but routeBasePath is set to blog.
• fails in navbar, footer and if used in .js files
• Version 3.1.0