`robots.txt` generation from `robots.ts` fails with `output: "export"` config in Next 15 RC & Canary.108

[samakintunde]

Link to the code that reproduces this issue

https://codesandbox.io/p/devbox/robots-txt-bug-xx45yn

To Reproduce

1. Build a static export with next build

Current vs. Expected behavior

After building the site, I should see an equivalent robots.txt generated from my robots.ts file but the build failed with the following error instead:

yarn build
yarn run v1.22.19
$ next build
  ▲ Next.js 15.0.0-canary.108

   Creating an optimized production build ...
 ✓ Compiled successfully
 ✓ Linting and checking validity of types    
   Collecting page data  ..Error: export const dynamic = "force-static"/export const revalidate not configured on route "/robo
ts.txt" with "output: export". See more info here: https://nextjs.org/docs/advanced-features/static-html-export                   at new eP (/project/workspace/node_modules/next/dist/compiled/next-server/app-route.runtime.prod.js:14:28341)
    at 4208 (/project/workspace/.next/server/app/robots.txt/route.js:1:954)
    at t (/project/workspace/.next/server/webpack-runtime.js:1:127)
    at r (/project/workspace/.next/server/app/robots.txt/route.js:16:652)
    at /project/workspace/.next/server/app/robots.txt/route.js:16:682
    at t.X (/project/workspace/.next/server/webpack-runtime.js:1:1191)
    at /project/workspace/.next/server/app/robots.txt/route.js:16:665
    at Object.<anonymous> (/project/workspace/.next/server/app/robots.txt/route.js:16:709)
    at Module._compile (node:internal/modules/cjs/loader:1369:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1427:10)

> Build error occurred
Error: Failed to collect page data for /robots.txt
    at /project/workspace/node_modules/next/dist/build/utils.js:1246:15
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  type: 'Error'
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

The error suggests to perhaps add export const dynamic = "force-static" which doesn't fix the issue.

Provide environment information

Operating System:
  Platform: linux
  Arch: x64
  Version: #1 SMP PREEMPT_DYNAMIC Sun Aug  6 20:05:33 UTC 2023
  Available memory (MB): 8198
  Available CPU cores: 4
Binaries:
  Node: 20.12.0
  npm: 10.5.0
  Yarn: 1.22.19
  pnpm: 8.15.6
Relevant Packages:
  next: 15.0.0-canary.108 // Latest available version is detected (15.0.0-canary.108).
  eslint-config-next: 15.0.0-rc.0
  react: 19.0.0-rc-e948a5ac-20240807
  react-dom: 19.0.0-rc-e948a5ac-20240807
  typescript: 5.5.4
Next.js Config:
  output: export

Which area(s) are affected? (Select all that apply)

Metadata

Which stage(s) are affected? (Select all that apply)

next dev (local), next build (local)

Additional context

I tested the reproduction against the 15.0.0-rc.0 and the latest installable canary version 15.0.0-canary.108.

[wojtekmaj]

Unfortunately, stable Next.js 15.0.0 also suffers from this bug.

It also affects sitemap.xml generation.

[prokopsimek]

I can confirm that it affects the sitemap.xml too when running next build in next v15.0.1.

 ✓ Compiled successfully
   Skipping linting
 ✓ Checking validity of types    
   Collecting page data  ..ReferenceError: Cannot access 'l' before initialization
    at Object.k1 (/Users/prokop/repos/_dxheroes/dxheroes-web/web/.next/server/app/sitemap.xml/route.js:17:57770)
    at /Users/prokop/repos/_dxheroes/dxheroes-web/web/.next/server/app/sitemap.xml/route.js:17:58225

> Build error occurred
Error: Failed to collect page data for /sitemap.xml
    at /Users/prokop/repos/_dxheroes/dxheroes-web/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/next/dist/build/utils.js:1233:15 {
  type: 'Error'
}
 ELIFECYCLE  Command failed with exit code 1.

I am calling my class CMSClient in src/app/sitemap.ts with fetch set to 'force-cache'.

  // use next fetch to deduplicate requests
  private static HttpAgent = {
    get: async <T>(path: string, params: { params: object }): Promise<{ data: T }> => {
      const queryParams = qs.stringify(params.params, { arrayFormat: 'indices', encodeValuesOnly: true });

      const req = await fetch(`${CMSClient.url}/api/${path}?${queryParams}`, {
        headers: {
          Authorization: `Bearer ${process.env.NEXT_PUBLIC_STRAPI_JWT}`,
        },
        cache: 'force-cache',
      });

      const response = (await req.json()) as T;

      return { data: response };
    },
  };

The app fails only when running next build. It works correctly when running next dev --turbopack. Even localhost:3000/sitemap.xml works fine.

[iwebexpert]

I'm using Next.js version 15.0.2. I solved this problem by simply adding export const dynamic = 'force-static'. Here is my robots.ts file:

import type { MetadataRoute } from 'next'

export const dynamic = 'force-static'

export default function robots(): MetadataRoute.Robots {
  const isNoindex = process.env.APP_INDEX_MODE === 'NOINDEX'
  const domain = process.env.NEXT_PUBLIC_APP_DOMAIN

  return {
    rules: isNoindex
      ? {
          userAgent: '*',
          disallow: '/',
        }
      : {
          userAgent: '*',
          allow: '/',
        },
    sitemap: `${domain}/sitemap.xml`,
  }
}

[trainto]

I had the same issue with manifest.ts, and solved with export const dynamic = 'force-static' too.

But I've found this still occurs while dev mode, next dev --turbopack, however if I remove --turbopack, it works as expected.

Nextjs version 15.1.6

[junaidzamaahmed]

I had the same issue and I fixed it by adding export const dynamic = 'force-static' in both my sitemap.ts and robots.ts

Nextjs version 15.3.1

[easygofar]

I had the same issue and I fixed it by adding export const dynamic = 'force-static' in my sitemap.js

Nextjs version 15.4.3

[EnzoPV25]

On next 15.3.3, having this robot.ts :

import { baseUrl } from "@/lib/seo-utils";
import type { MetadataRoute } from "next";

export const dynamic = "force-static";

export default function robots(): MetadataRoute.Robots {
  return {
    rules: {
      userAgent: "*",
      allow: "/",
      disallow: ["*/404", "/_next/", "/api/"],
    },
    sitemap: `${baseUrl}/sitemap.xml`,
  };
}

Doesn't generate my robot.txt

However my sitemap is being created correctly with export const dynamic = "force-static";

Any update on this ?

[sushantgwr87]

@EnzoPV25 rename your file from robot.ts to robots.ts - missing "s" at the end. Nextjs file convention uses robots.ts.

[EnzoPV25]

Oh damn, thank you, infact in now works.