Async generateMetadata hangs the app without any visible sign of loading

[wiwo-dev]

Link to the code that reproduces this issue or a replay of the bug

https://github.com/wiwo-dev/generatemetadata-loading-issue https://codesandbox.io/p/github/wiwo-dev/generatemetadata-loading-issue/main

To Reproduce

1. Run the application using the command: yarn run dev.

2. Navigate to the root path (/). Here, you'll find two lists of "users" retrieved from jsonplaceholder. The list at the top has the problem as it uses generateMetadata. The one at the bottom doesn't use generateMetadata.

3. The top list contains links to /user/[userId], and these links have a generateMetadata function that fetches data. Upon clicking these links, for the initial 3 seconds, there is no visible loading indication, making users believe that something might be wrong with the link.

4. In contrast, if you click on any of the links in the bottom list, which leads to /user-no-metadata/[userId] that doesn't use generateMetadata, you'll notice that a Loading state is triggered by Suspense, providing a clear loading indication.

Current vs. Expected behavior

Current behaviour When a user navigates to a page containing a generateMetadata function that fetches data from a slowly working API, there's a problem. The application appears to hang, providing no indication that it's actively loading. Notably, the fetch operation within generateMetadata doesn't trigger React's Suspense, and the browser also fails to display any loading indicators.

To simulate a slow API response, a setTimeout is intentionally used within the getUsers function.

I encountered this issue while working on a project with a tens of thousands number of subpages, and it's highly likely that a given subpage will be opened for the first time and only once, with new entries continually being added.

Expected behaviour It should trigger the Suspense or show any kind of loading indicator in the browser so the user knows that something is happening under the hood and wait for the page to be displayed.

Verify canary release

• [X] I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System:
      Platform: darwin
      Arch: arm64
      Version: Darwin Kernel Version 22.4.0: Mon Mar  6 21:00:41 PST 2023; root:xnu-8796.101.5~3/RELEASE_ARM64_T8103
    Binaries:
      Node: 18.7.0
      npm: 8.15.0
      Yarn: 1.22.15
      pnpm: 6.11.0
    Relevant Packages:
      next: 13.4.20-canary.36
      eslint-config-next: N/A
      react: 18.2.0
      react-dom: 18.2.0
      typescript: 5.1.3
    Next.js Config:
      output: N/A

Which area(s) are affected? (Select all that apply)

Metadata (metadata, generateMetadata, next/head)

Additional context

Is there a way to display a loading state or any form of indication that something is loading while still utilising the generateMetadata function to fetch metadata information from a slow API?

[jetaggart]

We're also seeing this issue, which is a pretty big issue with for us. We have slow endpoints that generate our metadata and right now it feels like a tradeoff between bad SEO and using this feature. Is there a workaround to get metadata on the page without blocking or getting a loading ui/suspense boundary for page navigation?

I don't know if it helps but it seems the rsc is called twice in a row. If we have generateMetadata on, the first request takes as long as the metadata does to return, which doesn't seem to be caught by any suspense/loading ui boundary. If we take out the generateMetadata, both requests are still fired, but the first one is nearly instant with the second one taking some time but that is correctly picked up by the suspense loading boundary.

[leon-marzahn]

Same issue, couldn't generateMetadata be called compile time as well with a revalidation?

[borispoehland]

What helps a little is using unstable_cache to cache all subsequent requests via ISR:

import { unstable_cache as cache } from 'next/cache'

const getData = cache(
  (id) => fetch(...),
  ['getData'],
  { revalidate: 300 }
)

export async function generateMetadata() {
  const data = await getData();
  return { title: data.title };
}

Now it will block with getData on the first request (still bad), from then on always return cached data and revalidate in the background.

To not let it be blocking on the first request, you'd need a way of pre-running the getData on build time. Currently everything inside generateMetadata does not get called on build time

[eijil]

same !!!

Edit by maintainer bot: Comment was automatically minimized because it was considered unhelpful. (If you think this was by mistake, let us know). Please only comment if it adds context to the issue. If you want to express that you have the same problem, use the upvote 👍 on the issue description or subscribe to the issue for updates. Thanks!

[eijil]

This should be a very commonly used feature, and it is surprising that the official team has not provided any fixes.

[lpwprojects]

Looks like this issue is the same as #45418

[ArianHamdi]

Same issue here. At least I expect generateMetadata to trigger suspense, and also I would like to bypass calling generateMetadata on client-side navigation, show the page instantly, and update metadata in the background.

[jaakkohurtta]

Sad to see this has not gotten any traction from the Next team as the issue is months old already. This is very lousy UX and is making me seriously consider moving my app back to pages.

[Matt-roki]

This is now a blocker for a production release on my app. Below are two screenshots of loading times, the first is excluding generateMetadata from the page file. The second is with generateMetadata resulting in a 4 SECOND SLOWER page load.

I have already cached the request with this function const getCachedData = cache(async (params) => { const { locale, slug, searchParams } = params; const preview = false; const domainVersion = process.env.DOMAIN_VERSION; return await fetchCommonData({ locale, preview, slug, domainVersion, searchParams }); });

It is then reused in the same page file for getting the pageData and the metadata.

Going to have to figure out a way to load the metadata after the first request is done...

[steve-marmalade]

Same issue here. At least I expect generateMetadata to trigger suspense, and also I would like to bypass calling generateMetadata on client-side navigation, show the page instantly, and update metadata in the background.

I do hope this issue gets attention, as @ArianHamdi's comment is exactly what I hoped/expected the functionality to be.

@Matt-roki , in the meantime, I was able to implement a work-around inspired by https://github.com/vercel/next.js/issues/45418#issuecomment-1914076788 that has seemed to resolve our issues.

[popovidis]

Same issue. Metadata taking ~6 seconds.

[maxmdev]

Any updates?

Edit by maintainer bot: Comment was automatically minimized because it was considered unhelpful. (If you think this was by mistake, let us know). Please only comment if it adds context to the issue. If you want to express that you have the same problem, use the upvote 👍 on the issue description or subscribe to the issue for updates. Thanks!

[RuslanAsadov]

+, same problem

[roman-veryovkin]

Upvote the issue as well. Waiting for the solution out of the box, please)

Edit by maintainer bot: Comment was automatically minimized because it was considered unhelpful. (If you think this was by mistake, let us know). Please only comment if it adds context to the issue. If you want to express that you have the same problem, use the upvote 👍 on the issue description or subscribe to the issue for updates. Thanks!

[Matt-roki]

Okay the best solution I have found for this is to remove the async part of generateMetadata.

Now for me all of my seo comes from a seo and that wouldnt work without async.

I now have a prebuild file which runs and generates a seo.json file that I can read my metadata from

"scripts": { ... "build": "node ./utils/generateSEO.js && next build", //this here runs a script to get all my seo data and sort it accordingly. ... },

Simplified code below.

export function generateMetadata({ params: { fullSlug } }) { try{ ... const seoData = jsonSEO.find(v => v.locale === locale && v.slug === findSlug) || {};

This fixes most issues the only downside is when a seo change is made a build is needed.

[feedthejim]

Hey, I wanted to give a few updates on this.

• like @borispoehland, if the data is cache-able you should use unstable_cache to cache it
• however that still will cause the first hit to be slow, since it won't be cached, this is a known limitation. When we release PPR, you'll be able to statically pregenerate the metadata and combine it with a dynamic/ssr page.
• if the data truly can't be cached, we're also planning on making metadata streamable, which means that we would not block on it during render, allowing you to serve the response asap and potentially the metadata 6s+ after, like some cases in this thread.

[steve-marmalade]

Thank you for the update @feedthejim .

While those enhancements seem worthy, they also don't seem like quick wins. Did you consider a solution similar to what @ArianHamdi wrote above (at least in the short-term):

I would like to bypass calling generateMetadata on client-side navigation, show the page instantly, and update metadata in the background.

I think that would go a long way to alleviating this issue, as we would be paying the cost on first-load only, whereas now it's per-page.

[dannytlake]

+1 for bypassing the blocker waiting for async generateMetadata to resolve during client side navigation.

[yarinsa]

+1 here. Would have been nice to have an "optimistic" behavior.

[KajSzy]

• if the data truly can't be cached, we're also planning on making metadata streamable, which means that we would not block on it during render, allowing you to serve the response asap and potentially the metadata 6s+ after, like some cases in this thread.

If it won't be blocking loading.tsx and will be able to use deduping of fetch request across layout/page it might be the most popular and versatile solution. Will it be shipped in v14 or is designed for v15 only?

[pdelfan]

Hey, I wanted to give a few updates on this.

• like @borispoehland, if the data is cache-able you should use unstable_cache to cache it
• however that still will cause the first hit to be slow, since it won't be cached, this is a known limitation. When we release PPR, you'll be able to statically pregenerate the metadata and combine it with a dynamic/ssr page.
• if the data truly can't be cached, we're also planning on making metadata streamable, which means that we would not block on it during render, allowing you to serve the response asap and potentially the metadata 6s+ after, like some cases in this thread.

Thanks for this. Is there any update on streamable metadata?

[cjcheshire]

If it does become streamable will search engines accept this?

[ebidrey]

The problem is very clear in the following code snippet using app router.

1. Page got stuck for 2 seconds in loading state while generateMetadata and TestContent are running in parallel
2. When generateMetadata finish, <Suspense> start running for 2 seconds waiting TestContent to finish.
3. The page renders.

The first point is the problem, because <Suspense> must start running from the beginning and the full page with its metadata should render together.

import { Suspense } from "react"

export async function generateMetadata() {
    await new Promise(resolve => setTimeout(resolve, 2000))
    return {
        title: 'Test',
        description: 'Test',
    }
}

async function TestContent() {
    await new Promise(resolve => setTimeout(resolve, 4000))
    return  <h1>Test</h1>
}

export default async function Test() {
    return (
        <Suspense fallback={<div>loading</div>}>
            <TestContent />
        </Suspense>
    )
}

Neither latest nor canary are working as expected. With this issue is impossible to create an elegant, efficient and SEO friendly page with app router. Please help.

[flexdinesh]

I'm working on a pretty big next.js app that's on pages router. Our pages would look somewhat like this:

my-app/
├─ src/
│  ├─ pages/
│  │  ├─ foo/
│  │  │  ├─ bar/
│  │  │  │  ├─ index.tsx (ssr with `getServerSideProps`)
│  │  │  ├─ index.tsx (ssr with `getServerSideProps`)

Our biggest pain with pages router is that getServerSideProps of /foo/bar would run every time when the user navigates from /foo page to /bar even though bar page shares a lot of data with foo page. This resulted in a sluggish navigation experience where the user will receive no UX feedback for ~500ms (that's how long our roundtrip time takes). However, pages router had routeChangeStart and routeChangeComplete event listeners which we could use to know if the user requested a client side navigation. We used that to show a loader to the user in the /foo page before /foo/bar page loaded. It was not ideal, as ideal would be able to show the loader in /foo/bar page but it was still better than not showing the user any feedback at all when we're waiting for the server response to continue with the navigation.

We were super excited when app router was announced with persisted layouts. We waited two years to give it some time for app router to be stable. We recently did an app router migration spike and we found that navigation from /foo to /foo/bar was super fast because of two reasons:

1. We can create a layout.tsx file for /foo that would persist when we navigate to /foo/bar so the bar page only has to load data necessary to render bar
2. Because of Suspense, we were able to take the user to the /foo/bar page as soon as the user clicked/pressed the link to the page and show a loader while the data was loaded for the bar page.

But when we experimented with app router we only tested the layouts and pages, we did not test metadata as the docs did not mention anything related to generateMetadata function blocking client side navigations. It only mentions that the initial load will be blocked before the data can be streamed. It does not mention anything about suspense loader getting blocked during client side transitions.

On the initial load, streaming is blocked until generateMetadata has fully resolved, including any content from loading.js

We're working on a very big app. Our team spent a few weeks migrating some of our pages from pages router to app router. With app router, our pages look somewhat like this:

my-app/
├─ src/
│  ├─ app/
│  │  ├─ foo/
│  │  │  ├─ bar/
│  │  │  │  ├─ page.tsx (has generateMetadata() and generateViewport())
│  │  │  ├─ page.tsx (has generateMetadata() and generateViewport())
│  │  │  ├─ layout.tsx

Things were looking good until we added the generateMetadata function. After adding the function to bar page the user receives no feedback at all until the server response comes back after navigation. The user is now just waiting there not knowing what happened after clicking/pressing on a link for approxmiately ~400ms when their connection is fast, when it's slow, the user is waiting without any feeback for ~2s. This is terrible for UX. Unlike pages router, we don't have something like routeChangeStart and routeChangeComplete in the app router, so we can't show any loading UX even on the current page. Now we're caught between the pages router and the app router world with little to no hope.

@feedthejim none of the options you suggested would work for us. It's a self-hosted app. Our data is very fast and cached in a different layer and not in the render layer for freshness reasons. We can't use unstable_cache, PPR or use cache. We don't want to bother setting up our infra with streaming either. We just want to be able to server render a fully dynamic page with persisted layouts and not have generateMetadata and generateViewport functions block the Suspense loader. Our UX was a lot better in pages router even though we were loading more data than necessary during client-side navigations to /foo/bar page.

[ebidrey]

I'm working on a pretty big next.js app that's on pages router. Our pages would look somewhat like this:

my-app/

├─ src/

│  ├─ pages/

│  │  ├─ foo/

│  │  │  ├─ bar/

│  │  │  │  ├─ index.tsx (ssr with `getServerSideProps`)

│  │  │  ├─ index.tsx (ssr with `getServerSideProps`)

Our biggest pain with pages router is that getServerSideProps of /foo/bar would run every time when the user navigates from /foo page to /bar even though bar page shares a lot of data with foo page. This resulted in a sluggish navigation experience where the user will receive no UX feedback for ~500ms (that's how long our roundtrip time takes). However, pages router had routeChangeStart and routeChangeComplete event listeners which we could use to know if the user requested a client side navigation. We used that to show a loader to the user in the /foo page before /foo/bar page loaded. It was not ideal, as ideal would be able to show the loader in /foo/bar page but it was still better than not showing the user any feedback at all when we're waiting for the server response to continue with the navigation.

We were super excited when app router was announced with persisted layouts. We waited two years to give it some time for app router to be stable. We recently did an app router migration spike and we found that navigation from /foo to /foo/bar was super fast because of two reasons:

1. We can create a layout.tsx file for /foo that would persist when we navigate to /foo/bar so the bar page only has to load data necessary to render bar

2. Because of Suspense, we were able to take the user to the /foo/bar page as soon as the user clicked/pressed the link to the page and show a loader while the data was loaded for the bar page.

But when we experimented with app router we only tested the layouts and pages, we did not test metadata as the docs did not mention anything related to generateMetadata function blocking client side navigations. It only mentions that the initial load will be blocked before the data can be streamed. It does not mention anything about suspense loader getting blocked during client side transitions.

On the initial load, streaming is blocked until generateMetadata has fully resolved, including any content from loading.js

We're working on a very big app. Our team spent a few weeks migrating some of our pages from pages router to app router. With app router, our pages look somewhat like this:

my-app/

├─ src/

│  ├─ app/

│  │  ├─ foo/

│  │  │  ├─ bar/

│  │  │  │  ├─ page.tsx (has generateMetadata() and generateViewport())

│  │  │  ├─ page.tsx (has generateMetadata() and generateViewport())

│  │  │  ├─ layout.tsx

Things were looking good until we added the generateMetadata function. After adding the function to bar page the user receives no feedback at all until the server response comes back after navigation. The user is now just waiting there not knowing what happened after clicking/pressing on a link for approxmiately ~400ms when their connection is fast, when it's slow, the user is waiting without any feeback for ~2s. This is terrible for UX. Unlike pages router, we don't have something like routeChangeStart and routeChangeComplete in the app router, so we can't show any loading UX even on the current page. Now we're caught between the pages router and the app router world with little to no hope.

@feedthejim none of the options you suggested would work for us. It's a self-hosted app. Our data is very fast and cached in a different layer and not in the render layer for freshness reasons. We can't use unstable_cache, PPR or use cache. We don't want to bother setting up our infra with streaming either. We just want to be able to server render a fully dynamic page with persisted layouts and not have generateMetadata and generateViewport functions block the Suspense loader. Our UX was a lot better in pages router even though we were loading more data than necessary during client-side navigations to /foo/bar page.

Exactly the same issue here, caught between page and app router.

[philwolstenholme]

👀 https://x.com/stewiemcstews/status/1863680862308667716

[huozhi]

👋 Hi everyone! We heard your requests.

We're introducing Streaming Metadata as the enhancement of the recent canary releases to improve metadata rendering performance.

Introduction

This feature is designed to improve page loading time for users, especially when dealing with slow metadata generation. With Streaming Metadata enabled:

• The generateMetadata returned metadata is treated as suspenseful data, allowing the page content to render immediately.
• Metadata is then injected asynchronously into the page on the client side once it’s resolved.

Since metadata is critical for SEO, we’ve ensured that bots continue to receive fully rendered metadata in the HTML. For human users, who primarily care about page content, metadata can be added later without impacting their experience.

Usage

The following new options in next.config.js are for configuring Streaming Metadata:

• experimental.streamingMetadata (boolean): Enables the Streaming Metadata feature (disabled by default)
• experimental.htmlLimitedBots optional (RegExp): Specifies a regex to identify bot user agents that should opt out of the streaming metadata feature and continue receiving fully blocking metadata in the HTML. (We have a list of common bots which have limited ability to handle metadata rendering)

By default, only the Google bots which can act like headless browsers will receive streaming metadata when you enable the feature. Other bots will receive blocking metadata like the ones configured in htmlLimitedBots. If you noticed any bots also need to consume blocking metadata, please let us know to add it into default list.

If you don't need to customize the bots user agents for the blocking metadata and default config is enough for you, then only need to have one flag enabled in your next.config.js

module.exports = {
  experimental: {
    streamingMetadata: true,
  },
}

Notes

We’re still working on optimizations for PPR mode. However, if you’re not using PPR, you can try out this feature now to accelerate pages with slow generateMetadata() calls.

We’d love to hear your thoughts! Please feel free to share any comments, suggestions, or concerns in this thread. We'd love to hear your feedbacks! Thank you! 🚀

[ssotomayor]

@huozhi

👋 Hi everyone! We heard your requests.

We're introducing Streaming Metadata as the enhancement of the recent canary releases to improve metadata rendering performance.

Introduction

This feature is designed to improve page loading time for users, especially when dealing with slow metadata generation. With Streaming Metadata enabled:

• The generateMetadata returned metadata is treated as suspenseful data, allowing the page content to render immediately.
• Metadata is then injected asynchronously into the page on the client side once it’s resolved.

Since metadata is critical for SEO, we’ve ensured that bots continue to receive fully rendered metadata in the HTML. For human users, who primarily care about page content, metadata can be added later without impacting their experience.

Usage

The following new options in next.config.js are for configuring Streaming Metadata:

• experimental.streamingMetadata (boolean): Enables the Streaming Metadata feature (disabled by default)
• experimental.htmlLimitedBots optional (RegExp): Specifies a regex to identify bot user agents that should opt out of the streaming metadata feature and continue receiving fully blocking metadata in the HTML. (We have a list of common bots which have limited ability to handle metadata rendering)

By default, only the Google bots which can act like headless browsers will receive streaming metadata when you enable the feature. Other bots will receive blocking metadata like the ones configured in htmlLimitedBots. If you noticed any bots also need to consume blocking metadata, please let us know to add it into default list.

If you don't need to customize the bots user agents for the blocking metadata and default config is enough for you, then only need to have one flag enabled in your next.config.js

module.exports = { experimental: { streamingMetadata: true, }, }

Notes

We’re still working on optimizations for PPR mode. However, if you’re not using PPR, you can try out this feature now to accelerate pages with slow generateMetadata() calls.

We’d love to hear your thoughts! Please feel free to share any comments, suggestions, or concerns in this thread. We'd love to hear your feedbacks! Thank you! 🚀

Just tested this on latest canary, it's amazing when running with next dev with turbopack, exactly what's expected, however when i make a production build i don't have the same behaviour, the navigation becomes slow yet again when navigating to a dynamic route with a slow call for generateMetadata.

[huozhi]

@ssotomayor Thanks for the feedback, can you share a bit more about user case? A reproduction would be the best, I can help look into it.

[pondorasti]

We're introducing Streaming Metadata as the enhancement of the recent canary releases to improve metadata rendering performance.

yesss, this makes me so happy! e-comm style apps with slow metadata has been one of my biggest gripes with Next.js as it absolutely tanks INP for link navigation. I just migrated the entirety of emojis.com to the latest canary with streamingMetadata and results are positive!