next.js icon indicating copy to clipboard operation
next.js copied to clipboard

Missing prefetch headers in Middleware

Open lucasmotta opened this issue 1 year ago • 9 comments

Link to the code that reproduces this issue

https://codesandbox.io/p/devbox/nextjs-missing-prefetch-header-5ll75f?file=%2Fapp%2Flayout.tsx%3A38%2C17

To Reproduce

  1. Build the application for production CleanShot 2024-03-26 at 13 08 19@2x
  2. Run production server CleanShot 2024-03-26 at 13 08 54@2x
  3. Inspect the terminal logs when the server starts image

You will notice that as soon as you load the / page, the middleware logs the pages the current page and also the pages being prefetched, and that later should not happen.

Current vs. Expected behavior

The current documentation suggests that if you don't want the middleware to run for pages that are being prefetched, you can target the next-router-prefetch and purpose headers:

export const config = {
  matcher: [
    {
      source: '/((?!api|_next/static|_next/image|favicon.ico).*)',
      missing: [
        { type: 'header', key: 'next-router-prefetch' },
        { type: 'header', key: 'purpose', value: 'prefetch' },
      ],
    },
  ],
};

Unfortunately that does not seem to work in the latest version of Next.js - those headers are never present, therefore "missing" - so the middleware is running for pages that are being prefetched, which causes issues if you are handling redirects/rewrites, etc...

In my example app, I've a couple Link components in the root layout with a prefetch prop. Those links are being prefetched, but they are also running in the middleware (see the logs) and no prefetch headers are added to those prefetched pages.

I would expect those headers to be available in the middleware for pages that are being prefetched or at least another way for the middleware to know when a page is being prefetched or not.

Provide environment information

Operating System:
  Platform: linux
  Arch: x64
  Version: #1 SMP PREEMPT_DYNAMIC Sun Aug  6 20:05:33 UTC 2023
Binaries:
  Node: 20.11.1
  npm: 10.2.4
  Yarn: 1.22.19
  pnpm: 8.15.4
Relevant Packages:
  next: 14.1.4
  eslint-config-next: 14.1.4
  react: 18.2.0
  react-dom: 18.2.0
  typescript: 5.4.3
Next.js Config:
  output: N/A

Which area(s) are affected? (Select all that apply)

App Router, Middleware / Edge (API routes, runtime)

Which stage(s) are affected? (Select all that apply)

next build (local), next start (local), Vercel (Deployed)

Additional context

I also tested with version 13.5.6 and the issue remains.

I also remove the missing property and added logs to those headers, but they always return as null no matter what.

lucasmotta avatar Mar 26 '24 16:03 lucasmotta

Upon some further investigation, it looks like the headers are present when using the pages/ directory. I forked my example and switched to pages/ directory and now the middleware correctly filters out prefetched pages from the middleware. https://codesandbox.io/p/devbox/nextjs-missing-prefetch-header-pages-5ckpjs

And this can be confirmed by inspecting the Network request: CleanShot 2024-03-26 at 14 40 28@2x

So is it safe to assume that this discrepancy between pages/ and app/ router is a bug? Or is that intentional?

lucasmotta avatar Mar 26 '24 17:03 lucasmotta

It is a major blocker for my project as well, can we fix this for app/ router

akshit-mdr avatar Jun 12 '24 05:06 akshit-mdr

need this

juliansommer avatar Jun 12 '24 06:06 juliansommer

missing: [
    { type: 'header', key: 'next-router-prefetch' },
    { type: 'header', key: 'purpose', value: 'prefetch' },
  ],

is also not working as expected in middleware.ts in our application using v14.0.3 of Next.js.

This is a big problem for us and we are due to go live soon. We don't want to be running our middleware for every link that is prefetched. We are worried that this could run up the quotas for Edge Middleware functions on Vercel if we receive a lot of traffic.

lmac-1 avatar Jun 27 '24 17:06 lmac-1

I encountered the same issue. After some debugging, I discovered that the headers are missing when using <Link prefetch={true} /> where prefetch is explicitly set to true. However, when it's left as the default (undefined), the headers are present in the prefetch request.

kamisone avatar Sep 21 '24 17:09 kamisone

any solutions yet? I'm making analytics but I can't add all the prefetched routes as visit

kimhwanhoon avatar Sep 26 '24 16:09 kimhwanhoon

This is pretty annoying as we cant run our own analytics through middleware, as every prefetch triggers it.

dimitrisnl avatar Oct 24 '24 08:10 dimitrisnl

This is pretty annoying as we cant run our own analytics through middleware, as every prefetch triggers it.

On middleware if (req.headers.get("next-router-prefetch")) { return NextResponse.next(); // Ignore Prefetch request }

kimhwanhoon avatar Oct 24 '24 08:10 kimhwanhoon

This is pretty annoying as we cant run our own analytics through middleware, as every prefetch triggers it.

On middleware if (req.headers.get("next-router-prefetch")) { return NextResponse.next(); // Ignore Prefetch request }

thank you for the reply, but this does nothing unfortunately.

dimitrisnl avatar Oct 24 '24 11:10 dimitrisnl

This is currently an issue on the latest canary 15.0.4-canary.24

bel7aG avatar Nov 23 '24 16:11 bel7aG

This is an issue, any potential fixes?

Edit by maintainer bot: Comment was automatically minimized because it was considered unhelpful. (If you think this was by mistake, let us know). Please only comment if it adds context to the issue. If you want to express that you have the same problem, use the upvote 👍 on the issue description or subscribe to the issue for updates. Thanks!

blazorin avatar Nov 30 '24 14:11 blazorin

Any updates on this? Or maybe any workarounds for distinguishing the prefetch calls in middleware?

justenau avatar Dec 13 '24 07:12 justenau

For me the solution was to set the header in the nginx:

location / { .... proxy_set_header x-next-router-prefetch $http_next_router_prefetch; ... }

kamisone avatar Dec 13 '24 10:12 kamisone

Instead of disabling middleware with the missing matcher, you can detect prefetch requests by checking the request headers:

const isPrefetch =
  request.headers.get('sec-fetch-mode') === 'cors' ||
  request.headers.get('sec-fetch-dest') === 'empty';

zymantas-katinas avatar Dec 30 '24 12:12 zymantas-katinas

what is the update on this @lucasmotta

Edit by maintainer bot: Comment was automatically minimized because it was considered unhelpful. (If you think this was by mistake, let us know). Please only comment if it adds context to the issue. If you want to express that you have the same problem, use the upvote 👍 on the issue description or subscribe to the issue for updates. Thanks!

harisvsulaiman avatar May 13 '25 13:05 harisvsulaiman

Hey guys, I'm having the same issue in the latest version of NextJS. Is there any update on this?

dartilesm avatar May 20 '25 16:05 dartilesm