next.js icon indicating copy to clipboard operation
next.js copied to clipboard

New middleware slows down routing in production

Open ch3rn1k opened this issue 3 years ago • 14 comments

Verify canary release

  • [X] I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System: Platform: win32 Arch: x64 Version: Windows 10 Pro
Binaries: Node: 16.14.2 npm: N/A Yarn: N/A pnpm: N/A Relevant packages: next: 12.2.0 eslint-config-next: 12.2.0 react: 18.2.0 react-dom: 18.2.0

What browser are you using? (if relevant)

Chrome 103.0.5060.66

How are you deploying your application? (if relevant)

next start

Describe the Bug

New middleware slows down on routing in production (dev is okay), example code that slows down:

import { NextResponse } from 'next/server';

const Middleware = (req) => {
  /** To lower case redirect. */
  if (req.nextUrl.pathname === req.nextUrl.pathname.toLowerCase())
    return NextResponse.next();

  return NextResponse.redirect(new URL(req.nextUrl.origin + req.nextUrl.pathname.toLowerCase()));
};

export default Middleware;

Expected Behavior

Work as expected with no slow down, with previous _middleware.js there was no problem

To Reproduce

Create middleware.js file under root folder and use code from example

ch3rn1k avatar Jul 03 '22 10:07 ch3rn1k

@ch3rn1k I have the similar issue but the reason (at least for me) is that the pages are being re-rendered if I use middleware.js in the root. I described it here: https://github.com/vercel/next.js/issues/38267. Could you please check and confirm if you have the same issue?

imangm avatar Jul 03 '22 10:07 imangm

@ch3rn1k I have the similar issue but the reason (at least for me) is that the pages are being re-rendered if I use middleware.js in the root. I described it here: #38267. Could you please check and confirm if you have the same issue?

Looks same so, but I got no error or slow work when no checks inside, like this:

import { NextResponse } from 'next/server';

const Middleware = (req) => {
 return NextResponse.next();
};

export default Middleware;

ch3rn1k avatar Jul 03 '22 10:07 ch3rn1k

I suppose this can also play a part in the performance degradation I filed in an issue a bit earlier:

  • #38235

willemliufdmg avatar Jul 04 '22 09:07 willemliufdmg

Also seeing this, also seeing the lack of slowdown when there are no checks / redirects as per @ch3rn1k's example. With 12.3, too.

jalada avatar Sep 16 '22 12:09 jalada

Well, same as before. Removed middleware when problem is started and no in use for me until this will be fixed.

ch3rn1k avatar Sep 16 '22 14:09 ch3rn1k

Also seeing this, tried it on 12.3.2 canary and the issue persists, we're talking massive slowdown. It seems to read many .json files all at once (loading SSR product detail page for example), pending for 10-20 seconds. Everything goes back to normal if middleware.ts is removed.

Everything was normal in the old _middleware.ts which was used in /pages

TurkaljGoran avatar Oct 25 '22 14:10 TurkaljGoran

I can confirm that the issue persists with node version 16.18.0 or 16.17.1. In any node version 16.5.0 and lower that I have tested the issue does not exist

11koukou avatar Oct 31 '22 10:10 11koukou

It may or may not be relevant that the middleware is now re-initialized on each request when in dev mode, rather than being initialized only once and then persisting like in previous versions.

GrantGryczan avatar Oct 31 '22 19:10 GrantGryczan

We are experiencing slow down when middleware is used together with large number of redirects. Probably because it affects also client side routing as described in documentation: "Redirects are not applied to client-side routing (Link, router.push), unless Middleware is present and matches the path."

Any idea how we can handle 30-60k of redirects of old urls (which cannot be changed to rules)? We are using Cloudlfront in front of our app but there are also smaller limits. So far we are working good with disabled middleware but thats only a temporary workaround.

michal-hans avatar Nov 05 '22 20:11 michal-hans

The problem also exists in the 13 release. Just the presence of a middleware increases the response time tenfold.

I did a test with a fresh next@canary release and a request to http://localhost:3000/vercel.svg takes 4ms without middleware. The same request with a middleware present, takes 150 - 300ms on my local machine. There seems to be a problem with the middleware loader, as the middleware does not have to do anything to cause that kind of delay.

// middleware.ts

export function middleware() {}

Is enough to drop the response time

cdev-enter avatar Nov 07 '22 21:11 cdev-enter

@ch3rn1k @imangm @willemliufdmg @jalada @TurkaljGoran @11koukou @michal-hans @cdev-enter

Does this change not fix the problem? https://github.com/vercel/next.js/pull/42547

baraeb92 avatar Nov 07 '22 22:11 baraeb92

@baraeb92 in 13.0.3-canary.1 this still exists.

cdev-enter avatar Nov 08 '22 21:11 cdev-enter

updating from 13.0.2 to the latest canary release fixed the issue in my project

artsiom-dorakhau avatar Nov 10 '22 14:11 artsiom-dorakhau

Having the same problem with the latest release 😢

gurleensethi avatar Nov 17 '22 04:11 gurleensethi

Without a fix for this, the framework is basically unusable right now ...

benfavre avatar Nov 18 '22 22:11 benfavre

It s not only in production. It happens in dev also. I just tested middleware.js with next 12.3.3 and it works perfectly. When replacing next version with 13.0.4, i get +1 second load time on my page.

alexandru-wd avatar Nov 21 '22 12:11 alexandru-wd

Do we have any update on this issue? Is it acknowledged by the Next team?

gurleensethi avatar Dec 03 '22 18:12 gurleensethi

Also having same problem. Wait time is too high. Because of that Web Core Vitals hurting.

metawise avatar Dec 06 '22 02:12 metawise

Also having this issue - is there any previous version of Next / middleware that doesn't suffer from this?

salokod avatar Dec 14 '22 17:12 salokod

I think 12.3.3 is safe - haven't seen any issues with it while using middleware for the past few weeks.

mgreenw avatar Dec 14 '22 18:12 mgreenw

Can anyone provide a reproduction? I think it will get resolved faster if there is a reproduction that the NextJS team can take a look at?

baraeb92 avatar Dec 15 '22 02:12 baraeb92

Next.js v13.0.7 still extremely slow, to a point where middleware is literally unusable

xyba1337 avatar Dec 22 '22 00:12 xyba1337

Can't confirm this with v13.0.7. Strangely enough, everything looks fine , I had this issue with previous releases and I definitely know how it is. Tested in node version 16.14.1 and 16.18.0

11koukou avatar Dec 22 '22 07:12 11koukou

Can't confirm this with v13.0.7. Strangely enough, everything looks fine , I had this issue with previous releases and I definitely know how it is. Tested in node version 16.14.1 and 16.18.0

Even in development environment? Tested it on pretty much every version from 13.0 to the latest one without any success. Also tried node version 16.14.1 and 16.18.0 just like you did, still extremely slow. Not having this issue with everything under 13.0 where it still used the old _middleware.js inside pages.

xyba1337 avatar Dec 22 '22 13:12 xyba1337

Adding a matcher to exclude resources helped in my case, of course it depends on your use case, in my case I didn't need a middleware for api/resources calls:

  matcher: [
    /*
     * Match all request paths except for the ones starting with:
     * - api (API routes)
     * - _next/static (static files)
     * - favicon.ico (favicon file)
     * - assets (static assets)
     */
    '/((?!api|_next/static|favicon.ico|assets).*)'
  ]
};

jasantaella15 avatar Jan 02 '23 12:01 jasantaella15

13.1 got released recently with some updates to middleware, is this issue fixed?

gurleensethi avatar Jan 05 '23 05:01 gurleensethi

Same issues. Middleware, even with a lot of exceptions, slows down (dev & production) builds.

wottpal avatar Feb 11 '23 13:02 wottpal

I've tested this out with: 13.1.6 canary release and its slowing down requests on dev mode (~80ms to 250ms delay per request). But on a production build its not causing any noticeable delay after the first request has been made to the server.

So it appears https://github.com/vercel/next.js/pull/42547 did solve the issue for production only.

However, The delay is still quite disruptive while working on dev.

I understand cache is disabled on the dev build to allow the compiler to catch code changes, but since the middleware.ts code is rarely changed, Would it be wise to provide a config option to force middleware cache in dev mode too?

majidkuhail avatar Feb 12 '23 17:02 majidkuhail

Adding a matcher to exclude resources helped in my case, of course it depends on your use case, in my case I didn't need a middleware for api/resources calls:

  matcher: [
    /*
     * Match all request paths except for the ones starting with:
     * - api (API routes)
     * - _next/static (static files)
     * - favicon.ico (favicon file)
     * - assets (static assets)
     */
    '/((?!api|_next/static|favicon.ico|assets).*)'
  ]
};

thanks this improved the performance a little bit

TechnoRahmon avatar Mar 06 '23 09:03 TechnoRahmon

I understand cache is disabled on the dev build to allow the compiler to catch code changes, but since the middleware.ts code is rarely changed, Would it be wise to provide a config option to force middleware cache in dev mode too?

Similar issues: https://github.com/vercel/next.js/issues/43999 https://github.com/vercel/next.js/discussions/44053

mgreenw avatar Mar 16 '23 13:03 mgreenw