Cache-Control headers set in next.config.js are overwritten

[hartshorne]

What version of Next.js are you using?

10.0.7, 10.0.8-canary.3

What version of Node.js are you using?

15.8.0

What browser are you using?

curl, Chrome

What operating system are you using?

macOS

How are you deploying your application?

next start

Describe the Bug

Custom Cache-Control headers configured in next.config.js are overwritten in some cases. It looks any page that use getStaticProps will have their cache headers overwritten with Cache-Control: s-maxage=31536000, stale-while-revalidate which seems to come from https://github.com/vercel/next.js/blob/80c9522750a269a78c5636ace777a734b9dcd767/packages/next/next-server/server/send-payload.ts#L34

Expected Behavior

When we configure a Cache-Control header, don't set it to something else.

To Reproduce

Starting from yarn create next-app --example headers headers-app, here's a sample project that demonstrates the bug: https://github.com/hartshorne/headers-app

You can clone the project, and run something like yarn build && yarn start to start it in production mode. (dev mode overwrites all Cache-Control headers to prevent the browser from caching during development, which makes sense.)

Here's the next.config.js: https://github.com/hartshorne/headers-app/blob/main/next.config.js

Here's props.js (which exports getStaticProps) – this is broken: https://github.com/hartshorne/headers-app/blob/main/pages/props.js

% curl -I http://localhost:3000/props 
HTTP/1.1 200 OK
Cache-Control: s-maxage=31536000, stale-while-revalidate
X-Custom-Header: with static props
X-Powered-By: Next.js
ETag: "978-AzRqkosC2YfRQvbfuY7KiKb51e8"
Content-Type: text/html; charset=utf-8
Content-Length: 2424
Vary: Accept-Encoding
Date: Thu, 18 Feb 2021 22:09:41 GMT
Connection: keep-alive
Keep-Alive: timeout=5

Here's static.js — this works: https://github.com/hartshorne/headers-app/blob/main/pages/props.js

% curl -I http://localhost:3000/static     
HTTP/1.1 200 OK
Cache-Control: public, max-age=3600, s-maxage=60, stale-while-revalidate=86400
X-Custom-Header: no props, no link
X-Powered-By: Next.js
ETag: "945-MX0a4VCO/O001kyDcNn9a55taLQ"
Content-Type: text/html; charset=utf-8
Content-Length: 2373
Vary: Accept-Encoding
Date: Thu, 18 Feb 2021 22:09:10 GMT
Connection: keep-alive
Keep-Alive: timeout=5

Note that X-Custom-Header comes through, but the Cache-Control header is overwritten. Same behavior in Chrome, and with a regular GET request.

[hartshorne]

@ijjk, it looks like https://github.com/vercel/next.js/commit/a32b1f487062c2976aa953a81f46069be2584425 introduced code that would overwrite custom Cache-Control headers -- maybe we can check to see if the Cache-Control headers were set before overwriting them?

[hartshorne]

The bug exists in https://github.com/vercel/next.js/releases/tag/v10.0.8-canary.3 as well: https://github.com/hartshorne/headers-app/tree/canary

[hartshorne]

Related: https://github.com/vercel/next.js/issues/19914

[hartshorne]

This would fix the issue: https://github.com/vercel/next.js/compare/canary...hartshorne:preserve-custom-cache-control-headers

But seems to run against the grain of the documentation: https://nextjs.org/docs/api-reference/next.config.js/headers#cache-control

Cache-Control headers set in next.config.js will be overwritten in production to ensure that static assets can be cached effectively. 

@timneutkens reasonable defaults are great, but it seems like there should be a way to easily override something like a Cache-Control header.

[FDiskas]

it works for me on [email protected] - all images served from public directory with cache control header at next.config.js

  async headers() {
    return [
      {
        source: '/:all*(svg|jpg|png)',
        locale: false,
        headers: [
          {
            key: 'Cache-Control',
            value: 'public, max-age=9999999999, must-revalidate',
          }
        ],
      },
    ]
  },

[ramiel]

Any news on this? I have an api endpoint that produces images and I want to cache them on the client. Why can't I specify the desired cache-control headers?

When the documentation says

Cache-Control headers set in next.config.js will be overwritten in production to ensure that static assets can be cached effectively.

makes too many assumptions on the user use-case.

[FDiskas]

@ramiel vote for https://github.com/vercel/next.js/pull/23328 :)

[ramiel]

@FDiskas I voted for that one, but it's not exactly the same issue. That is specific for the images through next/image I guess, this is for anything

[FDiskas]

Take a look at my example in the comments, this works on production as well

[flockonus]

This is a considerable unchangeable default can cause pages to be stale up to 1 years for CDNs, which is bonkers(!) And agree with @ramiel , these are too many assumptions about use case, for a config that can't be changed.

Also note that stale-while-revalidate seems to be used being used incorrectly, as it should have =seconds to it

[schlosser]

Big +1 here – when using next/image with images from the /public folder deployed outside of Vercel, we're getting:

cache-control: public, max-age=0, must-revalidate

No matter what we put in next.config.js. This makes our images load extremely slow. We'd like our CDN to use stale-while-revalidate behavior but we can't get our webserver to respect this setting...

[cweekly]

I'm flabbergasted by:

"Cache-Control headers set in next.config.js will be overwritten in production to ensure that static assets can be cached effectively."

Sane defaults would be one thing. But this is just nuts. "Using next.js means you cannot configure your app's HTTP response headers."

Please, maintainers, provide some way to opt out of this heavy-handed, unpredictable, one-size-fits-some behavior.
Please.

[yarolegovich]

Was quite disappointed to find this out. The plan was to have the following setup for a hosted CMS:

• Content creators have their pages that change infrequently.
• When a page is requested incremental static generation is used and the page is cached.
• When a page is updated path is revalidated. The problem is revalidate_unstable can be used to invalidate the file on the server, but it's still stuck in a CDN.

I'm using CDN-Cache-Control header to overwrite CDN s-maxage, but it's frustrating we can't control resources headers 👎

[jekh]

Is there truly no way to set your own cache-control headers for statically-generated pages? You can set the s-maxage value by remembering to set revalidate in every return statement within every getStaticProps(), but what if you don't want to use the default s-maxage=X, stale-while-revalidate pattern for the cache-control header?

Would a middleware potentially work for getStaticProps pages? Does that fire for all requests for static pages, even when the page is still fresh? If so, will setting response headers still work with the new middleware?

The default for getStaticProps pages seems like quite the footgun: unlike every other page, it automatically adds a very high default s-maxage -- don't forget, or that page could be stuck in your CDN forever!

[imconfused218]

Really frustrating. Seeing bugs with ISR pages that will sometimes default to s-maxage=31536000 instead of using their revalidate page setting. Pages that should be re-generating every 2 minutes are suddenly stuck for a year. Please consider setting more appropriate defaults or allow a cache-control or max-age configuration

[yangxuHBO]

async headers() {
    return [
      {
        source: '/:all*(svg|jpg|png)',
        locale: false,
        headers: [
          {
            key: 'Cache-Control',
            value: 'public, max-age=9999999999, must-revalidate',
          }
        ],
      },
    ]
  },

My nextJS is v11. Did not work for me. Still got some other max-age rather than this one.

[Mihai-github]

Hi, I'm having the same issue with a nex.js website with a static build hosted on S3. Even though I'm setting in the next.config.js file rule for cache-control:

async headers() {
    return [
      {
        source: '/:all*(svg|jpg|png)',
        locale: false,
        headers: [
          {
            key: 'Cache-Control',
            value: 'public, max-age=9999999999, must-revalidate',
          }
        ],
      },
    ]
  },

In network tab I end up with Request Headers cache-control: no-cache

[AdamZaczek]

We've spent a week trying to fix this issue. The app is broken for thousands of CodeAlly users because of this behavior. We use Vercel + Next and migrated from Gatsby. Right now, we can not serve a production fix for our user base because cache is stuck.

We need an option to set the cache header or we can't deliver the product in any shape or form. Please.

[u3u]

Before the #39707 merge, if anyone else is bothered by it, my solution is to modify node_modules/next/dist/server/send-payload/revalidate-headers.js and then use patch-package to generate the patch. If you use pnpm then you can just use pnpm patch next to generate the patch using pnpm patch-commit after making changes in the temporary directory.

[AdamZaczek]

I tried to force revalidation @u3u:

diff --git a/node_modules/next/dist/server/send-payload/revalidate-headers.js b/node_modules/next/dist/server/send-payload/revalidate-headers.js
index 4d9250f..fddad05 100644
--- a/node_modules/next/dist/server/send-payload/revalidate-headers.js
+++ b/node_modules/next/dist/server/send-payload/revalidate-headers.js
@@ -6,15 +6,15 @@ exports.setRevalidateHeaders = setRevalidateHeaders;
 function setRevalidateHeaders(res, options) {
     if (options.private || options.stateful) {
         if (options.private || !res.hasHeader("Cache-Control")) {
-            res.setHeader("Cache-Control", `private, no-cache, no-store, max-age=0, must-revalidate`);
+            res.setHeader("Cache-Control", `private, no-cache, no-store, s-max-age=0, max-age=0, must-revalidate`);
         }
     } else if (typeof options.revalidate === "number") {
         if (options.revalidate < 1) {
             throw new Error(`invariant: invalid Cache-Control duration provided: ${options.revalidate} < 1`);
         }
-        res.setHeader("Cache-Control", `s-maxage=${options.revalidate}, stale-while-revalidate`);
+        res.setHeader("Cache-Control", `s-maxage=${options.revalidate``}, must-revalidate`);
     } else if (options.revalidate === false) {
-        res.setHeader("Cache-Control", `s-maxage=31536000, stale-while-revalidate`);
+        res.setHeader("Cache-Control", `s-maxage=0, max-age=0, must-revalidate`);
     }
 }
 

This among all the other solutions, did not work for me. Could you share the patch file?

[u3u]

@AdamZaczek You should follow the link above to modify the content:

diff --git a/dist/server/send-payload/revalidate-headers.js b/dist/server/send-payload/revalidate-headers.js
index 4d9250fa1902ecdca0a4220df9863082d49aa00a..b554381956007582e7a5fb678b89e2841a8fee16 100644
--- a/dist/server/send-payload/revalidate-headers.js
+++ b/dist/server/send-payload/revalidate-headers.js
@@ -5,16 +5,20 @@ Object.defineProperty(exports, "__esModule", {
 exports.setRevalidateHeaders = setRevalidateHeaders;
 function setRevalidateHeaders(res, options) {
     if (options.private || options.stateful) {
-        if (options.private || !res.hasHeader("Cache-Control")) {
+        if (options.private || !res.getHeader("Cache-Control")) {
             res.setHeader("Cache-Control", `private, no-cache, no-store, max-age=0, must-revalidate`);
         }
     } else if (typeof options.revalidate === "number") {
         if (options.revalidate < 1) {
             throw new Error(`invariant: invalid Cache-Control duration provided: ${options.revalidate} < 1`);
         }
-        res.setHeader("Cache-Control", `s-maxage=${options.revalidate}, stale-while-revalidate`);
+        if (!res.getHeader("Cache-Control")) {
+            res.setHeader("Cache-Control", `s-maxage=${options.revalidate}, stale-while-revalidate`);
+        }
     } else if (options.revalidate === false) {
-        res.setHeader("Cache-Control", `s-maxage=31536000, stale-while-revalidate`);
+        if (!res.getHeader("Cache-Control")) {
+            res.setHeader("Cache-Control", `s-maxage=31536000, stale-while-revalidate`);
+        }
     }
 }
 

Then you can override the Cache-Control headers in next.config.js

/** @type {import('next').NextConfig} */
const nextConfig = {
  headers: async () => {
    return [
      {
        headers: [
          {
            key: 'Cache-Control',
            value: 'private, no-cache, no-store, max-age=0, must-revalidate',
          },
        ],

        source: '/:path*',
      },
      {
        headers: [
          {
            key: 'Cache-Control',
            value: 'public, max-age=31536000, immutable',
          },
        ],

        source: '/:path(.+\\.(?:ico|png|svg|jpg|jpeg|gif|webp|json|js|css|mp3|mp4|ttf|ttc|otf|woff|woff2)$)',
      },
      {
        headers: [
          {
            key: 'Cache-Control',
            value: 'private, no-cache, no-store, max-age=0, must-revalidate',
          },
        ],

        source: '/_next/data/:path*',
      },
      {
        headers: [
          {
            key: 'Cache-Control',
            value: 'private, no-cache, no-store, max-age=0, must-revalidate',
          },
        ],

        source: '/_next/:path(.+\\.(?:json)$)',
      },
    ];
  },
};

[AdamZaczek]

Thank you @u3u! It appears, our problem is not fixed even with your version of the revalidate0headers file but this might help others.

[MartinDavi]

This is a really bad issue - we're trying to use SSG with on demand ISR. We don't want to use the revalidate prop in getStaticProps because we don't need our next.js server to regenerate pages every n number of seconds. Pages will be regenerated automatically on demand when their content changes using on demand ISR.

But because I can't influence the cache-control header in any other way than using the revalidate prop, I'm unable to set a reasonable s-maxage so that our CDN will phone home every once and a while to get the fresher SSG pages.

This is both a complete head scratcher as to why this isn't configurable, and a deal breaker for the workflow we want to use.

edit:

@jekh

Would a middleware potentially work for getStaticProps pages? Does that fire for all requests for static pages, even when the page is still fresh? If so, will setting response headers still work with the new middleware?

No, this doesn't work - I just tried. I'm able to set custom response headers in the middleware, but trying to set Cache-Control header gets overwritten down the line. Great idea, though.

[Rototu]

@u3u Your solution worked perfectly for me. However I still find the fact that patching next.js package is the only method to do overwrite SSG/ISR headers a huge downside of using Next.js.

For multiple clients I've worked with (giant e-commerce clients) this always proved to be a major headache.

In one instance we completely abandoned SSG and just used SSR with caching handled at CDN level.

Encountering same issue with another client now, trying solution above, but it's crazy that this is the only solution available if we want to keep SSG.

Many organisations I've worked with would be much happier to abandon Next.js than to abandon their current CDN and Cloud providers in favour of Vercel.

Therefore, the value added by enabling custom headers therefore would be huge for both users and for the adoption of Next.js, at least from my experience 😃 .

[amannn]

My use case was to modify the cache-control header for SSR requests with the App Router.

Since there's no official way to achieve this and headers set within next.config.js for these paths are ignored, I had no other choice than to set them in revalidate-headers.js:

diff --git a/dist/server/send-payload/revalidate-headers.js b/dist/server/send-payload/revalidate-headers.js
index eae390d4927601057e38e95d6cda615342e9674e..b83df25eebfee4db68049f8bdf267dd0cd22dd2c 100644
--- a/dist/server/send-payload/revalidate-headers.js
+++ b/dist/server/send-payload/revalidate-headers.js
@@ -10,9 +10,14 @@ Object.defineProperty(exports, "setRevalidateHeaders", {
  
 });
  function setRevalidateHeaders(res, options) {
       if (options.private || options.stateful) {
-        if (options.private || !res.getHeader("Cache-Control")) {
-            res.setHeader("Cache-Control", `private, no-cache, no-store, max-age=0, must-revalidate`);
-        }
+        if (options.private) {
+            res.setHeader("Cache-Control", `private, no-cache, no-store, max-age=0, must-revalidate`);
+        } else {
+            // IMPORTANT: This matches all dynamic content routes and applies the same 
+            // caching headers (10min CDN caching with infinite stale-while-revalidate).
+            // You can also use `res.req.url` to match specific routes.
+            res.setHeader("Cache-Control", "public, s-maxage=600, stale-while-revalidate=31557600");
+         }
     } else if (typeof options.revalidate === "number") {
               if (options.revalidate < 1) {
                       throw new Error(`invariant: invalid Cache-Control duration provided: ${options.revalidate} < 1`);

[cdpark0530]

It's very strange to me that Vercel, the mother of Next.js, says two things contradictory to each other. This Vercel page guides how to configure headers in next.config for ISR, while this Next.js document notes that Cache-Control headers in next.config will be overwritten in production.

I applied @u3u's patch, and I saw the header applies well on production mode locally, but this patch creates another issue on Vercel platform, and it wasn't solved in v13.3.1-canary.0 unlike this says.

(I don't know since when but at least) on [email protected], it seems that Cache-Control headers set from headers in next.config are always removed before setRevalidateHeaders is excuted, so res.getHeader("Cache-Control") in the function is always undefined. Nasty workaround would be setting Cache-Control header in different name in next.config and then checks it in setRevalidateHeaders, but surely Vercel/Next.js should give us a way to make on-demand ISR

[matolink]

August 2023 and still nothing, please Vercel! this is a must for many use cases!

[FDiskas]

https://github.com/vercel/next.js/pull/27200

[darl0ck]

@timneutkens Check pls this

[darl0ck]

They break it again in 13.4.13, In 13.4.12 all works fine If i set appDir:false, when using Page Dir, after 13.4.12 this hotfix doesnt work at all (((