Generated .js files 404 and cause outages every deployment

[georgesaliba]

Environment

nitro: { preset: 'firebase', firebase: { gen: 2 } }

Reproduction

Unfortunately I believe this issue likely only repros in production environments that leverage caching. I've tried to outline the high-level repro setup in the bug description.

Describe the bug

(Apologies in advance if this isn't the right place to request this, but I believe this is a Nitro issue and not a Nuxt issue.)

I'm running Nuxt 3 on Firebase, using the Nitro firebase preset. Whenever I deploy my site to production, it causes an outage. After digging into the issue, I've found the that .js files are returning 404 immediately after the deployment, and that 404 response is being cached for long periods (in both Cloudflare and user browsers). I have a hypothesis on what is happening:

1. When I deploy the site, the new pages are referencing .js files with new build hashes.
2. I believe that immediately after this deployment completes, the new Firebase cloud functions likely aren't fully spun up yet and/or the old functions are still taking some of the traffic. So it's the previously deployed Firebase cloud functions that receive & handle the new requests.
3. Because those old functions don't understand the new build hash, they're returning a 404 response when the new page is requesting the new build hash .js files. (I can see the 404 in the response body when trying to load the .js files with the new build hash directly in the browser; screenshot in comments).
4. That .js file with the 404 response is then being cached on the client browser.

Now, I think the workaround to this would be to never cache any of the .js files. But that's a useful performance optimization (and potentially cost prohibitive to disable). I think the right solution is for Nitro to remove the cache control header for any 4XX (and possibly 5XX) response, and to add a no-cache header in this scenario.

Additional context

No response

Logs

No response

[georgesaliba]

This is the response I see when I hit the .js file with the new build hash post-deployment, and before clearing my local cache. After clearing the cache, the new .js file content is loaded.

P.S. This issue seems similar: https://github.com/unjs/nitro/issues/1001 - but I'm wondering if it's enough to just remove the cache control header, or if it's better to also add the no-cache header.

[Hebilicious]

This is the response I see when I hit the .js file with the new build hash post-deployment, and before clearing my local cache. After clearing the cache, the new .js file content is loaded.

P.S. This issue seems similar: #1001 - but I'm wondering if it's enough to just remove the cache control header, or if it's better to also add the no-cache header.

I believe at some point we had one change there that never made it into main : d3d0824 (#989)

Something similar might be necessary to be applied to several presets.

[georgesaliba]

Just hit this again while trying to deploy this evening (it essentially causes an outage every deployment). And yeah, can confirm that I'm seeing the Cache-control headers coming through despite the 404 response, which the CDN is interpreting to mean the 404 response should be cached. This issue probably affects every preset/any site that has a layer of caching on top of it (e.g., I'm using Firebase hosting with Cloudflare on top for caching, not hosting with Cloudflare - but it's the same issue).

[georgesaliba]

Unfortunately I don't know how to resolve this myself, but if anyone is familiar with the Nitro codecase, I have a suspicion that the fix would be to update the logic around here to not add the cache control header (and/or remove it and/or send a no-cache header) when the response code is 404: https://github.com/unjs/nitro/blob/main/src/runtime/cache.ts#L346

[andresilva-cc]

If anyone is still getting this problem, just add this plugin to Nitro:

export default defineNitroPlugin(nitroApp => {
  nitroApp.hooks.hook('error', (error, { event }) => {
    if (event && (error as any).statusCode === 404) {
      setHeader(event, 'cache-control', 'no-cache')
    }
  })
})

[pi0]

moving to #2447