kit SvelteKit generates large Link header, causing a 502 with Nginx

Describe the bug

When the app grows, the Link header becomes bigger and bigger.

At a certain point, nginx will display 502s and error. This happens when the Link header nears 4kb.

It's possible to work around this by tweaking nginx config, but this error is hard to diagnose and it shouldn't happen out of the blue with the most popular reverse proxy.

Reproduction

In the handle hook, you can add this to reproduce it happening:

  response.headers.set("Link", '<../_app/immutable/assets/0.ff6c1858.css>; rel="preload";as="style"; nopush, <../_app/immutable/assets/Picture.c1d98be5.css>; rel="preload";as="style"; nopush, <../_app/immutable/assets/TagWidget.72afa9c1.css>; rel="preload";as="style"; nopush, <../_app/immutable/entry/start.0dc7e3e4.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/scheduler.988dae19.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/singletons.e5596c1f.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/index.a60158c4.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/parse.bee59afc.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/control.f5b05b5f.js>; rel="modulepreload"; nopush, <../_app/immutable/entry/app.1da9b740.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/index.1bccfb63.js>; rel="modulepreload"; nopush, <../_app/immutable/nodes/0.bd51b068.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/stores.d576e56f.js>; rel="modulepreload"; nopush, <../_app/immutable/nodes/2.ef69061c.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/each.b3fec9fd.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/spread.8a54911c.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/PriceTag.75c37345.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/Currency.f4aa72d9.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/navigation.d8b65ae8.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/UrlDependency.890b5d7a.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/Popup.8fb9ab92.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/Picture.0211812c.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/i18n.778889ac.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/get.281b793c.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/isObject.a95a320a.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/sumCurrency.70129a10.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/forms.95e660af.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/CartQuantity.2e07a451.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/Product.1a4e99bd.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/IconTrash.cae74cb4.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/menu-outlined.dac218ab.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/index.31109bbe.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/index.40b40403.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/fixCurrencyRounding.23ca317a.js>; rel="modulepreload"; nopush, <../_app/immutable/nodes/73.490ea325.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/globals.7f7f1b26.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/IconInfo.a1558468.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/User.f841f874.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/CmsDesign.9bb48773.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/ChallengeWidget.b02d14bf.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/Trans.03e3bf42.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/CarouselWidget.4a977ec0.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/bundle-mjs.312aa90d.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/ProductType.b9015098.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/typedKeys.2a0d164f.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/TagWidget.9ff1cfe2.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/hasIn.9eb1248f.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/index.c336cdc8.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/index.65d14ac1.js>; rel="modulepreload"; nopush, <../_app/immutable/chunks/index.ac617c56.js>; rel="modulepreload"; nopush');

Then if sveltekit is running behind nginx with default config, it will send a 502.

Logs

Nginx error log:

2023/11/20 01:18:04 [error] 788760#788760: *5288880 upstream sent too big header while reading response header from upstream, client: 145.224.88.62, server: XXX, request: "GET /XXX HTTP/2.0", upstream: "http://127.0.0.1:3000/XXX", host: "XXX"

System Info

System:
    OS: Linux 5.15 Ubuntu 22.04.2 LTS 22.04.2 LTS (Jammy Jellyfish)
    CPU: (2) x64 Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
    Memory: 2.62 GB / 7.76 GB
    Container: Yes
    Shell: 5.1.16 - /bin/bash
  Binaries:
    Node: 18.16.1 - /usr/local/bin/node
    npm: 9.5.1 - /usr/local/bin/npm
    pnpm: 8.3.1 - /usr/local/bin/pnpm
  npmPackages:
    @sveltejs/adapter-node: ^1.2.3 => 1.2.3 
    @sveltejs/kit: ^1.27.3 => 1.27.3 
    svelte: ^4.2.2 => 4.2.2 
    vite: ^4.2.1 => 4.2.1

Severity

serious, but I can work around it

Additional Information

This can be fixed on Nginx side with this:

proxy_busy_buffers_size   512k;
proxy_buffers   4 512k;
proxy_buffer_size   256k;

Still, this error is hard to debug and it shouldn't happen with the most popular reverse proxy

Nov 20 '23 01:11 coyotte508

I don't know what we could reasonably do about this besides document it. I don't think SvelteKit should drop down its maximum size of this header because you might have nginx in front of it with the default configuration. We can't in general tell anything about limitations of the proxy sitting in front of the application.

Nov 20 '23 03:11 Conduitry

For the nginx noobs like me, to view the error logs you need to run:

sudo tail -f /var/log/nginx/error.log

If that doesn't work, maybe your error logs are located somewhere else:

nginx -T | grep 'log'
# this comands highlights the 'log' word in your entire nginx config, so you can see where they are located

Feb 16 '24 11:02 Antoine-lb

SvelteKit cannot be placed behind Cloudfront once the header has reached Cloudfront's limit.

See documentation here about OriginHeaderTooBigError: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/AccessLogs.html

Feels very critical to me as turning off Cloudfront puts a strain on our servers

Apr 16 '24 17:04 http-teapot

@http-teapot note that another fix is in your app code, you can create a hooks.server.js with a handle hook that does:

/** @type {import('@sveltejs/kit').Handle} */
export async function handle({ event, resolve }) {
	const response = await resolve(event);
	response.headers.delete('Link');
	return response;
}

Apr 17 '24 07:04 coyotte508

I don't know what we could reasonably do about this besides document it. I don't think SvelteKit should drop down its maximum size of this header because you might have nginx in front of it with the default configuration. We can't in general tell anything about limitations of the proxy sitting in front of the application.

I do think it's reasonable to consider the state of the internet infrastructure when making design decisions though and at least document it as a concern.

IIS and Apache also have default limitations you should keep in mind.

Jul 06 '24 22:07 dboundz

@http-teapot note that another fix is in your app code, you can create a hooks.server.js with a handle hook that does:
/** @type {import('@sveltejs/kit').Handle} */
export async function handle({ event, resolve }) {
	const response = await resolve(event);
	response.headers.delete('Link');
	return response;
}

Any idea what the implications of this are?

Jul 06 '24 22:07 dboundz

For people using nginx ingress in k8s: There is an entry in the docs about JWT's becoming too large which basically results in the same issue because they're also transmitted via headers. Therefore one can use the same solution: https://kubernetes.github.io/ingress-nginx/examples/customization/jwt

TLDR, add this to the helm chart values:

controller:
  config: 
    proxy-buffer-size: 16k

Aug 14 '24 07:08 swenzel-arc

@http-teapot note that another fix is in your app code, you can create a hooks.server.js with a handle hook that does:
/** @type {import('@sveltejs/kit').Handle} */
export async function handle({ event, resolve }) {
	const response = await resolve(event);
	response.headers.delete('Link');
	return response;
}
Any idea what the implications of this are?

Judging from the MDN docs, with the link-header, you can get reduced page loading times because the browser can start fetching relevant resources slightly earlier. But apparently at this point in time that's not true for all link types:

In practice, most link types don't have an effect in the HTTP header. For example, the icon relation only works in HTML, and stylesheet does not work reliably across browsers (only in Firefox). The only relations that work reliably are preconnect and preload, which can be combined with 103 Early Hints.

So, I'm not an expert in this field but sounds like the implications are negligible. However, I could also imagine that support for the link header might increase over time, especially if sites are actually providing it, so keeping the header instead of dropping it might still be a good idea.

Aug 14 '24 08:08 swenzel-arc