spring-cloud-gateway icon indicating copy to clipboard operation
spring-cloud-gateway copied to clipboard

Connection closed before response - maxHeaderSize configuration not applied in Spring Cloud Gateway (Netty)

Open ArshadAhamedPM opened this issue 2 months ago • 13 comments

Even after explicitly configuring Netty’s maxHeaderSize and maxInitialLineLength in a Spring Cloud Gateway project, large HTTP headers still trigger connection closure errors such as:

reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response

It seems the configured limits are not being applied to the Netty HTTP server inside Spring Cloud Gateway.

What Was Tried

Added in configuration class:

httpServer.httpRequestDecoder(spec -> 
    spec.maxHeaderSize(32 * 1024).maxInitialLineLength(16 * 1024)
);

Tried:

server:
  max-http-header-size: 32768

Tried adding:

.maxHeaderSize(32 * 1024)

and

.maxInitialLineLength(16 * 1024)

on both client and server side.

Despite all of the above, the issue persists — large request headers (~>4KB) still cause premature connection closure.

The configured header size (e.g., 32KB) should be applied to the embedded Netty server and client, allowing the Gateway to handle larger headers gracefully.

Actual Behavior

Gateway still closes the connection when header size exceeds default Netty limit (8KB).

Debug logs show default Netty values, not the overridden ones.

Error stack shows:

ArshadAhamedPM avatar Oct 23 '25 11:10 ArshadAhamedPM

If you'd like us to spend some time investigating, please take the time to provide a complete, minimal, verifiable sample (something that we can unzip attached to this issue or git clone, build, and deploy) that reproduces the problem.

spencergibb avatar Oct 23 '25 14:10 spencergibb

It seems to occur in the client configuration, based on the following log:
reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response.

Does the same exception occur when you set
spring.cloud.gateway.server.webflux.httpclient.max-header-size as well?

ririnto avatar Oct 25 '25 04:10 ririnto

@spencergibb

https://github.com/ArshadAhamedPM/cloudgateway/tree/main/spring-cloud-gateway

When header size is more than 4k. we observed the above error else it is working fine with this code.

ArshadAhamedPM avatar Oct 25 '25 07:10 ArshadAhamedPM

@spencergibb I get this error when the number of headers added in the gateway is more than 100. is there any restriction to add headers in gateway to down services.

ArshadAhamedPM avatar Oct 28 '25 05:10 ArshadAhamedPM

How to disable maxHeader count 100 validation in spring cloud gateway. I want to send more than 100 headers for some request. So is there any config to disable this .

Aravinthp66 avatar Nov 05 '25 14:11 Aravinthp66

@Aravinthp66 @ArshadAhamedPM are all the errors you are seeing specific to netty? There's no max header setting in gateway.

spencergibb avatar Nov 05 '25 14:11 spencergibb

Yes.

Aravinthp66 avatar Nov 05 '25 14:11 Aravinthp66

Deployed on the server: If I send more than 100 headers, it blocks the request and throws the following error

error-1 The connection observed an error, the request cannot be retried as the headers/body were sent

io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer
ERROR [reactor-http-epoll-1] reactor.util.Loggers$Slf4JLogger: Operator called default onErrorDropped
reactor.core.Exceptions$ErrorCallbackNotImplemented:

I also tried following the config in the server (https://github.com/reactor/reactor-netty/issues/1774#issuecomment-908066283)

ConnectionProvider provider = ConnectionProvider.builder("fixed")
			.maxConnections(500)
			.maxIdleTime(Duration.ofSeconds(20))
			.maxLifeTime(Duration.ofSeconds(60))
			.pendingAcquireTimeout(Duration.ofSeconds(60))
			.evictInBackground(Duration.ofSeconds(120)).build();

	this.webClient = WebClient.builder()
			.clientConnector(new ReactorClientHttpConnector(HttpClient.create(provider)))
			.build();

error-2 After adding the config, we were getting the following error

  "reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response"

So we tried to reproduce the error, and we found that more than 100 headers only blocked and threw the error-1

Local: If I send more than 100 headers, Tomcat blocks the request with a proper error

"java.lang.IllegalStateException: More than the maximum allowed number of headers, [100], were detected."

Aravinthp66 avatar Nov 05 '25 15:11 Aravinthp66

The webflux gateway server doesn't use WebClient, create a org.springframework.cloud.gateway.config.HttpClientCustomizer bean and set the provider there.

spencergibb avatar Nov 05 '25 19:11 spencergibb

Could you please provide complete config for fix more than 100 headers blocked by Netty.
I have tried different differents configuration. I could not understand why the request is blocking more than 100 headers.

Aravinthp66 avatar Nov 09 '25 00:11 Aravinthp66

Did you try my previous https://github.com/spring-cloud/spring-cloud-gateway/issues/3960#issuecomment-3492920020 ?

create a org.springframework.cloud.gateway.config.HttpClientCustomizer bean and set the provider there.

spencergibb avatar Nov 09 '25 00:11 spencergibb

Yes, already we have tried this "org.springframework.cloud.gateway.config.HttpClientCustomizer"

httpServer.httpRequestDecoder(spec -> spec.maxHeaderSize(32 * 1024).maxInitialLineLength(16 * 1024) ); https://github.com/spring-cloud/spring-cloud-gateway/issues/3960#issue-3544277213

Aravinthp66 avatar Nov 09 '25 12:11 Aravinthp66

import org.springframework.cloud.gateway.config.HttpClientCustomizer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import reactor.netty.http.client.HttpClient;
import reactor.netty.resources.ConnectionProvider;

import java.time.Duration;
@Configuration
public class NettConfig {
  
  @Bean
  public ConnectionProvider provider() {
    return ConnectionProvider.builder("fixed")
        .maxConnections(500)
        .maxIdleTime(Duration.ofSeconds(20))
        .maxLifeTime(Duration.ofSeconds(60))
        .pendingAcquireTimeout(Duration.ofSeconds(60))
        .evictInBackground(Duration.ofSeconds(120))
        .build();
  }

  @Bean
  public HttpClientCustomizer nettyServerCustomizer(ConnectionProvider provider) {
    return httpClient -> HttpClient.create(provider)
        .httpResponseDecoder(spec -> spec
            .maxHeaderSize(32768)
            .initialBufferSize(256)
            .maxChunkSize(16384)
            .maxInitialLineLength(16384)
            .responseTimeout(Duration.ofSeconds(120)))
        .compress(true)
        .keepAlive(true)
        .followRedirect(true)
        .wiretap(true);
  }
}

Logs:

2025-11-09 16:05:43.680 INFO 24424 --- [parallel-1] c.d.o.s.g.gateway.filter.GlobalFilter : Original request body is...{} 2025-11-09 16:05:43.681 INFO 24424 --- [parallel-1] c.d.o.s.g.gateway.filter.CustomPreFilter : Session Id: 770bed04-6b1a-44b1-8022-********** 2025-11-09 16:05:43.713 INFO 24424 --- [parallel-1] c.d.o.s.g.gateway.filter.CustomPreFilter : Pre filter entered... path: /auth/au********** 2025-11-09 16:05:43.713 INFO 24424 --- [parallel-1] c.d.o.s.g.gateway.filter.DefaultGatewayContext : inside addRequestHeader key: headerde 2025-11-09 16:05:43.713 INFO 24424 --- [parallel-1] c.d.o.s.g.gateway.filter.CustomPreFilter : Skipped the .run method 2025-11-09 16:05:43.713 INFO 24424 --- [parallel-1] c.d.o.s.g.gateway.filter.UrlRewriter : UrlRewriter entered... actual path: ********** 2025-11-09 16:05:43.713 INFO 24424 --- [parallel-1] c.d.o.s.g.gateway.filter.RequestModificationFilter : Headers ********:{'accept-language': ...} 2025-11-09 16:05:43.713 INFO 24424 --- [parallel-1] c.d.o.s.g.gateway.filter.RequestModificationFilter : Header count: 100 2025-11-09 16:05:43.713 INFO 24424 --- [parallel-1] c.d.o.s.g.gateway.filter.RequestModificationFilter : Header size: 4107 2025-11-09 16:05:43.713 INFO 24424 --- [parallel-1] c.d.o.s.g.gateway.filter.RequestModificationFilter : Modify request before: [1a36ab2c-1, L:/10.46.1.133:60822 - R:netty.http.client.HttpClientConnect] 2025-11-09 16:05:44.023 WARN 24424 --- [ctor-http-nio-2] r.netty.http.client.HttpClientConnect : java.net.SocketException: Connection reset at java.base/sun.nio.ch.SocketChannelImpl.throwConnectionReset(SocketChannelImpl.java:394) at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:426) at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:255) at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1132) at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:356) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:151) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:796) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:732) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:658) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)

Aravinthp66 avatar Nov 09 '25 13:11 Aravinthp66