Encounter max dynamic table size change required when header table size is zero

[terryyrliang]

Jetty version(s) Jetty 12.0.6

Jetty Environment ee10

Java version/vendor (use: java -version) 17.0.8.1

OS type/version suse-150400.3.30.1-x8664, Linux kernel is 5.14.21-150400.24.63-default

Description I use a http2 client vertx which is based on the Netty, and we configure the header table size as zero but receive an invalid packet, our client reports below error.

io.netty.handler.codec.http2.Http2Exception$StacklessHttp2Exception: HPACK - max dynamic table size change required
	at io.netty.handler.codec.http2.HpackDecoder.decode(..)(Unknown Source)

And below is the error log in Jetty server

2024-04-29 07:27:06.759:DEBUG:oejh.HTTP2Session:qtp1076835071-60: Received GoAwayFrame@5335bfdd{2147483647/compression_error/HPACK - max dynamic table size change required} on HTTP2ServerSession@243d0e3{local:/127.0.0.1:8080,remote:/127.0.0.1:39990,sendWindow=65184,recvWindow=1048576,state=[streams=0,NOT_CLOSED,goAwayRecv=null,goAwaySent=null,failure=null]}

How to reproduce?

1. Create a maven project and import the vertx dependency as below. And run below java program to send a http2 request to Jetty server 12.0.6 with http2 enabled.

<dependency>
    <groupId>io.vertx</groupId>
    <artifactId>vertx-web-client</artifactId>
    <version>4.5.7</version>
</dependency>
<dependency>
    <groupId>io.vertx</groupId>
    <artifactId>vertx-core</artifactId>
    <version>4.5.7</version>
</dependency>

import io.vertx.core.Vertx;
import io.vertx.core.http.Http2Settings;
import io.vertx.core.http.HttpVersion;
import io.vertx.ext.web.client.WebClient;
import io.vertx.ext.web.client.WebClientOptions;


public class H2cClientTest {
    public static void main(String[] args) {
        Http2Settings http2Settings = new Http2Settings();
        http2Settings.setHeaderTableSize(0);
        WebClientOptions options = new WebClientOptions()
                .setProtocolVersion(HttpVersion.HTTP_2)
                .setInitialSettings(http2Settings)
                .setHttp2ClearTextUpgrade(false)
                .setUserAgent("My-App/1.2.3");
        options.setKeepAlive(false);
        WebClient client = WebClient.create(Vertx.vertx(), options);
        client.get(8080, "localhost", "/test")
                .send()
                .onSuccess(response -> {
                    System.out.println(response.body().toString());
                    response.headers().entries().forEach(i -> System.out.println(String.format("%s = %s", i.getKey(), i.getValue())));
                })
                .onFailure(Throwable::printStackTrace);
    }
}

[lorban]

The HPACK - max dynamic table size change required is reported by a class that seems to be part of Netty, and I see no trace of Jetty's involvement.

Aren't you mixing up the two projects?

[terryyrliang]

The HPACK - max dynamic table size change required is reported by a class that seems to be part of Netty, and I see no trace of Jetty's involvement.

Aren't you mixing up the two projects?

Thanks @lorban replying, and I agree with you that it might the error is from Netty. The background is that we are trying to uplift our Jetty from 9.4.39 (Please don't mind the version is old ^_^) to 12.0.6 , and our client keep the same during the uplift. However, we got error after uplifting to Jetty 12.0.6, high appreciate you can pick that version for comparison.

[lorban]

Oh okay, got it, and sorry for the confusion. Indeed it looks like your Netty client doesn't like something about H2 so it aborts its connection by sending Jetty a GoAway frame.

The quickest and easiest way to arbitrate between the client and the server to figure out which side is wrong would be to capture a sniffer trace of such conversation.

Would you be able to do that and post it here?

[terryyrliang]

Oh okay, got it, and sorry for the confusion. Indeed it looks like your Netty client doesn't like something about H2 so it aborts its connection by sending Jetty a GoAway frame.

The quickest and easiest way to arbitrate between the client and the server to figure out which side is wrong would be to capture a sniffer trace of such conversation.

Would you be able to do that and post it here?

Again, thanks your patience @lorban . And please find the tcpdump enclosed and I name the file with some key words which is used to identify the test case or test purpose. jetty-hpack.zip

And I compare the frame and finally found the version 12.0.6 misses a header "Header table size update"

Also, I find a reference about the error COMPRESSION_ERROR reported by Netty from https://www.rfc-editor.org/rfc/rfc9113#name-compression-state

An endpoint MUST treat a field block that follows an acknowledgment of the reduction to the maximum dynamic table size as a [connection error](https://www.rfc-editor.org/rfc/rfc9113#ConnectionErrorHandler) ([Section 5.4.1](https://www.rfc-editor.org/rfc/rfc9113#ConnectionErrorHandler)) of type [COMPRESSION_ERROR](https://www.rfc-editor.org/rfc/rfc9113#COMPRESSION_ERROR) if it does not start with a conformant Dynamic Table Size Update instruction.

Another reference is how the peer side to ack the dynamic table size change is here https://httpwg.org/specs/rfc7541.html#maximum.table.size

This dynamic table size update MUST occur at the beginning of the first header block following the change to the dynamic table size. 

[sbordet]

@terryyrliang it's not clear where you configure the table size to 0: client or server?

My take is that if you configure the max table size to 0 on server (Jetty), then Jetty sends a SETTINGS frame with SETTINGS_HEADER_TABLE_SIZE=0 at connection establishment.

It is that the case?

@lachlan-roberts thoughts?

[terryyrliang]

@terryyrliang it's not clear where you configure the table size to 0: client or server?

My take is that if you configure the max table size to 0 on server (Jetty), then Jetty sends a SETTINGS frame with SETTINGS_HEADER_TABLE_SIZE=0 at connection establishment.

It is that the case?

@lachlan-roberts thoughts?

Hello @sbordet , I already provide how to reproduce in the case description, we set the table size to 0 in the client, and according to https://httpwg.org/specs/rfc7541.html#maximum.table.size, there is an negotiation at the beginning when establish the connection with HTTP2 settings and should use the smaller one afterward.

[lachlan-roberts]

@terryyrliang can you please test again with the latest version of Jetty 12

I believe this was already fixed (in https://github.com/jetty/jetty.project/pull/11452), and I cannot reproduce the issue with your example code in jetty-12.0.7+.

[terryyrliang]

@terryyrliang can you please test again with the latest version of Jetty 12

I believe this was already fixed (in #11452), and I cannot reproduce the issue with your example code in jetty-12.0.7+.

Thanks @lachlan-roberts 's reply, I retest with Jetty 12.0.8, but still got the same issue, from the tcpdump, no header table size update returned by Jetty (You can check the Jetty 9.x tcpdump enclosed in my previous reply). Meanwhile, I realize the https://github.com/jetty/jetty.project/pull/11452 is not apply in branch 12.x but only in 10.x. Please feel free to voice out if any further test you require on my side. Thanks.

[terryyrliang]

@terryyrliang can you please test again with the latest version of Jetty 12

I believe this was already fixed (in #11452), and I cannot reproduce the issue with your example code in jetty-12.0.7+.

@lachlan-roberts Apologize that I made some mistake just now, I startup the wrong version of Jetty, let me double confirm and update you later.

[terryyrliang]

Hi @lachlan-roberts , after several round test, I confirm that the issue is fixed in Jetty 12.0.8 already. Thanks a lot @sbordet @lorban and @lachlan-roberts great and strong support.

[sbordet]

Thanks for the feedback!