jetty.project
jetty.project copied to clipboard
jetty
jetty traffic ends up in big tcp payloads
Jetty version 9.4.44.v20210927 Java version java version "11.0.3" 2019-04-16 LTS Question Hello! We are using jetty client in our project for http2 traffic. During one of stress tests we could observe that after reaching some TPS TCP frames containing http2 traffic grows quite a lot (see the screenshot). As a consequence there's no traffic received on the server side - here I suspect that some network elements may drop big frames - I can see the Don't Fragment bit set on IP level. With lower traffic it does not happen - the packets are withing the MTU size that is 1500
Does jetty client has any settings that I could tweak to prevent this to happen?
Thank you a lot for your help!
I suspect that some network elements may drop big frames - I can see the Don't Fragment bit set on IP level
You should see retransmits on the client then. Do you see these?
During load tests this scenario is quite common, and can be tuned with HTTP2Session.setWriteThreshold(int), which is set with the value from HttpConfiguration.getOutputBufferSize() on the server, but it's left at default (32 KiB) on the client.
We would need a new property, perhaps HTTP2Client.outputBufferSize and then pass that configuration to HTTP2Session.setWriteThreshold().
Would you like to prepare a PR for the jetty-10.0.x branch?
Thank you for a quick response! Let me try to get full wireshark recording and analyse it. I will come back with the feedback
Hello, there are no transmits in pcap file. Can it be related to some network configuration?
Have I understood HTTP2Session.setWriteThreshold(int) correctly - let's say I set it to 1000 - it would mean that after reaching 1000 bytes in a buffer a TCP write will be performed?
It means that if you exceed 1000 bytes a TCP write will be performed -- it's a threshold, not a strict value.
The default frame size for HTTP/2 is 16 KiB, so if an application sends a single DATA frame of, say, 12 KiB, that will not be split by Jetty. It will be beyond the threshold of 1000, so Jetty won't aggregate other frames in that write, but it's still a 12 KiB write.
Have to say that trying to fix network misconfigurations in the application is a rabbit hole you may never exit from.
I understand. Let me experiment a bit with the property that you have suggested.
I didn't have a time to actually get working on that. I guess we can close it and if the issue pops up again (it was really a corner case connected with some network setup) we can open a new issue and linked that to it. What do you think?
@Michal-Cho yes, open a new issue and link this one if you encounter the same problems.