Jetty 12 - HttpClientTransport network "modes"

[sbordet]

Jetty version(s) 12+

Description With support for UnixDomain sockets and QUIC, ClientConnector has now become too coarse, because it won't be possible to setup a HttpClientTransportDynamic for "mixed" transports such as HTTP/1.1 over UnixDomain, HTTP/2 over TCP and HTTP/3 over QUIC.

There is the need for a better abstraction than ClientConnector (or improve it), so that the above combination would be possible.

[sbordet]

@lorban thoughts?

[cowwoc]

@sbordet Is this issue blocked on @lorban replying? It's been a year...

Is this issue fixable in Jetty 12 or does it require a design-level change that can only come in Jetty 13?

[sbordet]

Cannot be fixed in Jetty 12, could be in Jetty 13. This work would require a substantial rewrite of the low level code for the client, so it is a moderately heavy task with likely non-backward compatible changes.

[cowwoc]

@sbordet Gah, that's sad :(

Okay, please confirm whether my understanding is correct then... In Jetty 12, the server should be able to bind all cleartext protocols on one port, and all encrypted protocols on a second port. This includes binding HTTP/2 and HTTP/3 to the same port, even though they require different connectors, because one binds to TCP/IP and the other to UDP.

But on the client-side, there is no way to unify HTTP/2 and HTTP/3 support under a single HttpClient object. One has to use one HttpClient for all protocols excluding HTTP/3 and a second one specifically for HTTP/3 client requests.

Is that correct?

If so, then it's not really a big deal because my customers can still use interact with my server over HTTP/3. This just limits my unit tests and the server's outgoing connections to HTTP/2, which isn't the end of the world. I hope that's correct.

Thank you.

(so HTTP/2 and HTTP/3 encrypted connectors can be bound to the same port) but the same

[sbordet]

In Jetty 12, the server should be able to bind all cleartext protocols on one port, and all encrypted protocols on a second port. This includes binding HTTP/2 and HTTP/3 to the same port, even though they require different connectors, because one binds to TCP/IP and the other to UDP.

Correct.

But on the client-side, there is no way to unify HTTP/2 and HTTP/3 support under a single HttpClient object. One has to use one HttpClient for all protocols excluding HTTP/3 and a second one specifically for HTTP/3 client requests.

Correct.

The problem with the client is that a "normal" HTTP request has to be associated with a "transport layer" (for lack of better terminology).

This transport layer can be:

• TCP sockets
• UnixDomain sockets
• UDP sockets
• "local" sockets (not yet implemented but request in #10080)

All of that is complicated by the fact that the transport layer can be upgraded to a different protocol.

The client is way more complicated than a server is, and the current situation is the result of the history: started with HTTP/1.1, then came HTTP/2, then UnixDomain socket and finally QUIC. It was natural to design abstractions at the time with support for just HTTP/1 over TCP; then try to fit the same abstractions for HTTP/2; then try to fit in UnixDomain, which is identical at one level, but differs at another; and then try to fit in QUIC which uses UDP, kind of identical at one level, but differs at another (and not the same as UnixDomain).

Now imagine UDP over UnixDomain; and a local transport where a request is immediately passed to the server for example without even serializing it to bytes and parsing it.

I know, all excuses, but we have not had time to sit down and re-design the lower layers for the client. HTTP/3 is about 1.5 years old, and before that designing support for UDP would have been dubbed as over-engineering 😃

[sbordet]

graph TB;

TCP --- IP
UDP --- IP
TCP --- UNIX
UDP --- UNIX
TLS --- TCP
TLS ---- LOCAL
QUIC --- UDP
H1 --- TCP
H1 --- TLS
H1 --- QUIC
H1 --- LOCAL
H2 --- TCP
H2 --- TLS
H2 --- QUIC
H2 --- LOCAL
H3 --- QUIC

[sbordet]

For the client, at connect time, we need two data: the "logical" address (e.g. backend-server-1) to be used in URIs (e.g. the Host header in H1, or :authority field in H2), and the "mode" (derived from TCP, UDP, IP, UNIX or LOCAL). Both must be part of the key for the Destination (in addition to other data), as I may want to connect to the same "logical" server in different ways.

The implementation must create either a SocketChannel (TCP) or a DatagramChannel (UDP), either over IP or UNIX, or it must fake this part to return a "local" equivalent. All of these are eventually wrapped into a Jetty EndPoint, so the layers above only need to depend on EndPoint.

The combo [ TCP|UDP / IP|UNIX] can share a SelectorManager, while LOCAL does not need it.

The current API uses SocketAddress, but that is not enough since it cannot differentiate between TCP and UDP (although can differentiate between IP and UNIX via instanceof).

The current API for UNIX only supports 1 server (yuck!) because the UNIX path is specified at the ClientConnector level.

Proposal

A proposal to abstract the combo [ TCP | UDP | IP | UNIX | LOCAL] would be to have an interface Mode to create EndPoint and implementations such as TCPMode which would establish the use of SocketChannel and would contain a SocketAddress from which we could derive whether it's IP or UNIX. Similarly UDPMode establishes DatagramChannel and has an SocketAddress for IP or UNIX. And LocalMode would just create a ByteArrayEndPoint.

Evolution

For SCTP for example, we would have SCTPMode, using com.sun.nio.sctp.SctpChannel.

For the lowest level there can be IPX or APPLETALK or ISDN, etc. (see socket.h), and the contract would be the same: return an EndPoint wrapping whatever combination of the lowest 2 levels is desired.