envoy icon indicating copy to clipboard operation
envoy copied to clipboard

quic: Connect client UDP sockets before usage

Open abeyad opened this issue 1 year ago • 14 comments

This change does a few things:

  1. Calls connect() on the QUIC UDP sockets, which allows the connection to receive async ICMP error messages on recvmsg calls throughout the lifetime of the connection.
  2. Does not call bind() on a QUIC client UDP socket unless the local address is specified.
  3. If the socket is already connected, call writev/send instead of sendmsg. Connected sockets should not specify a peer address and hence, writev/send is more appropriate.
  4. Previously, when EnvoyQuicClientConnection did preferred server address probing, it specified the existing remote address, not the new preferred server address. This still worked because the sendmsg call took in the actual destination IP address. However, when calling connect(), this caused the socket to connect to the wrong address. This issue is fixed in this PR by creating a socket connection to the preferred server address.
  5. Uses the loopback address for the local address, if one isn't specified, as it is only needed in createConnectionSocket to get an IoHandle. This avoids having to make the more expensive getifaddrs call, as mentioned in https://github.com/envoyproxy/envoy/issues/35137.
  6. The connect() behavior is guarded by the runtime guard envoy.reloadable_features.quic_connect_client_udp_sockets, defaulted to true.

Fixes #35137

abeyad avatar Aug 09 '24 20:08 abeyad

  1. Calls connect() on the QUIC UDP sockets, which allows the socket to know if the peer address is routable earlier.

Not really, we saw connect() succeeded but following write cause "no route to host". The reason we want to connect() here is that connected UDP socket will surface async socket error received from ICMP messages.

danzh2010 avatar Aug 09 '24 21:08 danzh2010

/assign @danzh2010

abeyad avatar Aug 19 '24 22:08 abeyad

Alyssa is OOO this week. Assigning Ryan who has more QuIC context. /assign @RyanTheOptimist

adisuissa avatar Aug 20 '24 14:08 adisuissa

@danzh2010 I pushed up new changes that I believe fixes all the tests. PTAL!

abeyad avatar Aug 22 '24 22:08 abeyad

/retest

abeyad avatar Aug 23 '24 21:08 abeyad

/retest

abeyad avatar Aug 24 '24 01:08 abeyad

/retest

abeyad avatar Aug 27 '24 00:08 abeyad

@danzh2010 friendly ping, thanks!

abeyad avatar Aug 27 '24 17:08 abeyad

/retest

abeyad avatar Aug 28 '24 04:08 abeyad

/retest

abeyad avatar Aug 28 '24 13:08 abeyad

  1. Calls connect() on the QUIC UDP sockets, which allows the connection to know if the socket is not routable upon the first recvmsg call via an async ICMP message.

"No route to host" can be reported synchronously during connect() as we observed in the tests. But we want connected UDP socket overall because it surfaces any async errors (from ICMP messages) via following recvmsg through out the connection life time.

danzh2010 avatar Aug 28 '24 15:08 danzh2010

  1. Calls connect() on the QUIC UDP sockets, which allows the connection to know if the socket is not routable upon the first recvmsg call via an async ICMP message.

"No route to host" can be reported synchronously during connect() as we observed in the tests. But we want connected UDP socket overall because it surfaces any async errors (from ICMP messages) via following recvmsg through out the connection life time.

Updated the PR description

abeyad avatar Aug 28 '24 17:08 abeyad

CC @envoyproxy/runtime-guard-changes: FYI only for changes made to (source/common/runtime/runtime_features.cc).

:cat:

Caused by: https://github.com/envoyproxy/envoy/pull/35652 was synchronize by abeyad.

see: more, trace.

/retest

abeyad avatar Aug 29 '24 13:08 abeyad

Thanks for the great reviews, @danzh2010 and @RyanTheOptimist !

abeyad avatar Aug 30 '24 15:08 abeyad