quinn
quinn copied to clipboard
Connection stuck when `sendmsg` call completes with`EPIPE` on `Darwin`.
Steps to reproduce
- Use
quinnto establishquicconnection to remote server in context ofiOSapplication via code like this (only UDP socket configuration provided):
let bind_addr = if addr.is_ipv4() {
SocketAddr::new(IpAddr::V4(Ipv4Addr::UNSPECIFIED), 0)
} else {
SocketAddr::new(IpAddr::V6(Ipv6Addr::UNSPECIFIED), 0)
};
let socket = tokio::net::UdpSocket::bind(bind_addr).await?;
let local_addr = socket.local_addr()?;
let runtime = quinn::TokioRuntime;
let endpoint = quinn::Endpoint::new(
quinn_endpoint_config,
None,
socket.into_std()?,
Arc::new(runtime),
)?;
- Minimize
iOSapplication and lock the phone screen while previously established quic connection was alive; - Wait several minutes in background (not sure about exact timing, maybe only important to app being actually suspended by iOS);
- Resume
iOSapplication and try to use previous quic connection such thatquinnsend some packets.
Actual result
quinnis unable to send any UDP packet over previously constructed UDP socket. It receives "Broken Pipe" error fromsendmsg; Non of the IP packets leave the device;- Idle timer on local side is reset before
sendmsgis called and failed, prolonging automatic detection of socket broken; quinnconnection is closed by local side once idle timeout is reached.
Expected result
- There is some quick way to know that underlying socket is broken or whole
quinnconnection is not usable anymore. - Maybe there is an kind of event that UDP socket rotation is required, so user code can listen/wait to it and call
Endpoint::rebindonce problem with socket detected. - Maybe additional API from
Connectionis exposed, like explicitpingmethod which can returnio::Errorfrom underlying UDP socket and user code can trigger rebind or completeEndpoint/Connection. - Idle timer preferably should not restart when
sendmsgcall failed.
More details
I've tracked down EPIPE error from sendmsg inside XNU kernel.
The backtrace to likely origin from XNU source code:
It look like the socket state contains SOF_DEFUNCT or SS_CANTSENDMORE so it's practically become unusable.
According to FreeBSD documentation EPIPE is also indicator of "you can not send more data via provided socket"
See: https://man.freebsd.org/cgi/man.cgi?send(2)
[EPIPE] The socket is unable to send anymore data (SBS_CANTSENDMORE has been set on the socket). This typically means that the socket is not connected.
UPD: The same "broken" state of socket produces ENOTCONN error when called with recvmsg, the origin of it seems to be this check in function soreceive.
UPD 2: Seems ENOTCONN should also have some special treatment on Darwin https://github.com/libevent/libevent/pull/1031/files