grpc-rust icon indicating copy to clipboard operation
grpc-rust copied to clipboard

GRPC Client on windows doesn't close TCP connection on drop.

Open geomaniac opened this issue 6 years ago • 3 comments

On Windows the GRPC client doesn't close its opened TCP connections on drop, only after the client process ends.

Trying to reproduce this, I created a slightly modified version of the hello world example client code. Here a new GRPC client is created once per iteration in a loop, sends a HelloRequest and is dropped at the end.

The modified example can be found at https://github.com/geomaniac/hello_world_grpc. The modified client code is in client.rs.

Now, starting the server and letting the client iterate three times, we can see the following on Windows or Linux.

On Windows

Before first iteration:

$ netstat -an | grep "50051"
  TCP    [::]:50051             [::]:0      LISTENING

After three iterations:

$ netstat -an | grep "50051"
  TCP    [::]:50051             [::]:0       LISTENING
  TCP    [::1]:50051            [::1]:64100  ESTABLISHED
  TCP    [::1]:50051            [::1]:64101  ESTABLISHED
  TCP    [::1]:50051            [::1]:64102  ESTABLISHED
  TCP    [::1]:64100            [::1]:50051  ESTABLISHED
  TCP    [::1]:64101            [::1]:50051  ESTABLISHED
  TCP    [::1]:64102            [::1]:50051  ESTABLISHED

The "ESTABLISHED" connections will only end, if we exit the client process. Otherwise they will remain open forever and new iterations create additional connections.

On Linux

Before first iteration:

$ netstat -a | grep "50051"
tcp6       0      0 [::]:50051              [::]:*                  LISTEN

After three iterations:

$ netstat -a | grep "50051"
tcp        0      0 localhost:52062         localhost:50051         TIME_WAIT  
tcp        0      0 localhost:52066         localhost:50051         TIME_WAIT  
tcp        0      0 localhost:52064         localhost:50051         TIME_WAIT  
tcp6       0      0 [::]:50051              [::]:*                  LISTEN     

The "TIME_WAIT" connections will end after about 60 seconds.

I would expect that the GRPC client also closes its connections on Windows, when it is dropped.

geomaniac avatar Mar 23 '18 13:03 geomaniac

Just tried with v0.3.0, but it shows the same behavior.

geomaniac avatar Apr 23 '18 13:04 geomaniac

Did some further digging and I think this might be related to carllerche/mio#776.

If I activate env_logger with RUST_LOG=trace, the last thing that gets logged after the client is dropped is

"TRACE 2018-06-27T07:04:55Z: mio::sys::windows::tcp: cancelling active TCP read"

(from mio/sys/windows/tcp.rs#L637).

Not sure if there is anything that can be done from grpc-rust (or rather rust-http2) to circumvent this issue.

geomaniac avatar Jun 27 '18 07:06 geomaniac

tokio 0.3 has been released and it includes mio 0.7. Updating tokio (and mio) might help fix issue.

oherrala avatar Oct 16 '20 09:10 oherrala