Allow accepting new connections during a graceful shutdown

[howardjohn]

Is your feature request related to a problem? Please describe. We want a very specific behavior for graceful shutdowns in our hyper-based proxy. We set two deadlines, min (typically 5-10s), and max (could be much longer). We want the following sequence during shutdown:

• t=0s: send GOAWAY on all HTTP2 connections and start responding with connection: close on all http1 requests. Continue to accept new connections, and handle them the same (GOAWAY/connection: close) them.
• After min deadline: stop accepting new connections. Continue existing connections, with the same GOAWAY/connection: close.
• After max deadline: shutdown everything.

The reason for the time when we accept new connections, but tell clients to go away, is that there is some time for the load balancers in front of the proxy (this could be something like AWS NLB, Kubernetes kube-proxy, etc) to de-register the proxy instance and stop sending traffic to it. So immediately dropping new connections results in downtime. However, during this time its nice to start to encourage clients to goaway, since its likely they will select another proxy instance (either because the fronting LB already de-registered the old instance (the min deadline is intended to be the worst case, so usually its much faster), or because of random chance it selects a different one).

Today, the best we can do is wait until after min deadline and stop accepting new connections and start GOAWAY/connection: close at the same time.

Describe the solution you'd like A new flag on hyper_util::server::conn::auto, and on hyper::server::conn::http1 to enable the "different graceful shutdown mode" (naming TBD!). This would change:

• https://github.com/hyperium/hyper/blob/f9f8f44058745d23fa52abf51b96b61ee7665642/src/proto/h1/dispatch.rs#L97 - skip the close() call
• https://github.com/hyperium/hyper/blob/f9f8f44058745d23fa52abf51b96b61ee7665642/src/proto/h1/conn.rs#L878 - skip the close() call
• https://github.com/hyperium/hyper-util/blob/66afc93debef02548c86e8454e6bc01cf4fca280/src/server/conn/auto/mod.rs#L341 - do no error here
• https://github.com/hyperium/hyper-util/blob/66afc93debef02548c86e8454e6bc01cf4fca280/src/server/conn/auto/mod.rs#L525-L531 -- if cancelled, then call graceful_shutdown() on the newly served connections (and do the same for UpgradeableConnection)

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context

If these changes are acceptable I would be happy to contribute both parts of the change

[seanmonstar]

Just to focus on HTTP/1 at first, this seems already possible, since hyper doesn't get involved with accepting new connections. You have control over when you would like to trigger the graceful_shutdown on any hyper::server::conn. Is there a piece I've missed that prevents you from doing that?

[howardjohn]

@seanmonstar the problem is I want, for any given response on a connection, to return connection: close. However, I do not want to reject a new request on the connection (or at least, I don't want to reject the first one; I don't mind rejecting them if they ignore my connection: close I suppose).

But if I call graceful_shutdown, this ends up calling Dispatcher::disable_keep_alive which calls self.close() if there has not been any data (here https://github.com/hyperium/hyper/blob/f9f8f44058745d23fa52abf51b96b61ee7665642/src/proto/h1/conn.rs#L878 and here https://github.com/hyperium/hyper/blob/f9f8f44058745d23fa52abf51b96b61ee7665642/src/proto/h1/dispatch.rs#L97).

So what I want is really a way to just call State::disable_keep_alive, without the abstractions over it in Dispatcher and Conn which conditionally close the connection as well.