backrest icon indicating copy to clipboard operation
backrest copied to clipboard

Send keep-alive messages in streams

Open edvgui opened this issue 7 months ago • 2 comments

Is your feature request related to a problem? Please describe. By default, nginx sets the proxy_connect_timeout to 60s:

Syntax: proxy_read_timeout time;
Default: proxy_read_timeout 60s;
Context: http, server, location

Defines a timeout for reading a response from the proxied server.
The timeout is set only between two successive read operations,
not for the transmission of the whole response.
If the proxied server does not transmit anything within this time,
the connection is closed.

http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout

If I understood correctly, the GetOperationEvents call sets up a stream between the client and the server, and the server will post messages on it whenever something is happening which is worth notifying the client about. If there is nothing interesting, the server doesn't send anything. This means that if nothing happens for more that a minute, no message is sent for more than a minute, and nginx will terminate the connection, resulting in a 504 which the web ui doesn't appreciate too much:

operations stream died with exception:  ConnectError: [unavailable] HTTP 504

It does however recover from it as it starts a new connection right away.

Describe the solution you'd like I think it would be nice to send keep-alive messages at regular intervals (i.e. 30 seconds, to make it work with default nginx settings) to make sure the connection doesn't die.

As a bonus, it would be even better if this keep-alive was configurable (via env vars).

Additional context

  • I have very little familiarity with grpc, I apologize if what I am proposing is an heresy for this protocol.
  • I have very little (no writing experience) with golang, I apologize if I missed any part of the code that might already address this issue in some way.

edvgui avatar Jul 16 '24 16:07 edvgui