xds-relay icon indicating copy to clipboard operation
xds-relay copied to clipboard

Introduce upstream stream send/recv min/max timeouts

Open jessicayuen opened this issue 4 years ago • 0 comments

WIP/POC.

  • Add concept of upstream gRPC stream Send / Recv timeouts.
  • Send timeouts (from the proto):
      // Optional. Maximum timeout for blocked gRPC sends.
      // If this timeout is reached, the stream will be disconnected and re-opened.
      //
      // The value of gRPC send timeout is generally,
      // min(stream_timeout - stream_open_duration, stream_send_max_timeout)
      //
      // A stream_send_min_timeout buffer is added for short final send timeout values to prevent
      // scenarios where the stream deadline is reached too quickly. The complete accuracy of the
      // value is therefore,
      // stream send timeout = max(stream_send_min_timeout, min(stream_timeout - stream_open_duration, stream_send_max_timeout))
      //
      // If unset, the stream_timeout value is used.
      //
      // Examples:
      //
      // Ex 1:
      //   stream_timeout = 15m
      //   stream_send_max_timeout = 5m
      //   stream_send_min_timeout = 1m
      //   ... 1m in send blocks
      //       final send timeout = max(1m, min(15m - 1m, 5m)) = 5m
      //   ... 11m in send blocks
      //       final send timeout = max(1m, min(15m - 11m, 5m)) = 4m
      //   ... 14.5m in send blocks
      //       A 1m buffer is added for short final send timeout values to prevent scenarios where
      //       the stream deadline is reached too quickly.
      //       final send timeout = max(1m, min(15m - 14.5m, 5m)) = 1m
      //
      // Ex 2:
      //   stream_timeout = 5m
      //   stream_send_max_timeout = 10m
      //   stream_send_min_timeout = "" // not configured
      //   ... 1m in send blocks
      //       final send timeout = min(5m - 1m, 10m) = 4m
      //   ... 4m in send blocks
      //       final send timeout = min(5m - 4m, 10m) = 1m
      //   ... > 5m in send blocks will never occur because of the 5m stream timeout
      //
      // Ex 3:
      //   stream_timeout = "" // not configured
      //   stream_send_max_timeout = 5m
      //   stream_send_min_timeout = 4m
      //   ... in all send block scenarios,
      //       final send timeout = max(4m, 5m) = 5m
      //
      // Ex 4:
      //   stream_timeout = 10m
      //   stream_send_max_timeout = "" // not configured
      //   stream_send_min_timeout = 1m
      //   ... in all send block scenarios,
      //       final send timeout = max(1m, 10m) = 10m
      //
    
  • Introduce a NACK from xds-relay -> upstream server on gRPC Recv timeouts in the form of code 4, DEADLINE_EXCEEDED

jessicayuen avatar Jan 25 '21 17:01 jessicayuen