go-grpc-middleware icon indicating copy to clipboard operation
go-grpc-middleware copied to clipboard

[grpc_retry] Broken retry logic on client streaming

Open imcom opened this issue 3 years ago • 1 comments

IIUC, this grpc_retry for client streaming is just broken ... here is the core retry logic ... where we have

func (s *serverStreamingRetryingStream) receiveMsgAndIndicateRetry(m interface{}) (bool, error) {
	s.mu.RLock()
	wasGood := s.receivedGood
	s.mu.RUnlock()
	err := s.getStream().RecvMsg(m)
	if err == nil || err == io.EOF {
		s.mu.Lock()
		s.receivedGood = true
		s.mu.Unlock()
		return false, err
	} else if wasGood {
		// previous RecvMsg in the stream succeeded, no retry logic should interfere
		return false, err
	}

Note that the wasGood once it is true ... it will be always true ... hence skipping all retry logic even if there is an error. The retry can take place only if the very first call failed with retry-able errors ...

A screenshot from my debug session can demonstrate it more clearly

image

imcom avatar Jul 03 '22 17:07 imcom

Do you mind trying v2 retry and potentially writing unit test to reproduce? That would be super helpful.

The data you provides is too little. We don't know where this method is used and how - more info would be needed.

bwplotka avatar Mar 19 '23 01:03 bwplotka