Response closed without grpc-status (Headers only)

[kevinprotoss]

Versions of relevant software used 0.12.0 What happened Throw error every about 10 minutes: Response closed without grpc-status (Headers only) What you expected to happen No error here, just normal message How to reproduce it (as minimally and precisely as possible):

Full logs to relevant components

Anything else we need to know Server side: using grpc-node to push message every minute( long polling ) Proxy: using envoy to transport h2 stream Client side: angular app using generated stub

It's still unclear, whether this is an error from server, client or proxy side. I guess the grpc-status header might not be set for some kind of response. Maybe normal message or ping message. If someone knows more details about that, please help me.

[johanbrandhorst]

I imagine this could well be the connection between Envoy and your gRPC server timing out. I've seen messages like this when a browser client closes the page or something like that too. It's a generic "unexpected disconnection" message.

[kevinprotoss]

I think so too, therefore I created another issue in the grpc-node repo. I'll take a further inspect on this problem. Maybe enable debug trace for three parts and take a look what happened.

[turbopape]

Hello guys, I am having this issue too, only when trying to run long polling streaming connections. Using envoy in front of plain Go grpc implementation. Do I have to use grpc-web?

[luhuaei]

Hi, I meet this problem too. Because the buffer size is too large, can use less buffer size to avoid this problem.

[am0wa]

grpc-web-proxy: has read/write timout and migh close your longPolling connection by timeout. Relevant for all client-server streaming calls.

[samelie]

In my case, the kubernetes Kong ingress controller doesnt seem to be sending the chunk chunkType === ChunkType.TRAILERS so during the onTransportEnd function, there was no grpc-status hence the same bailout message. The data was there however.

[stephane-arista]

I'm seeing the same error, but in the network tab of the dev tools the request has a header grpc-status with the status code. When I try to set a breakpoint and read the header, it's not there.

[samelie]

After switching to the Contour + Envoy setup this error vanished.

[kroksys]

This happens when Envoy or grpcwebproxy timeout happens. For grpcwebproxy you can set timeout for your needs like bellow.

--server_http_max_write_timeout=3600s 
--server_http_max_read_timeout=3600s

BUT this does not resolve that connection gets closed. So you have to figure out how to reconnect when this happens.

If you are using RxJS as I do you can add retryWhen for subscription if error happens. For example, if you have code like this.something.subscribe(...) Add following logic before .subscribe

.pipe(retryWhen(errorObservable => {
  return errorObservable.pipe(
    switchMap(err => {
      if (err.statusCode == 2
        && err.statusMessage == "Response closed without grpc-status (Headers only)") {
        return of(err)
      }
      return throwError(err);
    })
  )
}))

This would reconnect automatically on receiving this error. Beware that "Response closed without grpc-status" is returned when server is down and "Response closed without grpc-status (Headers only)" is returned when server closed connection. So filtering only by statusCode (2 == Unknown) is not possible.

Complete example would be:

this.something
  .pipe(retryWhen(errorObservable => {
    return errorObservable.pipe(
      switchMap(err => {
        if (err.statusCode == 2
          && err.statusMessage == "Response closed without grpc-status (Headers only)") {
          return of(err)
        }
        return throwError(err);
      })
    )
  }))
  .subscribe(...)

Good luck!