intermittent "Stream was terminated by peer (errorCode: 2)" error

[kar1981]

Unstable method call from GRPC dart client to server. Intermittent "Stream was terminated by peer (errorCode: 2)" occurs. pubspec.yaml: grpc: ^2.9.0

Repro steps

1. create ClientChannel with ip, ports final cc = ClientChannel(serverIP, port: serverPort, options: const ChannelOptions(credentials: ChannelCredentials.insecure() ));

2. created a stub based on above channel _userQueryStub = UserQueryGRPCClient(cc, options: CallOptions(timeout: Duration(seconds: 30)));

3. call function from stub (repeatedly) GetUserInformationRequest request = GetUserInformationRequest(userId: userId); GetUserInformationResponse response = await _userQueryStub.getUserInformation(request); print(response);

Expected result: return data from server without error

Actual result: largely working, however, every so often the call returns the error: gRPC Error (code: 2, codeName: UNKNOWN, message: HTTP/2 error: Stream error: Stream was terminated by peer (errorCode: 2)., details: null, rawResponse: null)

Seems to have no consistent frequency of error occurring. Can be every 4th call, or 6th call or 10th call.

Details

I used BloomRPC client and spammed the same calls to server and it always returns the response data and never the "Stream was terminated error". This would suggest there is something went wrong on the client either when making the call or receiving the data.

Possibly the same issue encountered by someone else and outlined here: https://github.com/grpc/grpc-dart/issues/222

[kar1981]

To illustrate the problem, I wrote a simple test stub at https://github.com/kwc1212/grpctest

Pressing on the single grey button on the screen makes a call to the grpc server (written in go), the server returns a list of objects and the program just outputs the count.

if you continuously spam this button, that stream error comes up. Some times on the first attempt, sometimes after 5 attempts and sometimes after 20 attempts. This does not happen if I used a java grpc client or the BloomRPC client.

Can someone please let me know what I may have done wrong? or even a hint as to how I can debug this?

ps: the stub opens a new channel and closes it on every call. I have tried both doing it this way, and creating the channel once and spamming the calls. The error occurs either way.

Any help will be greatly appreciated! thanks~

[acoutts]

I've noticed this in my app as well. Hopeful someone has some insight. I'm creating a channel once and re-using it.

The exact error I get is:

gRPC Error (code: 2, codeName: UNKNOWN, message: HTTP/2 error: Stream error: Stream was terminated by peer (errorCode: 0)., details: null, rawResponse: null)

[kar1981]

So since i made that post i have been doing a bit of troubleshooting and what i found was that if i switch off my proxy server (Kong) this problem goes away. So basically i had to open ports for all my services on aws. I know there is still a “bug” somewhere because as i said in my op that if i use a client such as bloom rpc, now matter how hard i spam the calls it doesn’t fail with the “stream was terminated by peer” even with my kong proxy on. But for now if you are using a proxy server between your host and app client, remove that layer and the problem should go away for now.