Unhandled ERR_STREAM_WRITE_AFTER_END due possible no 'error' handler on ClientWritableStreamImpl

[happy15]

Problem description

Unhandled ERR_STREAM_WRITE_AFTER_END due possible no 'error' handler on ClientWritableStreamImpl.

Reproduction steps

It happened on production and several node instances crash due this problem:

events.js:288
    throw er; // Unhandled 'error' event
    ^

Error [ERR_STREAM_WRITE_AFTER_END]: write after end
    at writeAfterEnd (_stream_writable.js:264:14)
    at ClientHttp2Stream.Writable.write (_stream_writable.js:313:5)
    at Http2CallStream.writeMessageToStream (/home/appops/approot/server/node_modules/@grpc/grpc-js/build/src/call-stream.js:332:26)
    at /home/appops/approot/server/node_modules/@grpc/grpc-js/build/src/call-stream.js:665:26
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
Emitted 'error' event on ClientWritableStreamImpl instance at:
    at errorOrDestroy (internal/streams/destroy.js:108:12)
    at onwriteError (_stream_writable.js:462:5)
    at onwrite (_stream_writable.js:483:5)
    at processTicksAndRejections (internal/process/task_queues.js:83:21) {
  code: 'ERR_STREAM_WRITE_AFTER_END'
}

Unfortunately It's unknown how to reproduce it. By checking grpc-js source code it seems there is no 'error' handler on ClientWritableStreamImpl.

Environment

• OS name, version and architecture: [e.g. Linux Ubuntu 18.04 amd64]
• OS Distributor ID: Debian Description: Debian GNU/Linux 9.1 (stretch) Release: 9.1 x86_64
• Node version v12.16.1
• @grpc/grpc-js v1.5.8

Additional context

There is a warning on event loop block before the crash:

(node:33) Warning: Event loop delay exceeded 1 hour: 6154818098838 nanoseconds

[murgatroid99]

First, ClientWritableStreamImpl is the class of an object that is returned to you when you call a client streaming method. You should register an error handler on it so that you can handle whatever errors that may occur.

Second, are you sure that you are not calling write after calling end in your own interactions with that object? That is the general cause of that error.

Third, I do not believe that warning can be caused by grpc-js, but it may be related to the error so you may want to investigate why it is happening.

[happy15]

Thanks for the reply.

We don't explicitly make stream calls. The services defined in proto files don't contain the keyword "stream" at all. And I also checked all the *_grpc_pb.js generated from proto files and all the "requestStream" and "responseStream" are set to false. The type definitions of service in *_grpc_pb.d.ts are all return grpc.ClientUnaryCall. Will it make stream call in this situation?

To the second question, we don't explicitly call "write" after "end". All what we do is just calling something like client.updateDocTitle() where updateDocTitle is the grpc.ClientUnaryCall defined. So we don't interact with that object at least directly.

Thanks again.

[murgatroid99]

What you are describing with just unary requests cannot cause the stack trace you shared. Something somewhere in your code is making a client streaming request.

[happy15]

Hi, we are currently using v1.5.8 and want to upgrade to v1.6.x to handle this issue. Is there any concern when upgrading? I checked the release note and it looks good to me to upgrade directly. But I think it's better to double check with officials on that. Thanks.