grpc
NodeJS @grpc/grpc-js High Memory Usage
Problem description
grpc-js - A gRPC call takes up more memory for a request & response of 100MB.
Reproduction steps
We are sending about 100 MB of data in the request and receiving 100 MB in the response. When the gRPC service is in progress, our app memory becomes very high.
Version - 1.10.67
- 100 MB - Before calling a gRPC Service
- 450 to 500 MB - When a request is in progress
- Up 800 MB for 1 to 2 seconds - On receiving the 100MB response.
I downgraded to 1.3.8 version, and I noticed that the memory size was reduced by 200MB. The most recent version is taking up more memory.
Version - 1.3.8
- 100 MB - Before calling a gRPC Service
- 450 to 500 MB - When a request is in progress
- Up 600 MB for 1 to 2 seconds - On receiving the 100MB response.
If this is something expected ?
Environment
- OS name, version and architecture: Linux Debian
- Node version 18.19.0
- Package name and version [email protected]
It is expected that the client makes a couple of temporary copies of each message it sends and receives. Those should eventually get cleaned up by the garbage collector. There is an open PR to sometimes avoid making one of those copies when sending messages from the server (#2658). A similar change could be made on the client, possibly.
There have been a lot of changes between versions 1.3.8 and 1.10.6, so I can't point to a specific reason why memory usage would have increased.
