help icon indicating copy to clipboard operation
help copied to clipboard

How to investigate RSS memory leak?

Open mwilniewiec opened this issue 2 years ago • 5 comments

Details

I have a problem with one of my microservices that its RSS memory is rising and I don't know what should be a proper way of finding the real problem. I already analyzed heapdumps and nothing can be found as I can see that heap is pretty stable.

process.memoryUsage() is currently showing:

external: 40866815,   (38.97 MB)
heapTotal: 135319552, (129.05 MB)
heapUsed: 114264216,  (108.97 MB)
rss: 1641189376,      (1565.16 MB)

It's a docker running inside of a k8s pod doing quite a lot of things.

  • it's a client of RabbitMQ (using amqplib) processing lots of messages
  • using prostgres with typeorm
  • using redis
  • making GRPC calls to other services
  • decompressing data using zlib.inflateSync
  • lots of other stuff

Heap: image

RSS: image

In the charts you can see that there are 2 instances of this microservice and both have exactly the same growth in RSS and exactly the same Heap.

Is there any way that I could analyse the other 1.3 GB of RSS memory which is not in the Heap ?

Node.js version

12.14.1

Example code

No response

Operating system

Alpine Linux v3.11 (in a Docker)

Scope

runtime

Module and version

Not applicable.

mwilniewiec avatar Apr 05 '23 10:04 mwilniewiec

Take a look at v8-profiler, may be it can help

preveen-stack avatar Apr 27 '23 10:04 preveen-stack

Also, pls check response to https://github.com/nodejs/help/issues/4148

preveen-stack avatar Apr 27 '23 15:04 preveen-stack

This may also be helpfull - https://github.com/nodejs/node/blob/main/doc/contributing/investigating-native-memory-leaks.md

mhdawson avatar Apr 27 '23 20:04 mhdawson

There has been no activity on this issue for 11 months. The help repository works best when sustained engagement moves conversation forward. The issue will be closed in 1 month. If you are still experiencing this issue on the latest supported versions of Node.js, please leave a comment.

github-actions[bot] avatar Mar 23 '24 01:03 github-actions[bot]

it is worthwhile to see what is the system characteristics. summary of https://github.com/nodejs/node/issues/31641 in this context is:

rss stays high for ever, if there is no demand for memory in the system

it is a known issue, and the current workaround is to ignore the observation, the growth will subside when there is a demand.

or in other words - the system will adapt to the need, don't measure too much! :)

gireeshpunathil avatar Mar 23 '24 01:03 gireeshpunathil

It seems there has been no activity on this issue for a while, and it is being closed in 30 days. If you believe this issue should remain open, please leave a comment. If you need further assistance or have questions, you can also search for similar issues on Stack Overflow. Make sure to look at the README file for the most updated links.

github-actions[bot] avatar Sep 20 '24 01:09 github-actions[bot]

It seems there has been no activity on this issue for a while, and it is being closed. If you believe this issue should remain open, please leave a comment. If you need further assistance or have questions, you can also search for similar issues on Stack Overflow. Make sure to look at the README file for the most updated links.

github-actions[bot] avatar Oct 20 '24 01:10 github-actions[bot]