saito-lite-rust
saito-lite-rust copied to clipboard
Find a suitable method of profiling nodes to find memory leaks.
The primary prod node uses increasingly more memory over time - eventually causing crashes if not restarted.
This is likely a memory leak. To find it we need to be able to profile memory usage to find which objects in the stack are 'bloating'.
Several methods are mentioned here of which 'clinic' looks the most promising.
https://reintech.io/blog/the-best-tools-for-profiling-nodejs-applications
We need to:
- [ ] find something that can identify memory usage by component (locally)
- [ ] find a way to implement this in prod if it is not obvious from local profiling where the issue is.
Note: to run nodejs debugging tools agains ts-node - the following syntax is needed:
clinic doctor -- node -r ts-node/register start.ts
(where "node -r ts-node/register" is added between the regular command and the target script.
There are two options I have found
v8-profiler-next
This approach means updating the GitHub repository with the necessary (very minimal) profiling code and deploying it without requiring it to stop or causing downtime. The profiling can be configured to run for a period after which the data is dumped. This data can then be analysed at using Chrome Dev Tools
- Advantage Minimal to Zero downtime
2. Clinic.js
This approach requires us to stop the application on Prod and run it with Clinic JS. Application crashing does not provide any dump data, so it needs to be manually stopped for profiling to be completed
Advantage Detailed User interface and graphs to help with analysing
@davikstone2 - with the first option:
- can we see where in the code the objects listed in the v8 profiler are?
- can you set out how we trigger the profiling on the node? We run a command?