ceremonyclient icon indicating copy to clipboard operation
ceremonyclient copied to clipboard

High CPU usage and potential memory leak?

Open zenodeapp opened this issue 1 year ago • 7 comments

Ola!

So my node has been running for a couple days now and I just realized that my CPU and RAM + SWAP were at max usage for quite a while. I kinda forgot to make a screenshot and in my haste restarted the node to see if it would happen again. So I'll send a screenshot once it's that time again. But wanted to address this 'concern' in the meantime.

Is this normal behavior for it to gradually use all my resources to the max? I've seen this behavior previously when there was no cap on the amount of peers it can discover in other cosmos chains (or if the genesis file was too big), but I'm a pleb not sure if this has anything to do with it at all. Could of course be so many other things that could result to memory leakages.

Currently it already started to use quite a lot of mem + CPU: image

PS: I'm running the ceremony client node using the binary installed via go install inside the /ceremonyclient/node folder. So it's the /root/go/bin/node file (I should have renamed that btw hehe.)


Oh yeah, specs: 64GB DDR4 RAM Configured 32GB swap 2x 512 GB NVMe SSDs AMD Ryzen 5 3600 CPU

zenodeapp avatar Mar 05 '24 09:03 zenodeapp

Hehe it started to get into Swappy swap area.

image

This is about 2 hours and 45 min after my first post.

zenodeapp avatar Mar 05 '24 11:03 zenodeapp

Oh lord. Forced to restart it.

image

zenodeapp avatar Mar 05 '24 15:03 zenodeapp

Oh, I just saw: https://github.com/demipoet/demipoet.github.io?tab=readme-ov-file#1-limit-node-cpu-usage

Will tell if it works better after applying this. I set it to 1080% (90% x 12 cores).

zenodeapp avatar Mar 05 '24 16:03 zenodeapp

I just find the mem usage odd, since I'm not getting this on my other node. Though that node has the grpc listener address configured + also the stats config.

This node that has this much mem usage has neither of those configured.

zenodeapp avatar Mar 05 '24 16:03 zenodeapp

This has nothing to do with the CPU. It should be a memory overflow or something wrong with the code.

Tofu-killer avatar Mar 05 '24 17:03 Tofu-killer

This has nothing to do with the CPU. It should be a memory overflow or something wrong with the code.

Yep. I btw am a goof for not being on the latest version. I created the node 2 days ago and coincidentally on that day a new release was made (v1.4.1). I was on v1.4.0, now on v1.4.2. It might have been the reason to why it was acting up.

Will report back in a couple hours.

zenodeapp avatar Mar 05 '24 17:03 zenodeapp

I have used all his versions and there are memory leaks.....:)

Tofu-killer avatar Mar 05 '24 17:03 Tofu-killer

Alright, enough time passed to conclude this is no longer an issue since V1.4.2. The other problem I now have is that frames seem stuck in the 4k range. Though this is outside the scope of this issue.

Will close this!

zenodeapp avatar Mar 06 '24 05:03 zenodeapp