Ubuntu server disk space fills up after running for 1-2 days
Hello,
After running the local node on Ubuntu for 1-2 days the storage is filled up and everything stops.
Hardware is AWS
- m6i.2xlarge
- 64GB storage
- 8 vCPUs
- ubuntu/images/hvm-ssd-gp3/ubuntu-noble-24.04-amd64-server-20250115
I can add more storage, but if it's going to continually get bigger and bigger, do we have a way to clean up the disk space?
Hey @madebymatty it's somewhat by design that the local node will consume more disk space the longer it runs as the state grows. That being said you can do a couple of things to decrease the disk usage.
- If you're currently using the
--fullflag, you can drop it. When using full mode the node state is stored in the Node, then exported to record files, then the files are uploaded to minio and then imported in the mirror node. So that's about 4 copies for the data. If not in full mode record streams will be exported to a directory where the mirror node will read them for import and then delete them. That is only 2 copies of the data (network node and mirror node, without local export for upload to minio and minio). - You could try to decrease the parameter here https://github.com/hiero-ledger/hiero-local-node/blob/main/compose-network/mirror-node/application.yml#L67 This is the number of transactions per second that the pinger service does. Decreasing it will make overall transaction latency higher (so transactions will execute slower overall), but will also decrease the traffic on the network and might decrease disk usage (YMMV).
Hope this helps. Increasing the disk space will obviously also work.
@isavov thank you. I'd imagined that might be the case.
Should I be expecting 64GB to be used up within a day or 2? would that be normal if I had it set to --full?
I'd followed the recommendation, but I'm assuming people are using this just for testing and not running long term?
@madebymatty to be completely honest I don't know what the exact spec here is, but it is true that most people would use the local node for a few hours and not run it for days, so your case is a bit different. You could drill down further into disk usage to figure out what's going on on your VM by running du -hs *|sort -h in the local node working directory and repeating as needed in the subdirs to get a better idea on what exactly is taking up how much space.
Running without --full should be fine for most cases as well, so you can try that as well.