go-sero
go-sero copied to clipboard
node sync problem
My nodes are slow to synchronize. After syncing to the latest block, after a while, it is behind the latest block. my service is 4c16G. Do I need to upgrade my service?
INFO [08-01|10:29:18.706] Imported new chain segment blocks=1 txs=18 mgas=13.863 elapsed=26.078s mgasps=0.532 number=3779491 hash=f46ed2…00cd27 cache=268.33mB
INFO [08-01|10:29:31.647] Imported new chain segment blocks=1 txs=8 mgas=7.949 elapsed=12.940s mgasps=0.614 number=3779492 hash=a3537e…90ce5d cache=268.33mB
INFO [08-01|10:30:01.687] Imported new chain segment blocks=1 txs=28 mgas=9.325 elapsed=30.040s mgasps=0.310 number=3779493 hash=c19397…070257 cache=268.33mB
INFO [08-01|10:30:30.420] Imported new chain segment blocks=1 txs=13 mgas=8.651 elapsed=28.732s mgasps=0.301 number=3779494 hash=445bad…f02ec6 cache=268.33mB
INFO [08-01|10:30:59.189] Imported new chain segment blocks=1 txs=18 mgas=10.260 elapsed=28.768s mgasps=0.357 number=3779495 hash=2d3daf…c016c9 cache=268.33mB
INFO [08-01|10:31:57.424] Imported new chain segment blocks=2 txs=35 mgas=20.570 elapsed=58.235s mgasps=0.353 number=3779497 hash=6c01e8…d733c9 cache=268.33mB
INFO [08-01|10:32:09.651] Imported new chain segment blocks=1 txs=11 mgas=1.307 elapsed=12.227s mgasps=0.107 number=3779498 hash=354629…6801ba cache=268.33mB
INFO [08-01|10:33:22.908] Imported new chain segment blocks=4 txs=42 mgas=24.536 elapsed=1m13.257s mgasps=0.335 number=3779502 hash=74e563…fabb4d cache=268.33mB
INFO [08-01|10:33:46.929] Imported new chain segment blocks=1 txs=14 mgas=3.959 elapsed=24.021s mgasps=0.165 number=3779503 hash=8aa9b8…19188d cache=268.33mB
INFO [08-01|10:34:03.385] Imported new chain segment blocks=1 txs=9 mgas=2.819 elapsed=16.455s mgasps=0.171 number=3779504 hash=438a45…370bad cache=268.33mB
Node sync is inefficient when trying to download the full chain from scratch.
Gero bottleneck during node sync is disk I/O, not network.
~1000 files are modified per minute in chaindata folder (Google leveldb), resulting is disk use (SSD) almost 100%. Node sync speed decreases drastically over time:
- ~500 blocks/min at block 2.400.00 (chaindata: 42GB)
- ~70 blocks/min at block 3.200.000 (chaindata: 93GB)
- We are currently at block 7.000.000! It would take weeks to sync if the rate doesn't decrease again!
There is a big scalability issue.
There should be a different way to download / verify / store the chain data when the client is very late behind the last block.
I would add that besides disk I/O, the CPU can also be a bottleneck. Recommend AMD Ryzen 3900x or higher. The constant disk IO requires many CPU threads and larger L3 cache.