snarkOS
snarkOS copied to clipboard
Use more cores with rayon while syncing with the CDN
Since we perform the CDN sync before starting the node, we can use a rayon threadpool with all the cores dedicated to syncing without sacrificing any other functionality.
I tested it back in testnet3 and saw higher CPU utilization.
How much faster is the rate of syncing? Can you share numbers for pre and post syncing for this PR?
Back when I was testing on testnet3 I noted that it doubled my CPU utilization; the exact increase in speed would depend on the number and power of cores, but this results in both faster deserialization and ledger insertion.
Do you have empirical data that you can share to confirm this finding?
Unfortunately no, not unless @zosorock has performed some measurements back when this was requested.
In any case, it's a pretty trivial change, so it can wait until the next time we're using a CDN if we'd like to collect some solid figures before merging.
Closing pending testing of newer optimizations for syncing logic.
@vicsn @ljedrz Please keep this PR in draft state as it's a valid optimization for snarkOS.
gm