reth
reth copied to clipboard
perf: dynamic worker scaling based on available workers for multiproofs
trafficstars
Implement adaptive worker pool scaling for parallel proof workers. Currently workers are statically allocated (see #19700), causing over-provisioning during low load and under-provisioning during spikes (e.g., Xen storage node bursts).
We should dynamically spawn/shutdown workers on demand based on avaliable workers, scaling between configurable min/max bounds to reduce thrashing.
This requires benchmarks and testing.
Proposal
- worker avaliability: if its >= 85% usage scale up, once back to 70% we can scale down back to normal
- pending multiproofs: if pending multiproof > 200 (scale up by 20%)