reth
reth copied to clipboard
feat: dynamic chunking instead of hardcoding
trafficstars
This is on #19256 @yongkangc @mattsse Please check if this behaviour is correct
Short summary :
Dynamic multiproof chunking now adapts to actual worker capacity instead of blindly slicing every target set. I added a WorkerSnapshot + select_dynamic_chunk_size helper in tree::payload_processor::multiproof that reads idle/pending counts from ProofWorkerHandle, and both on_prefetch_proof and on_state_update use it to limit fan-out to what can be consumed immediately.
When no spare workers exist (or work size is below the baseline), we dispatch the batch as a single proof to avoid inflating queues—addressing the PST latency regression called out in the issue.