reth icon indicating copy to clipboard operation
reth copied to clipboard

feat: dynamic chunking instead of hardcoding

Open 0xKarl98 opened this issue 1 week ago • 1 comments
trafficstars

This is on #19256 @yongkangc @mattsse Please check if this behaviour is correct

Short summary :

Dynamic multiproof chunking now adapts to actual worker capacity instead of blindly slicing every target set. I added a WorkerSnapshot + select_dynamic_chunk_size helper in tree::payload_processor::multiproof that reads idle/pending counts from ProofWorkerHandle, and both on_prefetch_proof and on_state_update use it to limit fan-out to what can be consumed immediately.

When no spare workers exist (or work size is below the baseline), we dispatch the batch as a single proof to avoid inflating queues—addressing the PST latency regression called out in the issue.

0xKarl98 avatar Nov 07 '25 05:11 0xKarl98