reth
reth copied to clipboard
Optimize Sender Recovery Process
An attempt to resolve this: https://github.com/paradigmxyz/reth/issues/10835
Description
Spawn a single thread to consume chunks from each batch. The Producer prepares and sends batch chunks, while the Consumer processes these chunks using global Rayon workers. Finally, the producer collects and processes the results, managing both successful completions and any potential errors.
flowchart TD
subgraph SenderRecoveryStage.execute
subgraph Producer
direction TB
A[Start] --> B[Create Chunks from Batches]
B --> C[Send Chunks to Channel]
K[Receive Results from Consumer]
K --> |Success| L[Process Success]
K --> |Error| M[Handle Error]
L --> N[End]
M --> N[End]
end
end
subgraph Consumer["Single thread"]
direction TB
E[Start] --> F[Receive Chunks]
F --> G[Process Chunks with Rayon]
G --> H[Send Results to Producer]
H --> I[End]
end
C --> |Chunks| F
H --> |Results| K
classDef default fill:#dc14,stroke:#000,stroke-width:1px;