reth icon indicating copy to clipboard operation
reth copied to clipboard

Optimize Sender Recovery Process

Open TheDhejavu opened this issue 4 months ago • 2 comments

An attempt to resolve this: https://github.com/paradigmxyz/reth/issues/10835

Description

Spawn a single thread to consume chunks from each batch. The Producer prepares and sends batch chunks, while the Consumer processes these chunks using global Rayon workers. Finally, the producer collects and processes the results, managing both successful completions and any potential errors.

flowchart TD
    subgraph SenderRecoveryStage.execute
        subgraph Producer
            direction TB
            A[Start] --> B[Create Chunks from Batches]
            B --> C[Send Chunks to Channel]
       
            K[Receive Results from Consumer]
            K --> |Success| L[Process Success]
            K --> |Error| M[Handle Error]
            L --> N[End]
            M --> N[End]
        end
    end

    subgraph Consumer["Single thread"]
        direction TB
        E[Start] --> F[Receive Chunks]
        F --> G[Process Chunks with Rayon]
        G --> H[Send Results to Producer]
        H --> I[End]
    end

    C --> |Chunks| F
    H --> |Results| K

    classDef default fill:#dc14,stroke:#000,stroke-width:1px;


TheDhejavu avatar Oct 01 '24 15:10 TheDhejavu