Async handling of Onion Messages with `BackpressureQueue`
This pull request refactors the onion message handling to be asynchronous, addressing potential performance bottlenecks in the synchronous design from #10089. This PR also depends on #9838 (commit 3511604e7b18291965cdffa0c564ccf62539b was cherry-picked into this PR). This PR is meant to be merged after those two PRs have been merged.
The main changes are:
-
Introduction of
BackpressureQueue: A new generic, bounded queue is added. This queue uses a "Random Early Drop" (RED) predicate to start randomly dropping incoming messages as it approaches capacity. This prevents the queue from blocking when full and provides backpressure to the network under heavy load. -
Asynchronous Message Processing: The
onion_message.OnionEndpointis modified to use this newBackpressureQueue.- When a new onion message is received from a peer, instead of being processed immediately, it is added to the backpressure queue. This allows the peer's main read loop to continue without waiting for the onion packet to be decrypted and forwarded.
- The endpoint now runs its own background goroutine that dequeues and processes messages, containing the forwarding logic that was introduced in the second PR.
In summary, where the previous PRs introduced the ability to forward onion messages, this PR makes the implementation robust. It ensures that a high volume of onion messages won't block critical peer communication, improving the node's stability and throughput as a router in the onion message network.
@gijswijs, remember to re-request review from reviewers when ready