go-algorand icon indicating copy to clipboard operation
go-algorand copied to clipboard

p2p: fan in incoming txns into backlog worker

Open algorandskiy opened this issue 5 months ago • 1 comments

Summary

While investigating p2p TX traffic and performance, I found transaction pool mutex congestion. This PR is a PoC to use backlogWorker as a pool only accessor similarly to wsnet. Implementation summary:

  • Added syncCh channel to work item wi
  • When it is initiated from gossipsub validator, syncCh is initiated and awaited making validation synchronous as before.
  • At each tx validation check backlogWorker checks if syncCh is set and responds with validation result to the channel.

Additionally, there are couple more fixes that helped with TPS as well:

  • add some workers to pull sub.Next() faster.
  • cache signed transaction ID to reuse in remember and txpool recomputing on a next block arrival.

Test Plan

  1. Unit tests passed
  2. Cluster TPS test showed +700-900 TPS gain (total 7.5k TPS) on a single txn payment scenario. image

algorandskiy avatar Sep 10 '24 20:09 algorandskiy