A local Overflow Pool to cache transactions during high traffic

[emailtovamos]

Description

• During high traffic, transactions are lost as the current legacy pool (Main Pool) has limits on its capacity and propagating them during high traffic also costs network congestions.
• This PR incorporates an Overflow Pool which stores transactions in memory for such cases and time to time transfers them to the Main Pool when the Main Pool isn't full. It is not propagated or announced while it is in the Overflow Pool. But when it is finally transferred to the Main Pool, at that point it is considered as a transaction that just arrived at the Main Pool and is subject to the usual checks and propagation logic.
• Overflow Pool's size can be chosen by a node depending on its hardware capacity.

Details

a. Main Pool There is no new structure for Main Pool, actually it is just the current Transaction Pool (Legacy Pool). When this Pool isn't full, the received new transactions will be broadcasted to all the relevant peers, just the same as the current behaviour.

b. Overflow Pool: the local buffer (LRU or Heap) When the Main Pool is overflowed during high traffic, then it would put lots of stress on the network if we keep broadcasting new transactions. So we put any new transaction into the Overflow Pool and don’t broadcast or announce. The size of the Overflow Pool could be very large, in order to hold a large traffic volume, like 500K. Suppose the average transaction size is 500B, it will take around 256MB memory, which is acceptable.

How to flush transactions from Overflow Pool to Main Pool:

• To satisfy the first-come-first-serve policy, when new transactions arrive & the Main Pool is full, we will push it to the Overflow Pool if it is not full.
• This transaction that is part of Overflow Pool doesn’t get announced or propagated. Its knowledge and existence is completely local to the particular node.
• We regularly try to pop new transactions from the Overflow Pool to add to the Main Pool when Main Pool has space. This can either be time-based (e.g. once every 10 seconds) or block based (once every new block or every 2 blocks)
• When a transaction moves from Overflow Pool to Main Pool, it is treated the same way when a transaction directly arrives in the Main Pool.
• When a new block is imported, we can try to flush the Overflow Pool to Main Pool.