erigon mining with bor consensus

[manav2401]

System information

Erigon version: erigon version 2022.99.99-dev-05243bb4 OS & Version: Ubuntu, 20.04 LTS Erigon Commit hash: 05243bb4954e29d9de9524c056128f9f9d24073e Erigon lib Commit hash: 404276494a0cae474145df2632ce75f1d5d8d032

Abstract

I am using the polygon fork for running some experiments on mining using a customised load bot. I have made some modifications in the metaTx.better function present in erigon-lib's txpool to honour the nonce as well while choosing better transaction. Also, I've increased the number of transactions to pick for mining to 1000 from 200.

Expected behaviour

When transactions are bombarded, we expect them to go to the pending txpool and they should be taken up for mining in next sequence of blocks. I am able to run some minimalistic low TPS single account experiments using the same setup. But, when I use somewhat high TPS multi-account load bot, there are 2 visible issues.

1. Liveness of the network is affected: When the network is bombarded with multi-account transactions, at some point of time it keeps preparing same block again and again and does not make progress. There are transactions present in the pending pool at that moment. Also, we're unable to see any trigger point which might cause this. It just keeps producing same block after 50-60ms.

2. Lot of transactions get into nonce too low error during the state transition. Basically it's trying to execute txs with low nonce where as the state already has some high nonce tx. I have confirmed that there are no duplicate transactions from same account being sent from the load bot. This might be a data race issue.

Actual behaviour

All transactions should be included in pending pool first and eventually should be included in mining. Liveness of the network shouldn't be affected.

Steps to reproduce the behaviour

1. Setup accounts and TPS in this loadbot.
2. Use the commit hashes provided for erigon and erigon-lib. The following line in metaTx.better function in txpool in erigon-lib has been updated and also has an additional check.

+               if mt.nonceDistance != than.nonceDistance && mt.Tx.SenderID == than.Tx.SenderID {
+                       return mt.nonceDistance < than.nonceDistance
+               }

3. Run using the following command: /home/ubuntu/erigon/build/bin/erigon --chain=bor-devnet --datadir ~/.erigon/data --mine --bor.withoutheimdall --txpool.accountslots 100000 --http --http.api "eth,txpool,debug"

The load bot will fund accounts initially (those txs should easily go through). When it starts sending transactions from those newly created accounts, the issues mentioned above should be visible.

Logs

https://gist.github.com/manav2401/b3402e225edf619749561c4b83fb370e