nethermind
nethermind copied to clipboard
Nethermind periodically responds to sendRawTransaction with "fee too low"
Raised by @MicahZoltu on Discord: Nethermind periodically responds to sendRawTransaction with "fee too low" when my fee is way above what should be necessary. 100 max fee, 1 priority fee. If I submit again afterwards it will usually go through. It happens fairly reliably, almost every transaction fails with "fee too low" first, then succeeds by just submitting it again.
I was not able to reproduce this issue on my local Goerli node. Can you share what is the accompanying message when you get "fee too low" error @MicahZoltu ?
I am now running with tracing enabled every time a fee too low transaction shows up. Unfortunately, I haven't been doing a lot of transactions lately so I haven't had an opportunity to grab the error (it goes away for a while after a node restart). As soon as it happens again I'll post the extra log details here.
This is a blocker for Chainlink to run on Nethermind
I am now running with tracing enabled every time a fee too low transaction shows up. Unfortunately, I haven't been doing a lot of transactions lately so I haven't had an opportunity to grab the error (it goes away for a while after a node restart). As soon as it happens again I'll post the extra log details here.
These logs will be extremely helpful. It's very hard to reproduce
What is the status of the issue?
I am still running with tracing enabled, but I haven't run into the problem again. This is complicated by the fact that I don't do a lot of transactions, so it may be a while before I reproduce.
Do we know the status of this issue as of this week?
We are not able to reproduce the issue on our end.
I found the original error message in discord history:
{"jsonrpc":"2.0","error":{"code":-32010,"message":"FeeTooLow, FeePerGas needs to be higher than 21000000000 to be added to the TxPool. Affordable FeePerGas of rejected tx: 20676950200."},"id":8}
Based on tx.CalculateAffordableGasPrice() and tx.CalculateEffectiveGasPrice(), it looks like this error should be expected occasionally whenever the tx pool is full and Nethermind doesn't think the transaction will be more valuable than the least valuable transaction it has pending.
In this case, the least valuable transaction was worth tx.GasBottleneck = 21 Gwei per gas.
The estimated "Affordable FeePerGas" (20.7 Gwei) is calculated based on the minimum of 3 potentially limiting factors: MaxGasFee, MaxPriorityGasFee, and the account balance of the Sender. MaxPriorityFeePerGas was 1 Gwei, which means the most value this transaction could have is BaseFee + 1 Gwei. And it could be less than that if the account balance cannot fully cover tx.Value + GasLimit * (BaseFee + 1Gwei).
Assuming a low account balance wasn't the problem, this means the current BaseFee was 19.7 Gwei, and that everyone in the queue had MaxPriorityFeePerGas set to at least 1.3 Gwei.
I'm not sure if it's possible for the base fee used to compute tx.GasBottleneck to be different from the base fee used to compute AffordableGasPrice, but if a slightly higher base fee was used for the first one, and it wasn't updated before this new tx came in, that could explain it too.
The "fee too low" dropping code path shouldn't be reachable for transactions sent directly to the client over JSON-RPC. Any transactions sent directly to the client should get preferential treatment compared to the rest of the mempool and be retained even if normally they would be kicked out.
You are correct though for transactions received over gossip, they should be kicked out of the mempool if it gets too full.
Ah, good point--in that case, I wonder if re-orgs could be the problem.
It removes each tx from the the mempool as soon as it receives a new head including it. But then if there's a re-org, it adds it back into the mempool, but in doing so it loses its preferential status as a local transaction: https://github.com/NethermindEth/nethermind/blob/master/src/Nethermind/Nethermind.TxPool/TxPool.cs#L186-L198
Any chance that's what might have happened?
In my case, the "fee too low" response came in response to sendRawTransaction and sendTransaction calls, which I think wouldn't be subject to reorgs like this because we get the response essentially immediately. If there was a race condition, I would be surprised given the frequency I got the error back when I was getting it regularly. I haven't got it in a while though, so maybe it has been accidentally fixed?
Chances this is related to #4829 and hence fixed in v1.15.0?
Definitely looks like it is the same bug!