Out of Gas error on subset of private transaction participants

[SatpalSandhu61]

System information

Version: 1.9.7-stable Quorum Version: 2.6.0 (or higher)

Expected behaviour

If a private transaction is executed to perform a simple stored value change, then the gas usage should be consistent on all participant nodes. Therefore, for a given gas value, if the transaction succeeds on one participant then it should succeed on all participants.

Actual behaviour

Gas usage is different if a stored variable has a different value across nodes. This means that private transactions may use different gas amount across nodes, and can lead to an 'out of gas' error on some of the participant nodes.

Steps to reproduce the behaviour

Use the standard simple storage contract from the 7nodes example and after removing the default initialiser value of 42, perform the following:

• Deploy the private simple store smart contract with privateFor=[node 1, node 2] with an initial value of 0.
• Set the store value to 10 on node 1 with privateFor as nil. Ensure that no gas value is passed in txArgs while executing this transaction (which forces an estimateGas() call to be used). As expected, node 1 store value gets updated to 10.
• Execute another transaction to set the value to 30 with the transaction being privateFor both node 1 and 2. Ensure that no gas value is passed in txArgs while executing this transaction.
• When the 2nd transaction is executed, the value at Node 1 changes to 30 but the execution fails on Node 2 with an out of gas error.

Root cause

Till version 2.5.0, if the gas value passed in the transaction was nil, it was hardcoded within geth to 90000. However in 2.6.0 the logic has changed to make a call to EstimateGas(), and the estimated gas is used in the transaction args.

When executing the second transaction the store value at node 1 is changing from a non-zero value to non-zero value, which results in a computed gas of 5000 for the StoreOp. However at node 2 when the transaction is executed the value is changing from zero to non-zero which requires gas of 20000, which is 15000 more than the estimated gas on node 1. This results in the vm reverting the transaction on node 2 with "out of gas" error.