go-ethereum icon indicating copy to clipboard operation
go-ethereum copied to clipboard

time to push first transaction in the network has increased since Monday

Open h4ck3rk3y opened this issue 1 year ago • 13 comments

Hey!

I am Gyani and I work on Kurtosis. We maintain an example https://github.com/kurtosis-tech/awesome-kurtosis/tree/main/smart-contract-example where we show how a smart contract can be deployed against an Etheruem test net spun up using Kurotsis https://github.com/kurtosis-tech/ethereum-package. By default we spin up geth + lighthouse, latest images of both.

From Tuesday we were noticing that the test was failing https://app.circleci.com/pipelines/github/kurtosis-tech/awesome-kurtosis/1476/workflows/7c9ac168-3b25-43fc-9232-b2da96660edc/jobs/3985

The failure talks about ProviderError: transaction indexing is in progress; after much debugging and not finding anything on the internet I decided to read your source code

which is rather new and only got added on Monday - https://github.com/ethereum/go-ethereum/pull/28703/files#diff-af55d4d4253e0b8b994cedaa88607f84009a0b2971f9b3514afa71efc906a9c5

When I switched to using 1.13.10 https://github.com/kurtosis-tech/awesome-kurtosis/pull/282/files things started working again as expected.

I am wondering if this is a bug or is there some change you expect us to make to the smart contract example.

Tagging @leoporoli from Kurtosis who I have been debugging this with

h4ck3rk3y avatar Jan 25 '24 14:01 h4ck3rk3y

Switching v1.13.10 to v1.13.11 you will get the bug that I am talking about

image

Note the run script inside the smart contract example is Linux only; but can be run with some tweaks on Mac using gsed or a different sed sed -i '' "s/<PORT>/$PORT/" hardhat.config.ts

h4ck3rk3y avatar Jan 25 '24 14:01 h4ck3rk3y

It seems like transactions that are being posted later are being processed just fine. If I try deploying the same thing with hardhat a little later it goes through.

h4ck3rk3y avatar Jan 25 '24 14:01 h4ck3rk3y

Summarizing, while running against the latest published since Monday

  1. The smart contracts we are deploying are here; these are example/toy smart contracts https://github.com/kurtosis-tech/awesome-kurtosis/tree/main/smart-contract-example/contracts
  2. The error i get is transaction indexing is in progress; if I try submitting after a while then the error goes away and the network resumes as normal
  3. Here is the log of my geth client running v1.13.11; notice how the error happens in the beginning but fades away and the network resumes as normal https://gist.github.com/h4ck3rk3y/169fe3d2374afaffed75e976bc7ef579

h4ck3rk3y avatar Jan 25 '24 14:01 h4ck3rk3y

Can you please describe the order of events of your test? E.g. are you starting a chain from scratch? When is the block produced? Do you query for receipt immediately after the block is produced?

With #28703, we no longer index txs during sync so after a node is synced you'll have to wait a few minutes to query. In your case, it appears you are starting a fresh chain though, so I wouldn't expect this error to pop up, but maybe if you outline the events it will be clearer.

lightclient avatar Jan 25 '24 17:01 lightclient

I am starting a chain from scratch. Going by the logs the first transaction is being posted even before block number=1.

h4ck3rk3y avatar Jan 25 '24 19:01 h4ck3rk3y

I am also hitting this issue in my local development node that uses geth. I believe my flow is similar:

  1. Init chain from genesis
  2. Make first transaction - a deploy - works great
  3. Second transaction - a contract function invoke - it fails if I try to wait for the tx too faster after sending it with this exact error message

I am not a Go dev but looking at the code, it looks like this error message is returned at the interface level: https://github.com/ethereum/go-ethereum/blob/master/internal/ethapi/errors.go#L78

antazoey avatar Jan 30 '24 15:01 antazoey

it fails if I try to wait for the tx too faster after sending it with this exact error message

What's the waiting time configured after sending the transaction? Originally the returned receipt will be none if it's not indexed yet internally, but now an error(transaction indexing is still in progress) is bubbled up.

Does this change affect your workflow?

rjl493456442 avatar Jan 31 '24 02:01 rjl493456442

What's the waiting time configured after sending the transaction?

After sending, we call wait right away. Here is the wait function:

tx = web3.eth.wait_for_transaction_receipt(hex_hash)

I opened a similar issue in web3.py to maybe adjust the waiting function to handle this error.

Does this change affect your workflow?

I believe it affects web3.py wait for transaction receipt method. right now, it fails instead of waiting.

So I am 2 dependencies down from this change

antazoey avatar Jan 31 '24 15:01 antazoey

@h4ck3rk3y are you using the hardhat in your project? I think it has the same issue as web3.py has https://github.com/ethereum/web3.py/issues/3212#issuecomment-1920398339

Would be nice to open a ticket there too and to align with the new behavior in Geth.

rjl493456442 avatar Feb 01 '24 09:02 rjl493456442

Related to the web3.py ticket, it would be quite nice to get unique error codes in some situations like these in order to distinguish errors a bit better without relying on the message alone.

fselmo avatar Feb 01 '24 21:02 fselmo

Yeah, I did plan to apply a unique error code for it. But it's more like a geth internal thing and error-code spec is not maintained properly? So i just return -320000, a common error code for everything.

We do need to improve it

rjl493456442 avatar Feb 02 '24 11:02 rjl493456442

@h4ck3rk3y @antazoey could you please try the latest web3.py?

rjl493456442 avatar Feb 08 '24 05:02 rjl493456442

@h4ck3rk3y @antazoey could you please try the latest web3.py?

Yes, the web3.py upgrade did resolve my issue.

antazoey avatar Feb 08 '24 06:02 antazoey

This issue seems to be resolved

holiman avatar Feb 22 '24 09:02 holiman

@h4ck3rk3y Which SDK you are using to interact with blockchain?

rjl493456442 avatar Feb 22 '24 09:02 rjl493456442

Hey, sorry for the silence so far @rjl493456442. I am using Hardhat for this project yes, v2.13. I'll try upgrading tomorrow to check if the behavior is fixed in hardhat or not.

h4ck3rk3y avatar Feb 25 '24 22:02 h4ck3rk3y