o1js icon indicating copy to clipboard operation
o1js copied to clipboard

Change wording of the error "Verification_failed (Invalid_proof "In progress")" to "Verification key is missing on account or does not match the verification key of the compiled contract"

Open dfstio opened this issue 1 year ago • 12 comments

I see many questions on the discord on the error Verification_failed (Invalid_proof "In progress")

Usually, it means that

  • The contract is not deployed
  • The contract was deployed using a different verification key (it can occur due to SmartContract code change or o1js version change that breaks verification keys)

Proposal:

Change the wording of the error to explicitly mention that the Verification key may be missing on account or does not match the verification key of the compiled contract.

dfstio avatar Dec 04 '23 09:12 dfstio

Totally agree! Does this error message pop up in o1js or is this something that is shown in an explorer when a transaction fails? If its the latter, this change needs to be made in the mina repository and should have a corresponding issue over there.

Trivo25 avatar Dec 04 '23 10:12 Trivo25

@Trivo25 I saw this error last time when I amended the contract code and forgot to upgrade the verification key. It resulted from the calling tx.send() or tx.prove() in o1js.

There was no info on minascan.io explorer about this transaction, so it is somewhere in o1js or in the result that o1js receives from RPC when calling tx.send().

dfstio avatar Dec 04 '23 10:12 dfstio

@Trivo25 I've emulated the error by changing the contract code again, and the error occurs on tx.send():

[2:47:56 PM] Error: Failed to send transaction {
  statusCode: 200,
  statusText: `Couldn't send zkApp command: (invalid (Invalid_proof "In progress"))`
}

https://github.com/o1-labs/o1js/blob/main/src/lib/mina.ts#L828 https://github.com/MinaProtocol/mina/blob/99449abefb69615567e1109f9124bee156086994/src/lib/network_pool/batcher.ml#L415

dfstio avatar Dec 04 '23 10:12 dfstio

Means we might want to add a handler for that error in src/lib/mina/errors.ts

mitschabaude avatar Dec 04 '23 11:12 mitschabaude

@mitschabaude You might also consider adding the new function SmartContract.checkVerificationKey() that would compare the verification keys of the compiled and deployed contracts. This function can be called externally and internally at the various stages of SmartContract's and transaction life - from constructor to sending the transaction.

I understand that it is not easy to implement, but if implemented, it would throw the error at a much earlier stage and with a much more precise error message pointing to the root cause of the problem.

dfstio avatar Dec 04 '23 13:12 dfstio

I've seen this error recently with o1js 0.18.0 on devnet with fully working code and properly deployed contracts:

Error in task Error: Transaction failed with errors:
- {"statusCode":200,"statusText":"Couldn't send zkApp command: (invalid (Invalid_proof \"In progress\"))"}
    at Object.send (/Users/mike/Documents/DeFi/zkCloudWorker/domain-name-contract/node_modules/o1js/dist/node/index.cjs:20902:15)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at DomainNameServiceWorker.createRollupBlock (/Users/mike/Documents/DeFi/zkCloudWorker/domain-name-contract/src/worker.ts:741:20)
    at DomainNameServiceWorker.txTask (/Users/mike/Documents/DeFi/zkCloudWorker/domain-name-contract/src/worker.ts:242:7)
    at DomainNameServiceWorker.task (/Users/mike/Documents/DeFi/zkCloudWorker/domain-name-contract/src/worker.ts:214:18)
    at Function.processLocalTasks (/Users/mike/Documents/DeFi/zkCloudWorker/domain-name-contract/node_modules/zkcloudworker/lib/ts/src/cloud/local.js:225:28)
    at Object.<anonymous> (/Users/mike/Documents/DeFi/zkCloudWorker/domain-name-contract/tests/contract.test.ts:318:10)

It occurs only when many txs are sent and compile() is called many times for many SmartContracts and ZkPrograms. Running the jest test two times instead of one - first adding .skip to the second part of the test and second time adding .skip to the first part of the test resolves the issue.

Maybe it relates to the memory problems: https://github.com/o1-labs/o1js/issues/1214#issuecomment-2044531062

dfstio avatar Apr 19 '24 18:04 dfstio

Ok this is surprising, that it occurs non-deterministically if I understand you correctly, and in a Jest test where you wouldn't expect a difference in verification key to be an issue

mitschabaude avatar Apr 19 '24 18:04 mitschabaude

Yes, you understand me correctly, it is non-deterministic:

  • It occurs only on devnet; the same test on the local blockchain is passing
  • It can be resolved by first running the first part of the test where the contract is deployed to devnet and the first tx is sent, and then by running the second part of the test where other txs are sent.

I also should say that I compile the same SmartContracts and ZkPrograms in the test two times: it is a side effect of running tests locally for the cloud workers. In real life, those compilations occur in different workers, but local test runs both compilations in the same worker. Still, the same double compilations on the local blockchain work without problems.

I've already sent many txs from AWS, and on AWS lambda, this error does not occur; all txs are sent successfully: https://minascan.io/devnet/account/B62qqNQ9kMtc4L9p19eK8SfLRy8EamtMRWAVqcCaJSM1Q5AD3DjNAME/txs?type=zk-acc

dfstio avatar Apr 19 '24 19:04 dfstio

where the contract is deployed to devnet and the first tx is sent, and then by running the second part of the test where other txs are sent.

could it be a timing issue? i.e. since you first deploy and then send proofs, the deployed vk isn't recognized yet by the node, even though you waited for inclusion?

it sounds almost like just waiting a bit longer is what resolves the problem. (it would seem weird to me to have anything to do with local memory, since this is an error returned by the node upon receiving a transaction)

mitschabaude avatar Apr 19 '24 19:04 mitschabaude

I've tried adding the delay, but it does not help. What I do is wait for the deploy tx to be included in the block, then wait 30 seconds, then send a second tx that is always successful, so the verification key is certainly written correctly. Then, I wait for the inclusion to the block for the second tx, then wait another 30 seconds, then send the third tx.

This error occurs only on the third TX, no matter what TX it is. I've tried to call different methods of the SmartContract, but errors occur for any method.

It is an example of one of the attempts: https://minascan.io/devnet/account/B62qoT67Xdx8fjUc4bTv1a9FAxeogQZzqMAGw8erzwuU8QyfzTgWv5K/txs?type=zk-acc In this specific example, the deploy tx was included in the block at 1:27:31 PM, and the error occurred at 1:31:56 PM.

This error is not a blocker as I need to deploy the contract once; I am simply reporting it to help with investigations.

Memory can be a problem because it can lead to the production of corrupted proof that is rejected by the node.

dfstio avatar Apr 19 '24 19:04 dfstio

Thanks for the reporting! It does sound strange

mitschabaude avatar Apr 19 '24 19:04 mitschabaude

Memory can be a problem because it can lead to the production of corrupted proof that is rejected by the node.

I don't believe this would happen, but I also don't want to reject the possibility off-hand 😅

mitschabaude avatar Apr 19 '24 19:04 mitschabaude