meta: contract verification experience

[zerosnacks]

Describe the feature you would like

Recently I wanted to verify contracts on multiple verification providers and we can improve the developer experience around this. I would argue that developers by default would want to verify everywhere at once.

We should make it very easy (a default) to at least verify automatically on Etherscan + Verifier Alliance in one go. Because of the https://verifieralliance.org/ both Blockscout and Sourcify (Argot Collective) can be verified at once by verifying on Sourcify. Ideally Etherscan would join the Verifier Alliance but this has not yet happened.

From personal experience - what currently happens is that if a user runs into any issues they eventually find a way to work around it and because they have resolved their own problem there is little incentive to report it as an issue.

Confused on what to do next if a verification failure occurs users often end up running the deploy script again. Not many users write their scripts to be idempotent so users can easily end up deploying contracts multiple times.

Let's discuss how we can improve the developer experience around this.

Goal

Goal is to be able to create actionable items for improving the developer experience around deployment and make it feel more robust and have more helpful dialogue when things do go wrong so users know what to do

Action list

(derived from examples below)

• [ ] Make it possible to verify on Etherscan + Sourcify in one go
• [ ] Make --verifier-api-key work with --verifier etherscan
• [ ] Perform a dry-run API key check prior to running to make sure it is valid, warn the user and do not deploy

Common questions

• How do I verify on Blockscout and Etherscan at once when I run forge script --verify?
• Let's say my verification (partially) failed but my contracts deployed, how do I rerun ONLY the verification process?

Common user experiences that raise the question "uh... what do I do now?" when verifying

Verifying on Blockscout

forge script script/Counter.s.sol --rpc-url <SEPOLIA_RPC_URL> --verify --verifier blockscout --broadcast --keystore <KEYSTORE>

and I get the following

==========================

##### sepolia
✅  [Success] Hash: 0xa7486709fbb1bc28da16126aceebf91a14aa4caf97fe4129938003869a8e9a75
Contract Address: 0xC8320C3CC70dc234298Bb84616f108d61a8215E0
Block: 8383084
Paid: 0.00000147893596998 ETH (151395 gas * 0.009768724 gwei)

✅ Sequence #1 on sepolia | Total Paid: 0.00000147893596998 ETH (151395 gas * avg 0.009768724 gwei)
                                                                                                                                                                

==========================

ONCHAIN EXECUTION COMPLETE & SUCCESSFUL.
##
Start verification for (1) contracts
Start verifying contract `0xC8320C3CC70dc234298Bb84616f108d61a8215E0` deployed on sepolia
EVM version: cancun
Compiler version: 0.8.30

Submitting verification for [src/Counter.sol:Counter] 0xC8320C3CC70dc234298Bb84616f108d61a8215E0.
Error: Encountered an error verifying this contract:
Response: `NOTOK`
Details:
                        `Missing/Invalid API Key`

My contract has been deployed but Blockscout says my API key is missing.

Ah right, okay let's retry this when passing a valid API key

forge script script/Counter.s.sol --rpc-url <SEPOLIA_RPC_URL> --verify --verifier blockscout --verifier-api-key <VALID_API_KEY> --broadcast --keystore <KEYSTORE>

[⠊] Compiling...
No files changed, compilation skipped
Enter keystore password:
Script ran successfully.

## Setting up 1 EVM.

==========================

Chain 11155111

Estimated gas price: 0.018543661 gwei

Estimated total gas used for script: 196813

Estimated amount required: 0.000003649633552393 ETH

==========================

##### sepolia
✅  [Success] Hash: 0x8c988c0cc03f5f33c4ffcc9149386338b7467ff7eee8504f29b5afab02d4c45b
Contract Address: 0x1544Fc9e63e0c1c27A6467E7e0f18aBB996f4251
Block: 8383090
Paid: 0.000001444392929805 ETH (151395 gas * 0.009540559 gwei)

✅ Sequence #1 on sepolia | Total Paid: 0.000001444392929805 ETH (151395 gas * avg 0.009540559 gwei)
                                                                                                                                                                

==========================

ONCHAIN EXECUTION COMPLETE & SUCCESSFUL.
##
Start verification for (1) contracts
Start verifying contract `0x1544Fc9e63e0c1c27A6467E7e0f18aBB996f4251` deployed on sepolia
EVM version: cancun
Compiler version: 0.8.30

Submitting verification for [src/Counter.sol:Counter] 0x1544Fc9e63e0c1c27A6467E7e0f18aBB996f4251.
Error: Encountered an error verifying this contract:
Response: `NOTOK`
Details:
                        `Invalid API Key (#err2)|SEPOLIA-`

Still fails, no clue why.

I also deployed the contract again now.

Verifying on Etherscan

On Etherscan - contract verification failing for whatever reason, no clear instructions on what to do now.

Start verification for (1) contracts
Start verifying contract `0x3163084A4C2Fd035BCC2950B0D8aFC51a4484653` deployed on sepolia
EVM version: cancun
Compiler version: 0.8.30

Submitting verification for [src/Counter.sol:Counter] 0x3163084A4C2Fd035BCC2950B0D8aFC51a4484653.
Warning: Could not detect the deployment.; waiting 5 seconds before trying again (4 tries remaining)

Submitting verification for [src/Counter.sol:Counter] 0x3163084A4C2Fd035BCC2950B0D8aFC51a4484653.
Warning: Could not detect the deployment.; waiting 5 seconds before trying again (3 tries remaining)

Submitting verification for [src/Counter.sol:Counter] 0x3163084A4C2Fd035BCC2950B0D8aFC51a4484653.
Warning: Could not detect the deployment.; waiting 5 seconds before trying again (2 tries remaining)

Submitting verification for [src/Counter.sol:Counter] 0x3163084A4C2Fd035BCC2950B0D8aFC51a4484653.
Warning: Could not detect the deployment.; waiting 5 seconds before trying again (1 tries remaining)

Submitting verification for [src/Counter.sol:Counter] 0x3163084A4C2Fd035BCC2950B0D8aFC51a4484653.
Warning: Could not detect the deployment.; waiting 5 seconds before trying again (0 tries remaining)

Submitting verification for [src/Counter.sol:Counter] 0x3163084A4C2Fd035BCC2950B0D8aFC51a4484653.
Error: Failed to verify contract: Could not detect the deployment.

Transactions saved to: /home/user/test/broadcast/Counter.s.sol/11155111/run-latest.json

Sensitive values saved to: /home/user/test/cache/Counter.s.sol/11155111/run-latest.json

Error: Not all (0 / 1) contracts were verified!

---

forge script script/Counter.s.sol --rpc-url <SEPOLIA_RPC_URL> --verify --verifier etherscan --verifier-api-key <ETHERSCAN_API_KEY> --broadcast raising: Error: Missing etherscan key for chain 11155111

if I run with --etherscan-api-key instead of --verifier-api-key I have no issues, this is counterintuitive.

At least it does not end up deploying my contract (but why not?)

[⠊] Compiling...
No files changed, compilation skipped
Enter keystore password:
Script ran successfully.

## Setting up 1 EVM.

==========================

Chain 11155111

Estimated gas price: 0.009528402 gwei

Estimated total gas used for script: 196813

Estimated amount required: 0.000001875313382826 ETH

==========================

Transactions saved to: /home/user/test/broadcast/Counter.s.sol/11155111/run-latest.json

Sensitive values saved to: /home/user/test/cache/Counter.s.sol/11155111/run-latest.json

Error: Missing etherscan key for chain 11155111

[0xJem]

Improving this flow would be fantastic! A few thoughts:

• While it gets easier with scripts layered on top (you can see some of what we do at Olympus here (deployment) and here (verification)), just figuring out the flags for verification on Etherscan is a challenge.
• Verifying across a number of different services would be fantastic
• API key management would make this easier
  • Imagine verifying on etherscan and blockscout, and having to specify multiple API keys on every forge script call.
  • At least in my usage, API keys remain the same across projects, so it seems intuitive to have a profile/account (like cast wallet) with API keys for different services stored.
• If verification failed and I wanted to continue with the process, appending --resume seems intuitive
• Submitting to 4byte (since it can import using the ABI or source code) also seems aligned
• Project-level config in foundry.toml:
  • Whether to automatically verify upon deployment (with an override available to forge script to negate this)
  • Which services to verify with (so that it's not required with forge script)

[kuzdogan]

This great to see, thank you 🙏 Happy to contribute as much here

... (a default) to at least verify automatically on Etherscan + Verifier Alliance in one go. Because of the https://verifieralliance.org/ both Blockscout and Sourcify (Argot Collective) can be verified at once by verifying on Sourcify.

• This is not fully accurate. Verifier Alliance is just a DB where verifiers "push" to. Pulling is optional and AFAIK none has implemented this yet. So if you verify on Blockscout/Routescan it will not necessarily appear on Sourcify and vice versa. So ideally it goes to all verifiers all at once.

...because they have resolved their own problem there is little incentive to report it as an issue.

• One way to tackle this on top of my mind is to have Github verification issue templates and lead the user to the URL to report these. The caveat would be that people report trivial errors.

users often end up running the deploy script again

• Maybe a warning and a user prompt would help here if a contract has been already deployed?

--verifier-api-key

• What do you mean with a --verifier-api-key? AFAIK blockscout does not require keys, neiter do Sourcify and Routescan. Only Etherscan does (now unified across chains with their v2)

Submitting to 4byte (since it can import using the ABI or source code) also seems aligned

• re: @0xJem Actually this is something we (verifiers) should've been doing, thanks for pointing this out

On a side note, we'd also love to see Foundry switch to our APIv2 (docs) which uses jobs + polling on the verification side (no hanging requests) and the lookup responses are a lot more richer (bytecodes, storageLayout, souceMaps etc.). We are deprecating the legacy API over time.

Edit: If you'll be updating the Sourcify implementation, it would be nice to have the creatorTxHash submitted alongside. Since we don't index chains we try to fetch this externally. Having it already in the req. makes things easier https://github.com/ethereum/sourcify/issues/2113

[CodeSandwich]

Make it possible to verify on Etherscan + Sourcify in one go

I have an opinion on that, could be wrong, YMMV and all.

It doesn't seem too important in forge verify-contract because it's probably easier to run this command twice than crafting a complex multi-verification configuration in the command line arguments. Verification is a flimsy process prone to random failures, so doubling the risk by doubling the points of failure is another reason to use the command twice. For the same reason IMO it's not that useful with forge create --verify and forge script --verify, it just rises the stakes when something randomly fails and makes it more difficult to recover. Those commands are already extremely complex as they are, adding even more features and behaviors certainly will affect the UX. Personally I actively avoid using --verify because of those reasons.

I think that we need to introduce contract verification artifacts. An artifact would be the input JSON ready to submit to the verifiers, the address and chain ID where the contract is deployed and the arguments that have been used in the constructor. Instead of using --verify, forge create and forge script would create and store verification artifacts that could be used to perform the verification in a separate step. The artifacts could be stored in the broadcast directory to keep it tidy. Those artifacts would be then used in forge verify-contract after the deployment is finished, without causing problems if verification randomly fails. forge verify-contract could automatically detect fresh deployments and choose their artifacts for verification if the user provides no explicit data about what they want to verify, this would smooth out the UX. If forge verify-contract could accept artifacts passed as plain files, this would also future-proof the deployments and allow verification even years later on future scanners if the user chooses to save a copy of the artifacts somewhere.

[akshatmittal]

Based on my experience with working with large projects, here's where I have previously faced issues:

1. It would be great if Foundry generated verification artifacts during broadcast/deployment. This is especially useful when verification fails for random reasons, but also important when the contracts include libraries. I currently have a custom script (ts) that generates these for me when the contracts are intended for self distribution.
2. Talking about libraries, there are several flags in Foundry that behave ever so slightly differently. The only time libraries are included in verification tags is when it's part of the broadcast/create-contract flow. Every other flow, manual verification, export solc-input, etc do not include this. It would be great for Foundry to check broadcast artifacts and include deployed libraries in all of those places when trying to verify. I usually have to manually edit the solc input json or specify libs manually in the verify command if the verification failed during deployment for any reason.
3. API key management is already mentioned, and I do hope Etherscan joins the Verifier Alliance so there's one less key to worry about. (bi-directional verifications would go hard!)
4. Let me set verification flags at the project level, so that every deployed contract is automatically verified.

[Hugoo]

ok - sorry for the spam - but 100% support for this initiative :)) ✊ ⎦˚◡˚⎣

[sakulstra]

Chipping in for a team that currently relies mostly on https://github.com/catapulta-sh/catapulta-verify opposed to foundry based verification.

In the last couple years it happened more than a handful of times that there were issues on the etherscan & routescan side. While doing --resume can work, sometimes (e.g. if having multiple scripts in a single file) run-latest.json and thus --resume can be cumbersome to work with. So having verification artifacts imo would be a good thing :+1:.

With catapulta-verify we currently follow the strategy of verifying on all explorers available. So, usually routescan + etherscan and in some cases blockscout. The different flags on foundry i find a bit confusing. Would personally prefer a foundry.toml section:

[verification]
etherscan = "${ETHERSCAN_API_KEY}"
sourcify = true
routescan = true

---

Idk. if/how foundry currently does explorer discovery, on catapulta-verify we query supported chains from:

• https://cdn-canary.routescan.io/api/evm/all/explorers
• https://api.etherscan.io/v2/api

[kuzdogan]

Adding to @sakulstra Blockscout and Sourcify endpoints: https://chains.blockscout.com/api/chains https://sourcify.dev/server/chains

Following the foundry.toml example, I'll emphasize again the verification should be enabled by default on each verifier that does not require an API key. So it should be sourcify = false if someone wants to turn of support for whatever the reason.

[sakulstra]

@kuzdogan If we go the route of default enable, there should be a global disable (e.g. verification = false).

There are good reasons not wanting to verify every contract immediately (e.g. when we test things pre audit on a testnet)

[zerosnacks]

Hi all, thank for all the amazing suggestions and feedback!

Concretely to improve the verification experience we propose the following:

• Provide clear feedback to the user upon failure with concrete instructions on what to do next.
• Make the verification flow fault tolerant and make the --resume flag work not just for deployment but for verification as well. This means that if a verification fails for whatever reason (and whatever step), --resume would pick up where from the step before the failure occured.
• When a user specifies a --verify flag we verify to multiple verification providers by default: etherscan and sourcify. From practical testing it does look like Blockscout pulls from the shared Verifier Alliance database (tested on Base Sepolia). Other verification providers are expected to actively pull from this database too.
• Add a [verification] section to foundry.toml. The existing [etherscan] section will be nested under the [verification] section accessible as [verification.etherscan]. For backwards compatibility defining [etherscan] would still be valid.
• Add a dry run API key check before contract deployment to make sure the API key is set and correct.

Regarding the contract verification artifact - if the --resume workflow works as expected with --verify would this still be required? What information is not available in the broadcast that you would need to submit a verification?

Optionally:

• To the [verification] section add a publish = <bool> | <regex> variable allowing you to prevent the disabling of verification (complete or selectively). This would nullify the --verify command. With the <regex> you would be able to specify a subset of contracts you would want to publish. By default publish would be set to true. Note that you would still need to specify the --verify flag to make the action explicit.
• Offer a set of default API keys for Etherscan so users are not required to always get those themselves. The reliability of these keys are equivalent to using a public RPC. Users will continue to be able to define the API key themselves.

Let me know what you think! Next step will be the creation of tickets and starting the work

[CodeSandwich]

Regarding the contract verification artifact - if the --resume workflow works as expected with --verify would this still be required?

I think so, yes. This is a thread about improving the contract verification UX and--verify is IMO bad by design, verification should be done explicitly and separately. The blizzard of unrelated flags, variables and settings that --verify introduces to commands that are about sending transactions is overwhelming, especially for the new users. Verification artifacts should enable doing things step by step while learning along the way, so the user doesn't need to plan and set everything up upfront and so they can do things later, maybe much later after the transactions are sent.

Make the verification flow fault tolerant

The changes to --resume don't fix the fundamental issue of mixing two largely independent steps into a single command. The partial fault tolerance only makes this mixture more confusing, e.g. what does it mean that the process succeeded, did verification succeed as well? What if the process failed, was the deployment successful? The user probably should carefully read the logs in case one of the contracts randomly failed to verify.

Offer a set of default API keys for Etherscan

I like the idea of providing default, public API keys, I hope that it isn't against Etherscan's ToC and won't get banned, after all they don't provide a public API for a reason. It's probably worth considering what will happen if there are issues with these keys AND they are used by default when Etherscan verification is done AND Etherscan verification is enabled by default.

verify to multiple verification providers by default

The amount of implicit reliance on external services may be getting out of hand here. Maybe it would be less confusing to require the users to pass the --verifier flag or explicitly configure foundry.toml. This is especially true when deploying on more exotic chains that my not have Etherscan or Sourcify, it's hard to tell what's the default behavior without digging deep into Foundry's dependencies' code.

Add a [verification] section to foundry.toml.

I like this idea, but IMO it shouldn't just copy the [etherscan] section, it's a good chance to have a redesign. We need to be able to define the API flavor. The API URL is already in [etherscan], which is good, it needs to be preserved. We need a way to optionally NOT have the API key in foundry.toml so it doesn't need to be checked into a potentially public repository and can be stored in a separate file that is .gitignored. We need a way to define that a key and a URL works on a set of chains, maybe even on all of them. This is similar to the proposed publish entry.

[CodeSandwich]

A very small and dumb but actually convenient change would be to introduce forge verify as an alias to forge verify-contract. It's a very long name for such a basic command and it's inconsistent with the --verify flag name.

[sakulstra]

I like this idea, but IMO it shouldn't just copy the [etherscan] section, it's a good chance to have a redesign. We need to be able to define the API flavor. The API URL is already in [etherscan], which is good, it needs to be preserved. We need a way to optionally NOT have the API key in foundry.toml so it doesn't need to be checked into a potentially public repository and can be stored in a separate file that is .gitignored. We need a way to define that a key and a URL works on a set of chains, maybe even on all of them. This is similar to the proposed publish entry.

Agreed on it being fine to have breaking change.

---

Just to add 2ct on problems i faced today:

• when you pass --verify and have an ETHERSCAN_API_KEY set, it will try to verify via etherscan no matter if the chain is supported or not. For me this broke on metis & soneium as foundry tried to verify via etherscan, which does not exist on these chains.
• on metis, where there is no blockscout the only way i have been able to verify was via --resume --verifier blockscout --verifier-url https://api.routescan.io/v2/network/mainnet/evm/1088/etherscan/api ... which is a bit weird, given it's not blockscout.
• on polygon today etherscan was multiple hours desynced and scripts started failing. This has been quite problematic as we got some multi-step scripts, which failed on first verification and resuming in a later step is quite painful (after 3 h i still didn't manage to verify everything).
• on soneium i just did not manage at all to verify via foundry. When i try to --resume just to rerun the verification on that chain i get a nonce error.

[kuzdogan]

• when you pass --verify and have an ETHERSCAN_API_KEY set, it will try to verify via etherscan no matter if the chain is supported or not. For me this broke on metis & soneium as foundry tried to verify via etherscan, which does not exist on these chains.

Just to confirm this, the exact same issue posted in our public chat today, for Telos chain:

https://discord.com/channels/1160912737539473410/1160912738374127689/1381960391554240564

[kuzdogan]

About the "public Etherscan API key". Keep in mind all keys are throttled to 100 verifications/day, regardless of their tier. Which I find bizzare. Compilation is not a trivial computation but it's not that expensive. What do I do if I want to bulk-verify?

https://docs.etherscan.io/api-endpoints/contracts#verify-solidity-source-code

[zerosnacks]

What is lacking from Foundry scripts? What are the issues you are running into?

Feedback from Uniswap:

Mainly multi-chain deployments and verification on multiple explorers at once. We frequently deploy contracts without constructor arguments to like a dozen chains at once (e.g., Permit2) and it would be nice to be able to do all of this in one go.

[kuzdogan]

One recent learning from Remix:

Remix recently added a checkbox to verify contracts at deployment, that is checked in by default:

Following that we received multiple take-down requests where people accidentally verified their contracts.

Now the checkbox is moved before the button and the button say explicitly "Deploy and Verify"

I'd very much want to nudge people to verify and verify more seamlessly but this speaks to the need of being explicit about the process.