Integrate with Tenderly and perform hash matching
Description
Please export the tenderly access token in the 1password EVM Safety vault as TENDERLY_ACCESS_TOKEN for the below to work.
- get-tenderly-hashes.sh now takes a simulation link as input
- MultisigTask.sol no longer generates a tenderly json payload
-
just simulate-tenderly <path-to-task> <optional-safe-name>insrc/improvementssimulates a task both locally and on tenderly, both single and nested. Returns the usual output, the tenderly simulation link (already simulated, to save some time) and the hashes. -
just verify-non-terminal-taskssimulates locally and on tenderly all tasks that are not SIGNED or EXECUTED, and compares the domain and message hashes in VALIDATIONS.md against the ones in the simulations.
Note that just verify-non-terminal-tasks fails with sep/001-opcm-upgrade-v200 because MultisigTask doesn't print out its individual domain and message hash. I think that's a bug in MultisigTask.
This approach is great. Few comments:
Running:
cd src/improvements
just simulate-tenderly sep/001-opcm-upgrade-v200 nested
I get an error:
Error: script failed: vm.readFile: failed to read from "/Users/blaine/code/superchain-ops/tasks/sep/001-opcm-upgrade-v200/config.toml": No such file or directory (os error 2)
error: Recipe `simulate-tenderly` failed with exit code 1
Running
cd src/improvements
just simulate-tenderly src/improvements/tasks/sep/001-opcm-upgrade-v200 nested
I get an error:
/Users/blaine/code/superchain-ops/src/improvements/script/simulate-task.sh: line 24: pushd: src/improvements/tasks/sep/001-opcm-upgrade-v200: No such file or directory
error: Recipe `simulate-tenderly` failed with exit code 1
It would be good to have some tests for this functionality. Also, is this a step that we're going to get signers to execute separately? How does it slot into the current flow? Or is that a future task?
Fixed, please try:
cd src/improvements
just simulate-tenderly `pwd`/tasks/sep/000-opcm-upgrade-v200 council
just simulate-tenderly `pwd`/tasks/sep/003-unichain-superchain-config-fix
Scripts working again, although most tasks are broken themselves.
just simulate-non-terminal-tasks works, but it exits with an error because 000-opcm-upgrade-v200 reverts on execution.
just simulate-tenderly tasks/sep/003-unichain-superchain-config-fix is the only one that works. It will extract the hashes but won't verify against VALIDATIONS.md because only just simulate-non-terminal-tasks does that.
I left some comments on slack, but the short of it is that the tasks in main are all broken in some way.
We can ignore exclude 000-opcm-upgrade-v200 because is already executed and 003-unichain-superchain-config-fix because it's a draft, but 001-opcm-upgrade-v200 needs to be fixed before it's signed.
I've added a CI step that will pass once that task is fixed. I suggest that the scripts in this branch are used to facilitate that:
Use just simulate-tenderly tasks/sep/001-opcm-upgrade-v200 child-safe-1 to quickly check if a task fails on Tenderly.
Once that passes, just verify-ready-tasks will tell you if that task (and any other ready ones) has matching hashes in VALIDATION.md, local simulation and tenderly simulation.
I'll try again to get this merged when I'm back from leave. The conflicts shouldn't build up much, just on config.toml files that can be easily fixed.
Going to close this PR. Superchain-ops has changed significantly since this PR was opened. If we want this feature, we will need to reprioritize it and then go back to the design stage. Let me know if I'm missing something here and you'd like to keep this PR open for another reason.