lighthouse
lighthouse copied to clipboard
Add assertoor tests to CI
Add assertoor tests to CI
This PR adds a new step to the local_testnet CI. This new step spins up a kurtosis testnet (defined in assertoor_network_params.yml) and runs assertoor tests as a github action via kurtosis-assertoor-github-action
Issues w/ assertoor github action
Note that the repository ppa:rmescandon/yq is no longer maintained and does not contain the most up to date version of yq
https://github.com/mikefarah/yq/issues/1308
Installation recommendations for yq can be found here:
https://github.com/mikefarah/yq?tab=readme-ov-file#install
I've updated the local_testnet.yml script to install yq via snap
Choosing which tests to run
The list of assertoor tests can be found here: https://github.com/ethpandaops/assertoor-test/tree/master/assertoor-tests
Theres a few tests that are broken at the momemnt (mev-block-proposal as an example) and a few that take 48 hours to complete. I've picked the most relevant tests that finish running at a reasonable time.
Theres some electra tests that we may want to add in the future
I don't think we need opcodes tests, those sound like they'd be EVM specific
big-calldata-tx-test might be useful for us as it effects blocks size
stability-check and synchronized-check also sound interesting but not sure what they are
the mev-block-proposal-check test fails for both lighthouse and prysm across multiple EL combos. not sure whats up with that, I've left that test out for now
Can you upload the logs to the CI run artifacts? To debug failures
logs should now be uploaded with a 3 day retention window