execution-spec-tests
execution-spec-tests copied to clipboard
new(tests): EOF - EIP-5450: RJUMP* vs CALLF tests
🗒️ Description
Adds a first batch of tests checking stack validation of these two kinds of operations RJUMP-ing and CALLF-ing (and RETF-ing ofc). I'm still missing JUMPF, but I want to go step by step as this has taken a while to prepare anyway.
First remark: I've decided to start off with an "automated" approach here, but as you'll see this has several complexities/inelegancies involved. There always is an alternative to just manually type out some interesting scenarios by hand, which we must bear in mind.
The problematic parts are:
- it would be quite hard for all the automatically generated codes to come with a 100% accurate max_stack_height, so I introduce a flag
no_expecations_on_validity
toeof_test
. This is probably the biggest controversy, and I'm also still unsold on this, let's discuss more in the diff. The nice part is that somewhat automatically we get to test invalid and valid codes en masse, without spending too much time on figuring out the exact cases. - it initially generated many many more tests, I've managed to trim this down aggressively to 400, however I took a - potentially unacceptable - shortcut and trimmed the "uninteresting" cases by
pytest.skip
. Without thepytest.skip
it goes up to 800, all of the extra cases are ones which never successfully validate. All that being said, 400 is still a lot of cases, and I'd like to add JUMPF and recursive calls to this in the future iteration - it is generally complex and has an air of "throw mud at wall, see if it sticks" and maybe gets into the court of what fuzzing should accomplish. Again, the "good part" of this is that we're not effectively re-writting the stack validation routines in EEST
🔗 Related Issues
NA
✅ Checklist
- [ ] All: Set appropriate labels for the changes.
- [ ] All: Considered squashing commits to improve commit history.
- [ ] All: Added an entry to CHANGELOG.md.
- [ ] All: Considered updating the online docs in the ./docs/ directory.
- [ ] Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
- [ ] Tests: A PR with removal of converted JSON/YML tests from ethereum/tests have been opened.
- [ ] Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
- [ ] Tests: Ran
mkdocs serve
locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.