VM: update test runner

[jochem-brouwer]

While debugging some stuff with @acolytec3 we ran into some issues:

• It is not possible to run a specific test "number" in a state test file. State tests have data/gas/value fields in case there are multiple tests. You could try to guess that if the 5th test fails you use --data=4 as option (to run the test with data item 4 - data items are zero indexed). But this does not work if the gas/value changes or if the data fields are not ordered.
• The --debug option in blockchain is only handy if you have a failing test. If a test fails, that means that while running the block, there is an error and the entire state will be reverted. Only after running the block, we verifyPostConditions which is interesting while debugging. However, this is never ran, because running the block threw due to (usually) failing state tries and thus the state is reverted. If the --debug option is done, run the blocks with the --generate option and then verifyPostConditions, and finally check if the state root is the actually reported root.

[acolytec3]

There's also the issue with not being able to use the StateTestRunner where specifying EIPS to be activated along with the fork like London+3540+3670.

I have this working in PR #1719 already following a suggestion from @jochem-brouwer but haven't committed it yet. Would y'all be good with me adding it to that PR since I think it's closer to merging?

[holgerd77]

Yes.

[holgerd77]

@acolytec3 not sure, has this (or: parts of this) been tackled and can the issue be closed?

[acolytec3]

only the part about running the StateTestRunner with specific EIPs specified. I haven't touched the points that @jochem-brouwer raised in the top of the issue.

[holgerd77]

Do we want to keep this open here?

[acolytec3]

Assuming we're going to continue to maintain the test runner, this is definitely still nice to have, though no imminent priority.