evmone icon indicating copy to clipboard operation
evmone copied to clipboard

statetest: Drop support for "fake" `BLOCKHASH`

Open chfast opened this issue 1 year ago • 8 comments

The state tests had a procedure of generating non-empty BLOCKHASH outputs based on the block number. This is deprecated feature not used in any existing tests. Remove support for it in evmone.

chfast avatar Oct 10 '24 09:10 chfast

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.22%. Comparing base (8399303) to head (b778311). Report is 11 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    ethereum/execution-spec-tests#1049      +/-   ##
==========================================
- Coverage   94.23%   94.22%   -0.01%     
==========================================
  Files         155      155              
  Lines       15968    15959       -9     
==========================================
- Hits        15047    15038       -9     
  Misses        921      921              
Flag Coverage Δ
eof_execution_spec_tests 17.70% <0.00%> (+0.01%) :arrow_up:
ethereum_tests 27.50% <100.00%> (+<0.01%) :arrow_up:
ethereum_tests_silkpre 19.36% <0.00%> (+<0.01%) :arrow_up:
execution_spec_tests 20.62% <0.00%> (+0.01%) :arrow_up:
unittests 89.01% <0.00%> (-0.02%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
test/state/host.cpp 100.00% <100.00%> (ø)
test/unittests/state_transition_block_test.cpp 100.00% <ø> (ø)

codecov[bot] avatar Oct 10 '24 10:10 codecov[bot]

Perhaps error out of invalid test?

winsvega avatar Oct 10 '24 11:10 winsvega

Perhaps error out of invalid test?

How exactly?

chfast avatar Oct 10 '24 11:10 chfast

We may get a protection against this in EEST: https://github.com/ethereum/execution-specs/issues/1500.

But it may also break goevmlab fuzzer (cc @holiman). However, they may use "blockHashes" to provide the value for BLOCKHASH(0).

chfast avatar Oct 18 '24 14:10 chfast

What do you mean "deprecated feature" - since the opcode BLOCKHASH is still not deprecated, what is a node supposed to do on that opcode, when executing a statetest?

holiman avatar Oct 18 '24 14:10 holiman

What do you mean "deprecated feature" - since the opcode BLOCKHASH is still not deprecated, what is a node supposed to do on that opcode, when executing a statetest?

State tests try to avoid it at all cost to the point a state test with BLOCKHASH is considered invalid. See https://github.com/ethereum/execution-specs/issues/1500.

chfast avatar Oct 21 '24 07:10 chfast

I think that is insufficient. Random tests are highly useful. I don't much care what the vm returns, but having all vms behave identically is important.

Why do this?

holiman avatar Oct 21 '24 09:10 holiman

Why do this?

It bothers me this is mixed with the semi-production ready code. I'd prefer returning null hash but I think we can find a place for the old behavior...

chfast avatar Oct 23 '24 08:10 chfast

The ethereum/execution-spec-tests#1059 moves this feature "more" into "tests only". This should be fine for now.

chfast avatar Oct 23 '24 20:10 chfast