erigon
erigon copied to clipboard
erigontech
cmd/evm --statetest does not seem to work
Trying to run the evm binary from erigon, on the statetest here. This happens:
user@debian-work:~/workspace/goevmlab/cmd/sstoresloadfuzz$ /home/user/workspace/erigon-evm --json statetest ./test.json
[]
Expected results are same as if I use the geth evm:
user@debian-work:~/workspace/goevmlab/cmd/sstoresloadfuzz$ /home/user/workspace/evm --json statetest ./test.json
{"pc":0,"op":91,"gas":"0x2d1cc4","gasCost":"0x1","memSize":0,"stack":[],"depth":1,"refund":0,"opName":"JUMPDEST"}
{"pc":1,"op":61,"gas":"0x2d1cc3","gasCost":"0x2","memSize":0,"stack":[],"depth":1,"refund":0,"opName":"RETURNDATASIZE"}
{"pc":2,"op":48,"gas":"0x2d1cc1","gasCost":"0x2","memSize":0,"stack":["0x0"],"depth":1,"refund":0,"opName":"ADDRESS"}
{"pc":3,"op":121,"gas":"0x2d1cbf","gasCost":"0x3","memSize":0,"stack":["0x0","0xc94f5374fce5edbc8e2a8697c15331677e6ebf0b"],"depth":1,"refund":0,"opName":"PUSH26"}
{"pc":30,"op":104,"gas":"0x2d1cbc","gasCost":"0x3","memSize":0,"stack":["0x0","0xc94f5374fce5edbc8e2a8697c15331677e6ebf0b","0xc940b5f2046740058558468f238b85db7f6bbe3f3d51e92a3e32"],"depth":1,"refund":0,"opName":"PUSH9"}
{"pc":40,"op":83,"gas":"0x2d1cb9","gasCost":"0x3","memSize":0,"stack":["0x0","0xc94f5374fce5edbc8e2a8697c15331677e6ebf0b","0xc940b5f2046740058558468f238b85db7f6bbe3f3d51e92a3e32","0xb7f7c4147541c695f3"],"depth":1,"refund":0,"opName":"MSTORE8","error":"gas uint64 overflow"}
{"output":"","gasUsed":"0x2d1cc4","time":243062,"error":"gas uint64 overflow"}
{"stateRoot": "a2b3391f7a85bf1ad08dc541a1b99da3c591c156351391f26ec88c557ff12134"}
[
{
"name": "randomStatetestmartin-Wed_10_02_29-14338-0",
"pass": false,
"fork": "Byzantium",
"error": "post state root mismatch: got a2b3391f7a85bf1ad08dc541a1b99da3c591c156351391f26ec88c557ff12134, want 0000000000000000000000000000000000000000000000000000000000000000"
}
]
There are a couple of things wrong, afaict. One was introduced here: https://github.com/ledgerwatch/erigon/commit/592fe32217baab51cb60b85eda8d3ae46451db68#diff-b65701f0424f8a0db413f64508b663f424cff1fd3b35095b09493edb0b39ca29issues - - appending to a list potentially reallocates it, so whatever is added to it is lost.
Can be fixed by :
diff --git a/cmd/evm/staterunner.go b/cmd/evm/staterunner.go
index 6abaa8882..d10553504 100644
--- a/cmd/evm/staterunner.go
+++ b/cmd/evm/staterunner.go
@@ -30,7 +30,7 @@ import (
"github.com/ledgerwatch/erigon/params"
"github.com/ledgerwatch/erigon/tests"
"github.com/ledgerwatch/erigon/turbo/trie"
- "github.com/ledgerwatch/log/v3"
+ log "github.com/ledgerwatch/log/v3"
"github.com/urfave/cli"
)
@@ -93,8 +93,8 @@ func stateTestCmd(ctx *cli.Context) error {
}
// Iterate over all the stateTests, run them and aggregate the results
- results := make([]StatetestResult, 0, len(stateTests))
- if err := aggregateResultsFromStateTests(ctx, stateTests, results, tracer, debugger); err != nil {
+ results, err := aggregateResultsFromStateTests(ctx, stateTests, tracer, debugger)
+ if err != nil {
return err
}
@@ -106,24 +106,23 @@ func stateTestCmd(ctx *cli.Context) error {
func aggregateResultsFromStateTests(
ctx *cli.Context,
stateTests map[string]tests.StateTest,
- results []StatetestResult,
tracer vm.Tracer,
debugger *vm.StructLogger,
-) error {
+) ([]StatetestResult, error) {
// Iterate over all the stateTests, run them and aggregate the results
cfg := vm.Config{
Tracer: tracer,
Debug: ctx.GlobalBool(DebugFlag.Name) || ctx.GlobalBool(MachineFlag.Name),
}
-
db := memdb.New()
defer db.Close()
tx, txErr := db.BeginRw(context.Background())
if txErr != nil {
- return txErr
+ return nil, txErr
}
defer tx.Rollback()
+ results := make([]StatetestResult, 0, len(stateTests))
for key, test := range stateTests {
for _, st := range test.Subtests() {
@@ -180,5 +179,5 @@ func aggregateResultsFromStateTests(
}
}
}
- return nil
+ return results, nil
}
This fix makes it a bit better, however:
$ ./erigon-evm --json statetest ./statetest1.json
[
{
"name": "randomStatetestmartin-Wed_10_02_29-14338-0",
"pass": false,
"fork": "Byzantium",
"error": "EOF"
}
]
Still missing the stateroot and the json ops.
oh, i think nobody ever supported cmd/evm
I mean, it did work until a while ago, and I did include it when doing fuzzing. However, if evm doesn't work any longer, then neither me nor @MariusVanDerWijden will be able to include erigon in cross-client fuzzing. So I'm hoping you'll fix it.
applied your patch: https://github.com/ledgerwatch/erigon/issues/5077#issuecomment-1217879729 by https://github.com/ledgerwatch/erigon/pull/5175
Quick update:
- message creation working (stateroot matches)
Todo:
- fix the logger so the terminal output matches
- possibly wrap some test around the evm using this scenario - @AskAlexSharov thoughts, where should this live? Also could do with some guidance on giving this a deeper test with a few more scenarios
Will push a PR shortly :).
@holiman I think this is OK now - I didn't get the JSON output you have above (containing: name, fork, pass, error), however all the other bits are in place, if you are happy to go ahead and re-try that would be great!
- I didn't get the JSON output you have above (containing: name, fork, pass, error), however all the other bits are in place, if you are happy to go ahead and re-try that would be great!
Yes, that's because the fix here is not included: https://github.com/ledgerwatch/erigon/issues/5077#issuecomment-1217879729. The patch in https://github.com/ledgerwatch/erigon/pull/5175/files doesn't look related at all. Something got missed, somewhere.
Otherwise the json output looks good!
Also, this seems pretty strange. A pretty simple test, with a tx to 0x00000000000000000000000000000000000000f1, which should immediately hit a bad opcode 0x26, but does not spit out any opcodes at all. Also erroneous state root.
Erigon:
[user@work sstoresloadfuzz]$ ~/go/src/github.com/ledgerwatch/erigon/cmd/evm/evm --json statetest ./00000000-storagefuzz-0.json.3
{"stateRoot": "2edcd887255288cb8a6c4ccecb5d9be59a0c75825b0c2096138fec2fba1c2095"}
[]
Geth:
[user@work sstoresloadfuzz]$ ~/go/src/github.com/ethereum/go-ethereum/cmd/evm/evm --json statetest ./00000000-storagefuzz-0.json.3
{"pc":0,"op":38,"gas":"0x79bc08","gasCost":"0x0","memSize":0,"stack":[],"depth":1,"refund":0,"opName":"opcode 0x26 not defined"}
{"pc":0,"op":38,"gas":"0x79bc08","gasCost":"0x0","memSize":0,"stack":[],"depth":1,"refund":0,"opName":"opcode 0x26 not defined","error":"invalid opcode: opcode 0x26 not defined"}
{"output":"","gasUsed":"0x79bc08","time":203310,"error":"invalid opcode: opcode 0x26 not defined"}
{"stateRoot": "cd110481bbeceff80c60b1e806cef771aaac2f06b19cf074dc13fd5041292ef8"}
[
{
"name": "00000000-storagefuzz-0",
"pass": false,
"fork": "London",
"error": "post state root mismatch: got cd110481bbeceff80c60b1e806cef771aaac2f06b19cf074dc13fd5041292ef8, want 0000000000000000000000000000000000000000000000000000000000000000"
}
]
[user@work sstoresloadfuzz]$ cat ./00000000-storagefuzz-0.json.3 :
{
"00000000-storagefuzz-0": {
"env": {
"currentCoinbase": "b94f5374fce5edbc8e2a8697c15331677e6ebf0b",
"currentDifficulty": "0x20000",
"currentGasLimit": "0x26e1f476fe1e22",
"currentNumber": "0x1",
"currentTimestamp": "0x3e8",
"previousHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
"currentBaseFee": "0x10"
},
"pre": {
"0x00000000000000000000000000000000000000f1": {
"code": "0x266000",
"storage": {},
"balance": "0x0",
"nonce": "0x0"
},
"0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b": {
"code": "0x",
"storage": {},
"balance": "0xffffffffff",
"nonce": "0x0"
}
},
"transaction": {
"gasPrice": "0x10",
"nonce": "0x0",
"to": "0x00000000000000000000000000000000000000f1",
"data": [
"0x96eee0431c64ae467cc1d3109cadbfa55de131d1f422119aff44f12a5f1c6c7212976267111c758349500dbb34b0bdb31de6729b78639ec705217c77d9ee59"
],
"gasLimit": [
"0x7a1200"
],
"value": [
"0xf0"
],
"secretKey": "0x45a915e4d060149eb4365960e6a7a45f334393093061116b197e3240065ff2d8"
},
"out": "0x",
"post": {
"London": [
{
"hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
"logs": "0x0000000000000000000000000000000000000000000000000000000000000000",
"indexes": {
"data": 0,
"gas": 0,
"value": 0
}
}
]
}
}
}
@holiman thanks for the additional info - I will apply that patch, then debug with the JSON above to see if I can get to the bottom of the issue.
@holiman additionally is your fuzzing tool/suite available so I can ensure we're compliant before I come back to you?
I'm using https://github.com/holiman/goevmlab/tree/master/cmd/sstoresloadfuzz . So for example
./sstoresloadfuzz --geth ~/workspace/evm --erigon ~/workspace/erigon-evm
My bad @holiman seems I can't do basic conversions to uint256... fixed that individual case and pushed - will go for running the fuzzer now to see how far I can get :)
Edit: getting some hex marshalling errors - will pursue those now
Hey @holiman - are you on discord? Tried to find you there but failed - wanted to ask you some questions about fuzzing and it's potential use for testing a scenario where we want to prevent regression.