reth icon indicating copy to clipboard operation
reth copied to clipboard
verified publisher icon paradigmxyz

Hive `ethereum/rpc` suite hangs on `http/CanonicalChain` test

Open Rjected opened this issue 2 years ago • 4 comments

Describe the bug

Currently, every test except for http/CanonicalChain completes in the ethereum/rpc test suite. In the following run from #1348, every other test succeeds - you can see this because the http/CanonicalChain test has the ID test=6, and every other test (besides test=1, which completes only after the suite is done) has a log message with either pass=true or pass=false.

The timed out ethereum/rpc actions run: https://github.com/paradigmxyz/reth/actions/runs/4272628614/jobs/7438032380

Local logs for this test show that the latest block is being polled, but the chain does not advance:

Simulator RPC logs 
>>  {"jsonrpc":"2.0","id":27,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":27,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":27,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":27,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":27,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":27,"method":"eth_blockNumber"}
<<  {"jsonrpc":"2.0","result":{"hash":"0x23234c585795e91d7a867b20116da9a5986b8f2ba08c04390fe0a74b6181aed0","parentHash":"0x0000000000000000000000000000000000000000000000000000000000000000","sha3Uncles":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","author":"0x0000000000000000000000000000000000000000","miner":"0x0000000000000000000000000000000000000000","stateRoot":"0x2ac377ac2d4e6fc9d38239f818814469077479601e18dadf570babdd4129cdb6","transactionsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","receiptsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","withdrawalsRoot":null,"number":"0x0","gasUsed":"0x0","gasLimit":"0x2fefd8","extraData":"0x0000000000000000000000000000000000000000000000000000000000000000658bdf435d810c91414ec09147daa6db624063790000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","timestamp":"0x1234","difficulty":"0x20000","mixHash":"0x0000000000000000000000000000000000000000000000000000000000000000","nonce":"0x0000000000000000","size":"0x26e","totalDifficulty":"0x20000","uncles":[],"transactions":[],"size":"0x273","withdrawals":null},"id":25}
>>  {"jsonrpc":"2.0","id":27,"method":"eth_blockNumber"}
<<  {"jsonrpc":"2.0","result":"0x0","id":27}
<<  {"jsonrpc":"2.0","result":"0x0","id":27}
<<  {"jsonrpc":"2.0","result":"0x0","id":27}
<<  {"jsonrpc":"2.0","result":"0x0","id":27}
<<  {"jsonrpc":"2.0","result":"0x0","id":27}
<<  {"jsonrpc":"2.0","result":"0x0","id":27}
<<  {"jsonrpc":"2.0","result":"0x0","id":27}
>>  {"jsonrpc":"2.0","id":28,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":28,"method":"eth_blockNumber"}
<<  {"jsonrpc":"2.0","result":"0x0","id":28}
<<  {"jsonrpc":"2.0","result":"0x0","id":28}
>>  {"jsonrpc":"2.0","id":26,"method":"eth_getBlockByNumber","params":["latest",true]}
>>  {"jsonrpc":"2.0","id":28,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":28,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":28,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":28,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":28,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":28,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":28,"method":"eth_blockNumber"}
<<  {"jsonrpc":"2.0","result":"0x0","id":28}
<<  {"jsonrpc":"2.0","result":{"hash":"0x23234c585795e91d7a867b20116da9a5986b8f2ba08c04390fe0a74b6181aed0","parentHash":"0x0000000000000000000000000000000000000000000000000000000000000000","sha3Uncles":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","author":"0x0000000000000000000000000000000000000000","miner":"0x0000000000000000000000000000000000000000","stateRoot":"0x2ac377ac2d4e6fc9d38239f818814469077479601e18dadf570babdd4129cdb6","transactionsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","receiptsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","withdrawalsRoot":null,"number":"0x0","gasUsed":"0x0","gasLimit":"0x2fefd8","extraData":"0x0000000000000000000000000000000000000000000000000000000000000000658bdf435d810c91414ec09147daa6db624063790000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","timestamp":"0x1234","difficulty":"0x20000","mixHash":"0x0000000000000000000000000000000000000000000000000000000000000000","nonce":"0x0000000000000000","size":"0x26e","totalDifficulty":"0x20000","uncles":[],"transactions":[],"size":"0x273","withdrawals":null},"id":26}
<<  {"jsonrpc":"2.0","result":"0x0","id":28}
<<  {"jsonrpc":"2.0","result":"0x0","id":28}
<<  {"jsonrpc":"2.0","result":"0x0","id":28}
<<  {"jsonrpc":"2.0","result":"0x0","id":28}
<<  {"jsonrpc":"2.0","result":"0x0","id":28}
<<  {"jsonrpc":"2.0","result":"0x0","id":28}
>>  {"jsonrpc":"2.0","id":29,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":29,"method":"eth_blockNumber"}
<<  {"jsonrpc":"2.0","result":"0x0","id":29}
<<  {"jsonrpc":"2.0","result":"0x0","id":29}
>>  {"jsonrpc":"2.0","id":27,"method":"eth_getBlockByNumber","params":["latest",true]}
>>  {"jsonrpc":"2.0","id":29,"method":"eth_blockNumber"}
<<  {"jsonrpc":"2.0","result":"0x0","id":29}
<<  {"jsonrpc":"2.0","result":{"hash":"0x23234c585795e91d7a867b20116da9a5986b8f2ba08c04390fe0a74b6181aed0","parentHash":"0x0000000000000000000000000000000000000000000000000000000000000000","sha3Uncles":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","author":"0x0000000000000000000000000000000000000000","miner":"0x0000000000000000000000000000000000000000","stateRoot":"0x2ac377ac2d4e6fc9d38239f818814469077479601e18dadf570babdd4129cdb6","transactionsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","receiptsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","withdrawalsRoot":null,"number":"0x0","gasUsed":"0x0","gasLimit":"0x2fefd8","extraData":"0x0000000000000000000000000000000000000000000000000000000000000000658bdf435d810c91414ec09147daa6db624063790000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","timestamp":"0x1234","difficulty":"0x20000","mixHash":"0x0000000000000000000000000000000000000000000000000000000000000000","nonce":"0x0000000000000000","size":"0x26e","totalDifficulty":"0x20000","uncles":[],"transactions":[],"size":"0x273","withdrawals":null},"id":27}
>>  {"jsonrpc":"2.0","id":29,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":29,"method":"eth_blockNumber"}
<<  {"jsonrpc":"2.0","result":"0x0","id":29}
>>  {"jsonrpc":"2.0","id":29,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":29,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":29,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":29,"method":"eth_blockNumber"}
<<  {"jsonrpc":"2.0","result":"0x0","id":29}
<<  {"jsonrpc":"2.0","result":"0x0","id":29}
<<  {"jsonrpc":"2.0","result":"0x0","id":29}
<<  {"jsonrpc":"2.0","result":"0x0","id":29}
<<  {"jsonrpc":"2.0","result":"0x0","id":29}

The test is implemented here: https://github.com/ethereum/hive/blob/master/simulators/ethereum/rpc/ethclient.go#L290

This is because the entire test suite depends on reth being able to mine blocks with clique: https://github.com/ethereum/hive/blob/e808ab73c768c2003f5a6138fa528e01ef8e38d3/simulators/ethereum/rpc/main.go#L30-L34

So we need to figure out a way to regularly produce new blocks, ~~or exclude this test from the CI run until we can do so properly.~~

EDIT: So many tests depend on this behavior that we need to implement some block production

Steps to reproduce

Run the hive tests using the instructions in #851, running the ethereum/rpc simulator.

Node logs

No response

Platform(s)

No response

What version/commit are you on?

No response

Code of Conduct

  • [X] I agree to follow the Code of Conduct

Rjected avatar Feb 28 '23 05:02 Rjected

This test only checks whether or not uncles, receipts, and transactions can be fetched over RPC, and are consistent. We can implement a --mine flag, or something similar, which produces blocks on an interval. We can do this without implementing clique block production.

This does not need to be a valid proof of work or clique chain, and we can use the HIVE_CLIQUE_PERIOD environment variable as the argument to the flag.

Rjected avatar Mar 03 '23 03:03 Rjected

Another note - this also effects at least these tests in the rpc simulator/suite:

  • http/EstimateGas
  • http/ContractDeployment
  • http/ContractDeploymentOutOfGas
  • http/TransactionCount
  • http/Receipt
  • http/BalanceAndNonceAt
  • http/ABITransact
  • http/TransactionInBlock

Rjected avatar Mar 03 '23 03:03 Rjected

Closed by https://github.com/paradigmxyz/reth/pull/1817 hopefully.

gakonst avatar Mar 20 '23 02:03 gakonst

@Rjected I debugged this and it's because the provided genesis state:

"config": {
    "clique": {
      "period": 1
    },
    "chainId": 7,
    "homesteadBlock": 0,
    "eip150Block": 0,
    "eip155Block": 0,
    "eip158Block": 0
  }

does not activate Paris and we end up here:

https://github.com/paradigmxyz/reth/blob/236a10e73b3887744198e2573b73c7fa5b945a03/crates/blockchain-tree/src/blockchain_tree.rs#L378-L385

2023-06-20T15:03:47.310044Z  INFO try_insert_validated_block{block=(1, 0x412c4f63d5c760589b6de4c604ae7dd250ef18cc772ac024ad7bbc6700a07647)}: blockchain_tree: return=Err(InsertBlockError { error: Execution(Validation(BlockPreMerge { hash: 0x412c4f63d5c760589b6de4c604ae7dd250ef18cc772ac024ad7bbc6700a07647 })), hash: 0x412c4f63d5c760589b6de4c604ae7dd250ef18cc772ac024ad7bbc6700a07647, number: 1, parent_hash: 0xf24584aea2f75656cf2064042fc1660108f7438a8bc585553c32611be6155f10, num_txs: 0, .. })
2023-06-20T15:03:47.310078Z DEBUG consensus::engine: Failed to insert downloaded block err=InsertBlockError { error: Execution(Validation(BlockPreMerge { hash: 0x412c4f63d5c760589b6de4c604ae7dd250ef18cc772ac024ad7bbc6700a07647 })), hash: 0x412c4f63d5c760589b6de4c604ae7dd250ef18cc772ac024ad7bbc6700a07647, number: 1, parent_hash: 0xf24584aea2f75656cf2064042fc1660108f7438a8bc585553c32611be6155f10, num_txs: 0, .. }

not sure how to fix properly, all solutions that come to mind rn are a bit hacky, like manually adding the paris hardfork for clique...

mattsse avatar Jun 20 '23 15:06 mattsse