Internal CL for debugging powered by Etherscan

[shekhirin]

Problem

When debugging or testing the node, we often use the --debug.tip flag that forces the node to sync up to the provided canonical block hash. An important difference between this flag and a real Consensus Layer Client is that once the sync to the provided hash is complete, the node stops syncing and stays idle, so new blocks aren't getting processed.

There might be several reasons for not wanting to run a real CL client:

1. Overall complexity of running two processes instead of one, and a need to point a CL client to Reth's Engine API endpoint and JWT token.
2. Increased disk space requirements (CL client has its own database that it needs to maintain).

Solution

This issue proposes an alternative trusted solution to running an actual Consensus Layer Client for people who:

1. Want to develop and debug parts of the Reth that aren't dependent on the behavior of the real CL client.
2. Want to do data processing/analytics and are dependent on Dune/Etherscan/3rd party services anyway.

And, importantly: okay with trusting the Etherscan to report the canonical headers, bodies and reorgs.

When running a node with --debug.etherscan flag (e.g. reth node --chain holesky --debug.etherscan), we start an internal task inside the Reth process that continuously checks Etherscan API for new blocks and reorgs and reports this data to the internal consensus engine, acting as a real CL client.

Design considerations

I am not sure if Etherscan API allows getting all the data we need. If not, the same idea can be applied to an external RPC that Reth task consumes and uses to drive the engine forward.

Check out the Geth/Parity Proxy API.

[shekhirin]

@sevazhidkov I think you should take this

[sevazhidkov]

Will do!

[onbjerg]

Hey @sevazhidkov, any progress? Feel free to open a draft PR if you need direction/are stuck :)

[sevazhidkov]

Hey @onbjerg, sorry for dropping the ball on this! Will try to ship this in the next week. First PR (not directly part of this, but blocks me currently) is up: #8010.

Otherwise, the general design right now (please let me know if you have any feedback):

• In NodeBuilder::launch optionally spawn a EtherscanConsensusClient from a new module.
• In EtherscanConsensusClient::spawned process periodically poll Etherscan (with some config passed from NodeBuilder::launch, e.g. API keys / Etherscan urls for L2s and testnets).
• Convert between Etherscan JSON block and ExecutionPayloadV3 (currently doing this to finish a proper POC).
• Send each new seen block to reth_rpc_api::EngineApiClient::<T>::fork_choice_updated_v3 and reth_rpc_api::EngineApiClient::<T>::new_payload_v3.

[sevazhidkov]

@onbjerg @shekhirin @mattsse early draft is up! https://github.com/paradigmxyz/reth/pull/8082 Quite raw yet, but I'm curious if you feel like it's on the right path.

[emhane]

this is cool. also like the --debug.tip flag, since sometimes it's very useful to sync to a fixed height for testing. reading code comment, seems like plan was maybe to replace --debug.tip? defo think we should keep both https://github.com/paradigmxyz/reth/blob/19e5fcb003aaa1a1caa7f502d791c91f2f83bd54/crates/node-core/src/args/debug.rs#L23