[EPIC] Execution API

[gupadhyaya]

Rollkit currently supports ABCI 2.0 (execution.go) for execution. This work will expand rollkit to support different execution engines by defining a common execution interface (via go-execution) which can be implemented by different execution engines such as ABCI (go-execution-abci), EVM (go-execution-evm), etc.

Details:

• The execution interface (go-execution) can support GRPC (similar to go-sequencing and go-da)
• The interface itself can be inspired by other works such as astria's execution api, omni's octane, berachain's beaconkit, and many more.
• The exact methods in the interface still needs to be determined. But, astria's reference is here.

Tasks

• [ ] Create go-execution repo with ABCI 2.0 methods as interface to begin with.
• [ ] Add grpc proxy and dummy execution implementation along with tests go-execution repo
• [ ] Create go-execution-abci and refactor code from rollkit's execution.go to this repo
• [ ] Refactor rollkit to use go-execution interface and go-execution-abci implementation via grpc endpoint
• [ ] https://github.com/gupadhyaya/go-execution-evm

---

Execution API

go-execution

• [x] rollkit/go-execution#10
• [x] rollkit/go-execution#11
• [x] rollkit/go-execution#12
• [x] rollkit/go-execution#17
• [x] rollkit/go-execution#34

Testing

• [x] rollkit/go-execution#13
• [x] rollkit/go-execution#14
• [x] rollkit/go-execution#15
  • [x] rollkit/go-execution#16
• [x] rollkit/go-execution#24
• [x] rollkit/go-execution#33

go-execution-evm

• [x] Create/move repository
• [ ] Setup repository
• [x] rollkit/go-execution-evm#1
• [ ] rollkit/go-execution-evm#2
• [x] rollkit/go-execution-evm#3
• [x] rollkit/go-execution-evm#4
• [x] rollkit/go-execution-evm#5
• [x] rollkit/go-execution-evm#6
• [x] rollkit/go-execution-evm#12
• [ ] rollkit/go-execution-evm#17

Testing

• [x] rollkit/go-execution-evm#7
• [x] rollkit/go-execution-evm#8
• [x] rollkit/go-execution-evm#9
• [x] rollkit/go-execution-evm#15
• [x] rollkit/go-execution-evm#18

go-execution-cosmos-sdk

• [ ] Implement InitChain
• [ ] Implement GetTxs
  • [ ] Re-import latest ABCI mempool implementation from cometbft
• [ ] Implement ExecuteTxs
• [ ] Implement SetFinal
• [ ] Implement cometbft APIs
  • [ ] Identify what has to be exposed from rollkit via RPC

Testing

• [ ] Setup automated testing
• [ ] Run reusable test suite from go-execution
• [ ] Implementation-specific unit tests

rollkit

• [x] #1896
• [x] #1897
• [x] #1898
• [x] #1899
• [x] #1900
• [x] #1901
• [ ] #1902
• [x] #1903
• [x] #1904
• [x] #1905
• [x] #1906
• [x] #1907
• [x] #1125
• [x] #1908
• [x] #1909
• [ ] #1931

Testing

• [ ] Figure out: re-use vs re-write integration tests

The list above may get outdated; please check the Exec API project board for reference.

[Manav-Aggarwal]

Relevant: https://github.com/gakonst/narwhal-abci-evm

[gupadhyaya]

Execution APIs for consensus client (e.g., rollkit) to interact with execution client (ABCI 2.0 or EVM):

// InitChain is used by the consensus client and execution client to sync
// genesisTime is the genesis time for the chain
// initialHeight is the initial height of the chain
// chainId is the chain id
// stateRoot is the genesis state root
// maxBytes is the maximum bytes that the execution client can allow
InitChain(
     genesisTime time.Time, 
     initialHeight uint, 
     chainId []byte
) (
     stateRoot Hash, 
     maxBytes uint, 
     err error
)

// GetTxs is used by the consensus client (rollkit) to fetch all available transactions from the execution client's mempool.
GetTxs() []Tx

// ExecuteTxs informs the execution client to execute transactions
// blockHeight is the height to be used to produce block/header
// timestamp is the timestamp to be used in the block/header
// prevStateRoot is the previous block hash to create the next block/header
// updatedStateRoot is the updated state root hash
// maxBytes is the maximum bytes that the execution client can allow
ExecuteTxs(
     txs []Tx, 
     blockHeight int, 
     timestamp time.Time, 
     prevStateRoot Hash
) (
     updatedStateRoot Hash, 
     maxBytes uint, 
     err error
)

// SetFinal marks a block as final
SetFinal(
     blockHeight uint
) error

Apart from the APIs, the components related to execution will be refactored away from the consensus client (rollkit) and moved to respective execution client repositories. For example, the mempool, the rpc, the state/storage, and indexers.

The execution client will also establish a direct grpc connection to the consensus client (rollkit) for querying consensus related information. e.g., node info, node status, p2p info, etc.

[gupadhyaya]

Execution API using ABCI 2.0:

InitChain => InitChain

GetTxs => no direct API, but can map to reaping from the mempool

ExecuteTxs => PrepareProposal -> ProcessProposal -> FinalizeBlock -> Commit

SetFinal => no direct API, but can map to an RPC endpoint

Execution API using Engine API:

InitChain => engine_forkchoiceUpdated

GetTxs => no direct API, but can map to reaping from mempool

ExecuteTxs => engine_newPayload, engine_getPayload

SetFinal => engine_forkchoiceUpdated

[Manav-Aggarwal]

Instead of a list of Txs: txs []Tx, we should change it to a single de-serializable bytes object in ExecuteTxs:

ExecuteTxs(
     blockRequest []byte, 
     blockHeight int, 
     timestamp time.Time, 
     prevStateRoot Hash
) (
     updatedStateRoot Hash, 
     maxBytes uint, 
     err error
)

[Manav-Aggarwal]

[jim380]

How's this look?

graph TB
    %% User can submit a tx via two paths
    subgraph User
      A1([User]) -->|"Submit (classic) "| B1[Tx]
      A1 -->|"Submit"| A2[Tx]
    end

    %% go-sequencing handles Rollup Tx ordering and submission in sequence
    subgraph go-sequencing
      direction TB
      A2 --> A3[Create Transaction Inclusion List] --> A4[Order Transactions] --> A5[Put in a Blob] --> A6[Submit to DA]
    end

    %% go-da for final state submission
    subgraph go-da
      direction TB
      A7[Submit]
    end
    
    %% go-execution-evm handles transaction reaping, processing, and execution
    subgraph go-execution-evm
      direction TB
      mempool[Mempool]      
      E1[State Tree]
      E2[Indexer]
      F1(Eth JSON-RPC call)

      %% GetTxs directly connecting to sequencing
      B1 --> mempool
      mempool --> B2[GetTxs]
      B2 --> A3

      %% ExecuteTxs processes canonical list and state root
      A4 -->|"Canonical list of txs to execute"| D1[ExecuteTxs]
      D1 -->|"Create StateRoot"| E1
    end

    %% call Submit in go-da
    A6 --> A7
    E1 -->|"Create Header"| A7

[tac0turtle]

Hey catching up on design, any chance there can be an ADR written on the design. Can be super simple but an issue is easily lost in the future.