Portal state network task-list

[kdeme]

Attempt to task-list for the development of the state network. This list is going to be incomplete so feel free to add/suggest items. Perhaps some tasks don't really make sense even depending on the path taken.

One important and still open question regarding the state network is whether to provide leaves only or also intermediate nodes of the tries.

The latest update to the specifications indicate only the leaves, but I don't think this should be seen as a chosen direction.

Now, we could start with providing the leaves, and as R&D goes on it should always be considered what benefit adding the intermediate nodes gives us for each problem tackled.

edit: Trie node storage is now being called Model A and flat storage of leaf data is Model B, see below. It has been suggested that the development of Portal state network should first focus on Modal A, hence we will have to adjust the task list a bit accordingly.

See message from Piper: https://discord.com/channels/890617081744220180/1089234065816821860/1187092509327884410

Task-list

Model A and Model B

• [x] State Content type encoding:
  • [x] Content Key and content id encoding is done already. It just needs potential removal of AccountTrieNodeKey and ContractStorageTrieNodeKey, which I wouldn't actually do, to allow testing with this approach also. But we can move them to the end to match the prefix values for the other 3 types.
  • [x] To match the prefix values, need to also implement unique content prefixes: https://github.com/status-im/nimbus-eth1/issues/1924
  • [x] Account Trie Proofs encoding / decoding: SSZ serdes: related issue Related issue: https://github.com/status-im/nimbus-eth1/issues/1921
    • [x] Currently a proof (which includes the leave) is defined. Need SSZ serdes for this + tests.
    • [x] As mentioned above, in a secondary step, might need something similar for intermediate nodes
  • [x] Storage Trie Proofs encoding / decoding: SSZ serdes
    • [x] Same as above
  • [x] Contract Bytecode
    • [x] Same as above
  • [x] For Account Trie Node. This would require a version with and without proof (Offer would work with proof, FindContent without). Does it need two different content keys for this? Or alter Portal wire protocol to accommodate for this?
  • [x] Contract Storage Trie Node. This would require a version with and without proof? See Account Trie Node.

Model A

• [x] Proof validation

  • [x] Validation of Account Trie Node Proofs from a state: test vectors https://github.com/ethereum/portal-spec-tests/tree/master/tests/mainnet/state/validation
  • [x] Validation of Storage Trie Node Proofs from a state: test vectors https://github.com/ethereum/portal-spec-tests/tree/master/tests/mainnet/state/validation
  • [x] Add validation into the state_network.nim
  • [x] Test: generation + verification loop.

• [ ] Proof generation

  • [ ] Generation of Account Trie Node Proofs from a state:
    • [ ] For only touched state, for leave nodes.
    • [ ] For all state, for leave nodes.
  • [ ] Generation of Storage Trie Node Proofs from a state:
    • [ ] For only touched state, for leave nodes.
    • [ ] For all state, for leave nodes.

• [x] Adding content decoding & proof verification for account trie node and storage trie node proofs requires access to state root.

  • [x] Grab it from a block through history network for now? Or just hardcode something?
  • [ ] Grab if from the Portal Beacon network (but this might require additional work on portal Beacon side first) for latest.

• [x] Gossip & storage:

  • [x] Further implement and test recursive NH gossip of trie nodes, test vectors: https://github.com/ethereum/portal-spec-tests/blob/master/tests/mainnet/state/validation/recursive_gossip.yaml
  • [x] Storage for single trie nodes
  • [x] Gossip for trie nodes with their proof.

• [x] Possible unittest between nodes

  • Start from a static state tree (e.g. from a genesis file)
    • build proofs
    • gossip proofs to a node
    • verify that they are stored on the node
    • If the above bullet is implemented, decoding and verification would be tested here also.

• [x] Implement eth_getBalance or equivalent JSON-RPC calls that needs to access state. This would require to walk the trie, but via the network.

• [ ] Potential bigger test between several nodes (local testnet):

  • build proofs
  • recursive NH gossip of proofs
  • Use the eth_getBalance to get the value for a specific account.
  • For testing the eth_getBalance or similar method, the recursive NH gossip could also be skipped and instead just have all trie nodes injected (even without the proofs/validation).

• [ ] State network bridge that generates the new state content and gossips them in the network:

  • [ ] Be able to start from a state (for full archive this would have to start from genesis and update all the way to current)
  • [ ] Update Only on touched state inject updated ones

Model B

• [ ] Proof generation & verification: dedicated issue https://github.com/status-im/nimbus-eth1/issues/1934

  • [ ] Generation of Account Trie Proofs from a state:
    • [x] For all leaves
    • [ ] For only touched state
  • [ ] Generation of Storage Trie Proofs from a state:
    • [x] For all leaves
    • [ ] For only touched state
  • [x] Verification of Account Trie Proofs from a state
  • [x] Verification of Storage Trie Proofs from a state
  • [x] Test: generation + serdes + verification loop.

• [ ] Test: generation + serdes + verification loop.

• [ ] Add content decoding & proof verification for account trie and storage trie proofs to state network. This does require access to state root.

  • [ ] Might want to hardcode that initially?
  • [ ] Grab if from the Portal Beacon network (but this might require additional work on portal Beacon side first)

• [ ] Potential test between nodes:

  • Start from a static state tree (e.g. from a genesis file)
    • build proofs for leaves
    • offer proofs to a node (or gossip)
    • verify that they are stored on the node ()
    • If the above bullet is implemented, decoding and verification would be tested here also.

• [ ] Implement eth_getBalance or equivalent JSON-RPC calls that needs to access state.

• [ ] Potential bigger test between nodes:

  • Similar as above test, but in a local testnet.
  • Use the eth_getBalance to get the value for a specific account.
  • If that works, it could perhaps be demonstrated that with an actual wallet, but chances are it might error on other unsupported rpc calls.

• [ ] State network bridge that generates the above the state content and gossips them in the network: https://github.com/status-im/nimbus-eth1/issues/1902

  • [ ] The actual generation part here could be skipped by using files from https://github.com/morph-dev/young-ethereum initially.
    • Perhaps first does so for each leave
    • Eventually only on touched state (inject updated ones)
  • [ ] Generation for new blocks/state by grabbing it from a full node, can look into existing witness building code for generation of this.

• [ ] Similar tests as already mentioned above could be done by usage of the bridge instead.

• [ ] How to deal with cold leaf proofs: https://github.com/ethereum/portal-network-specs/blob/master/state-network.md#updating-cold-leaf-proofs

• [ ] Pruning of old leaves with proof

Neither model.

• [ ] Block witness / stateless execution: This should perhaps be moved into a separate task list as it is not really Portal related.
  • [x] Block witness building
    • Lower level: there is code in place already: https://github.com/status-im/nimbus-eth1/tree/master/stateless
    • Higher level, on execution: https://github.com/status-im/nimbus-eth1/blob/master/nimbus/db/ledger/accounts_cache.nim#L693, https://github.com/status-im/nimbus-eth1/blob/657379f484e6e0597a61af21f0613d36b5000aee/nimbus/evm/state.nim#L289
  • [ ] Block witness execution:
    • Tree building: https://github.com/status-im/nimbus-eth1/blob/master/stateless/tree_from_witness.nim