go-ethereum icon indicating copy to clipboard operation
go-ethereum copied to clipboard

overlay benchmark: enable proof generation

Open jsign opened this issue 2 years ago • 1 comments

This PR enables proof generation in the block processing pipeline.

The strategy is to allow the access witness to generate an adequately loaded tree in the background while txnContext.AccessWitness is merged in the main block AccessWitness:

  • When Merge() is called, it push a set of new keys that will be included in the proof.
  • The background loader will leverage a new API candidate in go-verkle that loads the paths for those keys, plus all children in touched InternalNodes, which is also required for proving.
  • That API candidate also returns the respective value of the key, which helps building also the keyValues for calling ProveAndSerialize(...).
  • This change indirectly offloads to the background calculating the Pedersen Hashing, which before this change was a blocking procedure when calling accessWitness.Keys(). Now this accessWitness.Keys() API could be removed (to be discussed).

This PR is sitting on top of the new access witness (not merged) PR since it’s the most reasonable base target. I couldn’t test this branch since the base branch sits atop another (temporarily) broken branch. At some point, I’ll be able to run it and confirm if it works as expected (and fix any remaining nit).

This is still a draft PR:

  • [x] When the base-base branch works, run and verify this PR works as expected.
  • [x] When the target branch of this PR is merged, rebase to the new base branch and open for review.
  • [X] Double-check/confirm new included API names.
  • [X] Decide on .Keys() API removal.
  • [x] Add extra logs/stats that can help in understanding overheads.
  • [ ] Rebase to Kaustinen
  • [ ] Update go-verkle

jsign avatar Aug 10 '23 13:08 jsign

Regarding the .Keys() removal, I would wait a tad longer in case it's still useful. I would just add a TODO for now.

gballet avatar Oct 05 '23 09:10 gballet