ethereumjs-monorepo icon indicating copy to clipboard operation
ethereumjs-monorepo copied to clipboard

Add benchmark for bytecode execution

Open JacekGlen opened this issue 1 year ago • 2 comments

This PR adds an EVM benchmark that can be run for individual bytecodes.

The main motivation for the tool is to be used in gas-cost-estimator project, but its purpose goes beyond it. You can learn more about the project from this short presentation, or visit the wiki Stage 4 Scope.

The benchmark implementation differs slightly as it does not add to the existing Benchmark.Suite. Instead, it creates a new Benchmark object for each run and handles async resolution. This was proven to produce more consistent results.

JacekGlen avatar May 23 '24 12:05 JacekGlen

Will try and get to a review of this PR this week!

acolytec3 avatar May 28 '24 18:05 acolytec3

Hi there, sorry that this takes such a somewhat solid amount of time to review here and comment, everyone on the team is pretty busy with relatively heavy implementational stuff regarding EIPs and the like.

Generally this is pretty interesting work though, we are really eager to continue to have the tooling around benchmarking improved!

@jochem-brouwer recently also worked on some benchmarking tooling regarding the performance of bytecodes for the VM - see https://github.com/ethereumjs/ethereumjs-monorepo/blob/master/packages/vm/DEVELOPER.md#benchmarks - and maybe a reason for a slow start here is also that we are (at least: I am) not really sure how these two tools relate and if we are producing some redundant tooling here or not.

Can you have some word on this? That would help for sure on the process! 🙏 🙂

holgerd77 avatar Jul 01 '24 12:07 holgerd77