builder icon indicating copy to clipboard operation
builder copied to clipboard

[TESTING] Script for benchmarking builders locally

Open kamilchodola opened this issue 2 years ago • 2 comments

📝 Summary

Script created gives a opportunity to test performance of builder implementation and compare it to latest builder version. This gives an opportunity to ensure that builder works correctly and did not introduce any performance degradation.

Usage of a script is well described in Readme.md

High level usage:

  1. Run a script to build image
  2. Run a script to create a local infrastructure
  3. Wait for X blocks to be executed
  4. Analyze results in Grafana Dashboard

Here is a image of comparison of latest docker image vs latest docker image (Baseline to see if results are the same) image

Next one is comparison of latest vs deneb branch built with command go run emulate_network.go build -t deneb - deneb seems to be a source for latest image so result should also be the same. image

And the last one shows a difference between latest image and main branch (main seems to be outdated and tool clearly exposes a memory usage difference which on main is more than 2x higher, slower bid production on main branch and a bit different disk usage) image

📚 References

Ethereum-package from Kurtosis which is used as a engine to run a devnet: https://github.com/kurtosis-tech/ethereum-package


kamilchodola avatar Nov 10 '23 11:11 kamilchodola

Some updated results as we've increased fuzzing rate since the description was added. Latest vs latest: image Latest vs PR: image

piwonskp avatar Nov 20 '23 14:11 piwonskp

@avalonche fixed

piwonskp avatar Nov 24 '23 19:11 piwonskp