Testing suite for builder
Rationale
To allow anyone to easily make changes to the builder, measure their impact, and ensure that they are safe.
Implementation
- Provide standardized tools and data (sample bundles)for testing
- Provide a framework for testing (e.g. performance metrics)
- Create guide to deploying and testing the builder
One way to approach this is to create a mass conflicting bundles at random and test merging against those. But thinking it through we might want a standardized dataset that allows for more precise measurement of performance (e.g. variance in the randomized generation of bundles doesn't affect performance)
I was thinking we need these things:
- provide a guide to setting up the builder for local development
- provide a test script to run
- provide testing data (bundles and txs)
the output should be something like:
- hey, the code all works as intended and these X number of checks were met!
- XX bundles were merged
- XX bundles were simulated
- in XX time XX block templates were generated
- the most profitable block was XY ETH with XX bundles merged alongside XX transactions
ideally this is somehow standardized with the same inputs so that you can directly compare 2 versions of the code and say "hey, this version performed better than the other, which performs better than the main branch"