helios
helios copied to clipboard
feat: Criterion + Iai Benchmarking [RFC]
Overview
Introduces criterion statistical benchmarking as well as iai one-shot cachegrind benchmarking.
This is really cool. Didn't know about the criterion crate.
We probably want to add the results of the benchmarks somewhere as well.
Probably worth benchmarking at a lower level (skipping network) in addition to the integration benches.
I would suggest a bench of a receipt proof + storage proof. Not a blocker on this PR though.
Probably worth benchmarking at a lower level (skipping network) in addition to the integration benches.
I would suggest a bench of a receipt proof + storage proof. Not a blocker on this PR though.
Would you be willing to spec this out in an issue so I (or some other interested party) can implement in a diff pr?
I'm also not sure if we wan't to exclude the network from the bechmark. Although we have no control of it, the network is the slowest part of every call. I was thinking the benchmarks would just give a sense of how long Helios takes to do most things, rather than some theoretical "how fast is the code executing" metric.
ye, taking offline so we can discuss best way forward