Measure compiler performance and overall execution time as part of our benchmarks
Proposed change: Our benchmarks today only measure changes to verification time. We should also include overall execution time and the compilation time.
Motivation: Overall execution time is probably the most important metric since it reflects the exact user experience. Additionally, it would be valuable to understand how the compilation time is impacted with changes to the compiler.
@karkhaz I need your help adding these. All the metrics beforehand were per-harness, while these are per package.
Thanks for the issue! Should be a matter of adding parsing for these values.
We still don't have a reliable way to measure the compiler performance. 😞