kani icon indicating copy to clipboard operation
kani copied to clipboard

Measure compiler performance and overall execution time as part of our benchmarks

Open celinval opened this issue 2 years ago • 3 comments

Proposed change: Our benchmarks today only measure changes to verification time. We should also include overall execution time and the compilation time.

Motivation: Overall execution time is probably the most important metric since it reflects the exact user experience. Additionally, it would be valuable to understand how the compilation time is impacted with changes to the compiler.

celinval avatar May 15 '23 22:05 celinval

@karkhaz I need your help adding these. All the metrics beforehand were per-harness, while these are per package.

celinval avatar May 25 '23 22:05 celinval

Thanks for the issue! Should be a matter of adding parsing for these values.

jaisnan avatar Oct 11 '23 20:10 jaisnan

We still don't have a reliable way to measure the compiler performance. 😞

celinval avatar Jan 24 '25 00:01 celinval