ScalaPB icon indicating copy to clipboard operation
ScalaPB copied to clipboard

Benchmarks: run current version, parameters, profiling, structured results

Open ak0rz opened this issue 1 year ago • 0 comments

Background

I'm going to introduce numerous changes related to performance of scalapb, but I noticed that existing suite is outdated and using template-based codegen to build suites. Also, I noticed that present results are incomparable because there was no platform information in result structure.

Solution

First, I've made a small local plugin that handles all dependencies for benchmark projects, allowing using either scalapb built from current sources or downloaded from remote ivy repo. Second, I performed refactoring of suites so they would be actual benchmark classes, possibly annotated with @State and having @Param parameters. Thus added ability to test repeated field with various number of elements. And lastly, I've made a custom JMH runner to be able to separate results for each benchmark in its own directory + put there profiling information too.

Pro-tip for review

Filter out all .html and .json files that was generated as a result of benchmark runs

ak0rz avatar Mar 10 '24 20:03 ak0rz