ScalaPB
ScalaPB copied to clipboard
Benchmarks: run current version, parameters, profiling, structured results
Background
I'm going to introduce numerous changes related to performance of scalapb, but I noticed that existing suite is outdated and using template-based codegen to build suites. Also, I noticed that present results are incomparable because there was no platform information in result structure.
Solution
First, I've made a small local plugin that handles all dependencies for benchmark projects, allowing using either scalapb built from current sources or downloaded from remote ivy repo.
Second, I performed refactoring of suites so they would be actual benchmark classes, possibly annotated with @State and having @Param parameters. Thus added ability to test repeated field with various number of elements.
And lastly, I've made a custom JMH runner to be able to separate results for each benchmark in its own directory + put there profiling information too.
Pro-tip for review
Filter out all .html and .json files that was generated as a result of benchmark runs