zilla icon indicating copy to clipboard operation
zilla copied to clipboard

Integrate OpenJMH for per-binding microbenchmarks

Open jfallows opened this issue 2 years ago • 3 comments

Today we have an OpenJMH microbenchmark BufferBM for the internal ring buffer data structure used by all zilla bindings.

However, this does not measure the code executed in reaction to reading a frame from the ring buffer, such as the dispatch logic in the runtime engine, nor any binding specific code needed to implement a specific protocol decoder for example.

Therefore, we need to provide an integration with OpenJMH that will make it straightforward to write microbenchmarks for each binding, driving frames into the ring buffer (and consuming any reply frames), measuring the performance of each binding in isolation.

As we make performance improvements to the codebase, we should see these benchmarks reflect the improvements, perhaps for a specific binding, or for all bindings if the improvement is made generically in the engine.

In a similar fashion to the integration with k3po, we can implement the integration with OpenJMH in engine test codebase, then reuse that from each binding to run binding-specific microbenchmarks.

We'll need to make decisions on how to define client and server behavior on either side of the binding, how to drive the load, how to record measured performance, and how to report measured performance.

jfallows avatar Feb 15 '23 02:02 jfallows