Link Tasks / HTML-Reports (quick and dirty solution - JVM only)

[teenriot]

trafficstars

When you have many benchmarks in a run, it could be difficult to analyze the console output. Also, when the console output is gone, you only have the json reports for later analysis. It would be nice if kotlinx-benchmark had a simple build in mechanism to create persisted human friendly summaries or if there are "end points" that make thirdparty solutions easy.

Thats where JMH Visualizer comes into play. A better solution would be if those visual reports are generated automatically. There is a gradle plugin for this tool, but obviously it doesn't work together with kotlinx-benchmark. So i wrote a "quick and dirty" workaround, which nearly replaces that plugin.

As newbie i wasn't quick writing it and it is more than dirty, since i was not able to link HTML-Report-Generation in a "gradlish" way . To me it looks like benchmark tasks are created on the fly and they have no output. So there is no common way you can link these tasks with other tasks. I think this should / could be improved, not only for the purpose of report generation, but also for other kinds of extended automation.

For those who want to try out what i came up with - (JVM only): Version 1: Messing up your build.gradle.kts Version 2: As Gradle-Plugin in buildSrc (doesn't work with: org.gradle.configuration-cache=true)

The results are linked in console

The output is equal to JMH Visualizer

(i apologize in advance if such tickets are not welcome, since it is more a comment than an issue)

[hakanai]

One of the things which bugs me about that particular reporting tool is that it takes multiple files, but then won't compare the results between them. So I have to click left and right and try to remember the numbers I was seeing each time.

I don't suppose these json files are trivial enough that I can just join the two files together to get one report?

[teenriot]

At the moment i'm working on a self made visualisation that includes some kind of a "comparing" feature. It is build on top of Plotly. When i put it public on Github it will be not a fully maintened project, but since its small enough, it will be a starting point for others by just copying. I'm not able to release a fully maintained project.