kotlinx-benchmark icon indicating copy to clipboard operation
kotlinx-benchmark copied to clipboard

Add an option to pass arguments to a runtime executing benchmarks

Open fzhinkin opened this issue 1 year ago • 1 comments
trafficstars

Currently, there's no way to configure a set of options to pass to a runtime when executing benchmarks.

There are several use cases when it could be useful:

  • running benchmarks with different VM options to check how it affects performance or to validate some hypothesis regarding optimizations performed by a runtime;
  • pass options to extract various debugging information that may help analyze benchmarking results (for instance, pass --trace-turbo to nodejs/d8).

For JVM, the lack of such an option is not a big deal - one may take and use the generated jar file directly, but for other targets, like wasmJs, the process of running benchmarks directly, without using gradle commands, is much more painful.

One can always add arguments by getting and configuring corresponding tasks (like NodeJsExec), but it would be easier to setup args along with other benchmark-config related options.

I propose adding advanced configuration options to pass options directly to a runtime that executes benchmarks:

benchmark {
    configurations {
        main {
            advanced("jvmArgs", listOf("-XX:TieredStopAtLevel=1"))
            advanced("nodeJsArgs", listOf("--trace-turbo", "--trace-turbo-path=/tmp/"))
        }
    }
}

fzhinkin avatar Jan 15 '24 11:01 fzhinkin

Related issue: https://github.com/Kotlin/kotlinx-benchmark/issues/81.

qurbonzoda avatar May 27 '24 15:05 qurbonzoda