FR: Add ability to ignore/disable benchmarks classes or benchmark methods.

[Kietyo]

Would like an @Ignore or @Disable annotation for this.

[qurbonzoda]

Hi, It is currently possible to include/exclude benchmark classes and methods with benchmark configuration settings. e.g. https://github.com/Kotlin/kotlinx-benchmark/blob/8a96e0d631bf2d813af41650583557167a6fec24/examples/kotlin-multiplatform/build.gradle#L75

See README.md for details.

Please let me know if those settings do not help in your use case.

[Kietyo]

Thanks that works but I would still like to be able to disable certain benchmark methods to avoid re-running methods that I've already ran in the past.

e.g:

@Benchmark
@Ignore
fun benchy() { ... }

[qurbonzoda]

Unfortunately, we are constrained by JMH capabilities. Adding an annotation that JMH does not support would require us to process the annotations within kotlinx-benchmark.

[fzhinkin]

Just curious, what advantages such annotations could give comparing to @Benchmark being removed / commented out?

When it comes to testing, @Ignoreed tests are usually reported as being skipped, which is a valuable outcome. However, there are no such reports for benchmarks.