BenchmarkDotNet icon indicating copy to clipboard operation
BenchmarkDotNet copied to clipboard

Unsure how to assign Baseline when using Arguments

Open JakenVeina opened this issue 1 year ago • 3 comments

I've got the following setup for a benchmark fixture:

public class MyBenchmarks
{
    [Params(1, 2, 3)]
    public int SomeParameter { get; set; }
    
    [Benchmark(Baseline = true)]
    public void ScenarioA();

    [Benchmark]
    [Arguments(4)]
    [Arguments(5)]
    public void ScenarioB(int someArgument);
}

I was taken aback when the benchmarks spit out...

Method SomeParameter someArgument Ratio
ScenarioA 1 ? 1.00
ScenarioB 1 4 ?
ScenarioB 1 5 ?
ScenarioA 2 ? 1.00
ScenarioB 2 4 ?
ScenarioB 2 5 ?
ScenarioA 3 ? 1.00
ScenarioB 3 4 ?
ScenarioB 3 5 ?

What I was going for, and had before I added someArgument was...

Method SomeParameter someArgument Ratio
ScenarioA 1 ? 1.00
ScenarioB 1 4 0.50
ScenarioB 1 5 0.50
ScenarioA 2 ? 1.00
ScenarioB 2 4 0.50
ScenarioB 2 5 0.50
ScenarioA 3 ? 1.00
ScenarioB 3 4 0.50
ScenarioB 3 5 0.50

Makes sense, in retrospect, I suppose. I could achieve this by just splitting a ScenarioC method off of ScenarioB, and getting rid of someParameter. Is there a simpler way to get these to group up?

JakenVeina avatar Jul 24 '24 06:07 JakenVeina