RunAllJoined should consider baseline per type

[khellang]

trafficstars

If you have something like this

public class Program
{
    public static void Main(string[] args)
    {
        BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).RunAllJoined();
    }

    private static readonly Guid GuidValue = Guid.NewGuid();

    private static readonly string StringValue = GuidValue.ToString();

    [MemoryDiagnoser]
    public class Formatting
    {
        [Benchmark(Baseline = true)]
        public string Old()
        {
            return CompactGuidOld.ToCompactString(GuidValue);
        }

        [Benchmark]
        public string New()
        {
            return CompactGuid.ToString(GuidValue.ToUuid());
        }
    }

    [MemoryDiagnoser]
    public class Parsing
    {
        [Benchmark(Baseline = true)]
        public Guid Old()
        {
            CompactGuidOld.TryParse(StringValue, out var result);
            return result;
        }

        [Benchmark]
        public Guid New()
        {
            CompactGuid.TryParse(StringValue, out var result);
            return result.ToGuid();
        }
    }
}

I would expect my results to show both baselines (Formatting.Old and Parsing.Old) and the other benchmarks in the same type scaled based on each baseline.

Instead, the report only shows one baseline and the rest scaled based on that:

Type	Method	Mean	Error	StdDev	Median	Ratio	RatioSD	Gen 0	Gen 1	Gen 2	Allocated
Formatting	Old	109.65 ns	2.5403 ns	4.1022 ns	109.02 ns	1.00	0.00	0.0191	-	-	80 B
Parsing	Old	17.03 ns	0.2582 ns	0.2416 ns	17.11 ns	0.15	0.01	-	-	-	-
Formatting	New	100.32 ns	2.2110 ns	6.2722 ns	98.11 ns	0.97	0.06	0.0191	-	-	80 B
Parsing	New	24.45 ns	0.5010 ns	0.4184 ns	24.44 ns	0.22	0.01	-	-	-	-

Is this expected? Wouldn't it be better to group one baseline per type? This seems semi-related to https://github.com/dotnet/BenchmarkDotNet/issues/617, but that seems to be about categories.

[khellang]

Oh, and this is running v0.11.5 😄

[AndreyAkinshin]

@khellang thanks for the report!