BenchmarkDotNet
BenchmarkDotNet copied to clipboard
Can't override `DefaultConfig.Instance` settings when using custom configs with `IConfigSource` attribute.
I've created benchmark config by using ManualConfig
derived class.
And using this config with [Config(typeof(CustomConfig))]
attribute.
When running benchmarks.
It seems config is always merged with DefaultConfig.Instance
by BenchmarkConverter::GetFullTypeConfig
And it causes unintended settings to be used when running benchmarks.
Minimum code to reproduce problems
internal class Program
{
static void Main(string[] args)
{
BenchmarkRunner.Run<SampleBenchmarks>(config: null, args: args);
}
}
public class CustomBenchmarkConfig : ManualConfig
{
public CustomBenchmarkConfig()
{
AddExporter(MarkdownExporter.GitHub);
}
}
[Config(typeof(CustomBenchmarkConfig))]
public class SampleBenchmarks
{
[Benchmark]
public void Benchmark01()
{
}
}
Warnings Following warnings occurred when running above code.
// * Warnings *
Configuration
Summary -> The exporter MarkdownExporter-github is already present in configuration. There may be unexpected results.
It seems default exporters are configured that are defined by DefaultConfig
.
Temporary workaround
This problem can be avoided by explicitly specifying the config or pass config that is created by ManualConfig.CreateEmpty()
.