BenchmarkDotNet
BenchmarkDotNet copied to clipboard
Ability to specify benchmark description in outputs
Hello, I think I've done this recomendations: https://github.com/dotnet/BenchmarkDotNet/pull/1651#issuecomment-1645092070
If you have some other recomendations I am open to them.
Closes #1447 Supersedes #1651
For the approval tests, please compare 2 types, 1 with
BenchmarkDescription
and 1 without. The table should look something like this:Type | Method | Job | Mean | Error | StdDev | Rank | LogicalGroup | Baseline | ----------------------------------- |------- |----- |---------:|--------:|--------:|-----:|------------- |--------- | JobBaseline_NoRenameJob_MethodsJobs | Base | Job1 | 102.0 ns | 6.09 ns | 1.58 ns | 1 | * | No | JobBaseline_NoRenameJob_MethodsJobs | Foo | Job1 | 202.0 ns | 6.09 ns | 1.58 ns | 2 | * | No | JobBaseline_NoRenameJob_MethodsJobs | Bar | Job1 | 302.0 ns | 6.09 ns | 1.58 ns | 3 | * | No | MyRenamedTestCase | Base | Job1 | 102.0 ns | 6.09 ns | 1.58 ns | 4 | * | No | MyRenamedTestCase | Foo | Job1 | 202.0 ns | 6.09 ns | 1.58 ns | 5 | * | No | MyRenamedTestCase | Bar | Job1 | 302.0 ns | 6.09 ns | 1.58 ns | 6 | * | No |
If I understand clearly, I need to add Type column to the table if class has BenchmarkDescription attribute with description, right? How can I change result table?
@microsoft-github-policy-service agree
If I understand clearly, I need to add Type column to the table if class has BenchmarkDescription attribute with description, right? How can I change result table?
It will be added automatically if more than 1 type is benchmarked and they are joined together. @AndreyAkinshin Can you help with how to setup the approval tests for that?
It will be added automatically if more than 1 type
I tried to change MockReport.CreateSummary, so method could get an array of types, but it would require too many changes, so I refused from it. How exactly do I need to send more than one type to the summary?
It will be added automatically if more than 1 type
I tried to change MockReport.CreateSummary, so method could get an array of types, but it would require too many changes, so I refused from it. How exactly do I need to send more than one type to the summary?
I think such changes are necessary. Please go ahead and add that functionality.
It will be added automatically if more than 1 type
I tried to change MockReport.CreateSummary, so method could get an array of types, but it would require too many changes, so I refused from it. How exactly do I need to send more than one type to the summary?
I think such changes are necessary. Please go ahead and add that functionality.
Ok, but I tried to create new benchmark, like with instrument. I created 2 classes, both with benchmarked methods, but results were divided into 2 files, like it is 2 separate runs for benchmarking. Also I didn't find any such verified files in the tests/BenchmarkDotNet.Tests/Exporters/VerifiedFiles. Could you please give me the example of the test, which gave you that one result higher or explain, how can I put results of two methods from different class to one result table?
About the benchmarked classes, sending you the code:
internal class Program
{
static void Main(string[] args)
{
var results = BenchmarkRunner.Run<Demo>();
var results2 = BenchmarkRunner.Run<Demo2>();
}
}
public class Demo
{
[Benchmark]
public string GetFullStringNormally()
{
string output = "";
for (int i = 0; i < 100; i++)
output+=i;
return output;
}
}
public class Demo2
{
[Benchmark]
public string GetFullStringNormally2() {
string output = "";
for (int i = 0; i < 100; i++)
output += i;
return output;
}
}
@Nepp3r Try this
BenchmarkSwitcher.FromTypes(new[] { typeof(Demo), typeof(Demo2) }).RunAllJoined(args: args);
I actually think we should just use System.ComponentModel.DescriptionAttribute
instead of creating a new BenchmarkDescriptionAttribute
. We also could use it for [Params]
fields/properties (doesn't need to be done in this PR).