BenchmarkDotNet icon indicating copy to clipboard operation
BenchmarkDotNet copied to clipboard

RPlotExporter images are not generated when ArtifactsPath is set

Open NinjaCross opened this issue 1 year ago • 0 comments
trafficstars

I have this configuration with version 0.13.12 on NET 8.0:

public class BenchmarkConfig : ManualConfig
    {
        public BenchmarkConfig()
        {
            AddDiagnoser(MemoryDiagnoser.Default);
            AddLogger(ConsoleLogger.Default);
            AddColumn(
                TargetMethodColumn.Method,
                StatisticColumn.Median,
                StatisticColumn.StdDev,
                StatisticColumn.Q1,
                StatisticColumn.Q3);
            
            AddExporter(MarkdownExporter.StackOverflow); 
            AddExporter(JsonExporter.Full);
            AddExporter(RPlotExporter.Default);
            
           ArtifactsPath = "../../../../../"; // THIS PREVENT RPlotExporter TO GENERATE IMAGES
           
        }
    }

When ArtifactsPath contains a relative path like in the sample code:

  • BuildPlots.R is generated into the path specified by ArtifactsPath
  • BuildPlots.log is generated into the path specified by ArtifactsPath, and is empty
  • No PNG file is generate anywhere
  • No errors or warnings are recorded/shown anywhere

This is the output i get in console:

// Benchmark Process 36016 has exited with code 0.

Mean = 1.033 ns, StdErr = 0.004 ns (0.44%), N = 15, StdDev = 0.017 ns
Min = 0.998 ns, Q1 = 1.019 ns, Median = 1.037 ns, Q3 = 1.047 ns, Max = 1.058 ns
IQR = 0.028 ns, LowerFence = 0.977 ns, UpperFence = 1.089 ns
ConfidenceInterval = [1.014 ns; 1.052 ns] (CI 99.9%), Margin = 0.019 ns (1.80% of Mean)
Skewness = -0.33, Kurtosis = 1.77, MValue = 2

// ** Remained 0 (0.0%) benchmark(s) to run. Estimated finish 2024-08-02 13:11 (0h 0m from now) **
Successfully reverted power plan (GUID: 8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c FriendlyName: High performance)
// ***** BenchmarkRunner: Finish  *****

// * Export *
  ../../../../../results\Dnbe.Linq.TrimBenchmark-report.csv
  ../../../../../results\Dnbe.Linq.TrimBenchmark-report-github.md
  ../../../../../results\Dnbe.Linq.TrimBenchmark-report.html
  ../../../../../results\Dnbe.Linq.TrimBenchmark-report-stackoverflow.md
  ../../../../../results\Dnbe.Linq.TrimBenchmark-report-full.json
  ../../../../../results\Dnbe.Linq.TrimBenchmark-measurements.csv
  ../../../../../results\BuildPlots.R
  ../../../../../results\*.png

// * Detailed results *
TrimBenchmark.Trim_NoLeadingOrTrailingSpaces: .NET 8.0(Runtime=.NET 8.0)
Runtime = .NET 8.0.7 (8.0.724.31311), X64 RyuJIT AVX2; GC = Concurrent Workstation
Mean = 1.033 ns, StdErr = 0.004 ns (0.44%), N = 15, StdDev = 0.017 ns
Min = 0.998 ns, Q1 = 1.019 ns, Median = 1.037 ns, Q3 = 1.047 ns, Max = 1.058 ns
IQR = 0.028 ns, LowerFence = 0.977 ns, UpperFence = 1.089 ns
ConfidenceInterval = [1.014 ns; 1.052 ns] (CI 99.9%), Margin = 0.019 ns (1.80% of Mean)
Skewness = -0.33, Kurtosis = 1.77, MValue = 2
-------------------- Histogram --------------------
[0.989 ns ; 1.011 ns) | @
[1.011 ns ; 1.034 ns) | @@@@@@
[1.034 ns ; 1.067 ns) | @@@@@@@@
---------------------------------------------------

// * Summary *

BenchmarkDotNet v0.13.12, Windows 11 (10.0.22631.3880/23H2/2023Update/SunValley3)
13th Gen Intel Core i9-13900K, 1 CPU, 32 logical and 24 physical cores
.NET SDK 8.0.303
  [Host]   : .NET Core 3.1.32 (CoreCLR 4.700.22.55902, CoreFX 4.700.22.56512), X64 RyuJIT AVX2
  .NET 8.0 : .NET 8.0.7 (8.0.724.31311), X64 RyuJIT AVX2

Job=.NET 8.0  Runtime=.NET 8.0

| Method                         | Mean     | Error     | StdDev    | Median   | Q1       | Q3       | Ratio | Allocated | Alloc Ratio |
|------------------------------- |---------:|----------:|----------:|---------:|---------:|---------:|------:|----------:|------------:|
| Trim_NoLeadingOrTrailingSpaces | 1.033 ns | 0.0186 ns | 0.0174 ns | 1.037 ns | 1.019 ns | 1.047 ns |  1.00 |         - |          NA |

// * Legends *
  Mean        : Arithmetic mean of all measurements
  Error       : Half of 99.9% confidence interval
  StdDev      : Standard deviation of all measurements
  Median      : Value separating the higher half of all measurements (50th percentile)
  Q1          : Quartile 1 (25th percentile)
  Q3          : Quartile 3 (75th percentile)
  Ratio       : Mean of the ratio distribution ([Current]/[Baseline])
  Allocated   : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
  Alloc Ratio : Allocated memory ratio distribution ([Current]/[Baseline])
  1 ns        : 1 Nanosecond (0.000000001 sec)

// * Diagnostic Output - MemoryDiagnoser *


// ***** BenchmarkRunner: End *****
Run time: 00:00:36 (36.67 sec), executed benchmarks: 1

Global total time: 00:00:45 (45.08 sec), executed benchmarks: 1
// * Artifacts cleanup *
Artifacts cleanup is finished

Process finished with exit code 0.

If i comment the line ArtifactsPath = "../../../../../";, everything works as expected, and the PNG images are generated.

NinjaCross avatar Aug 02 '24 11:08 NinjaCross