System.FormatException in AfterAssemblyLoadingAttached

[aaubry]

When I have a MsBuildArgument whose value ends in a backslash, there is an error somewhere along the intermediate code generation that causes the generated program to receive malformed arguments, resulting in a parsing exception. Here's an example of such:

var summary = BenchmarkRunner.Run<MyBenchmark>(DefaultConfig.Instance
    .AddJob(Job.Default
        .WithArguments(new[]
        {
            new MsBuildArgument(@"/p:a=b\"),
        })
    )
);

Attemptig to run this benchmark causes the following error:

// Validating benchmarks:
// ***** BenchmarkRunner: Start   *****
// ***** Found 1 benchmark(s) in total *****
// ***** Building 1 exe(s) in Parallel: Start   *****
BuildScript: C:\src\Experiments\BenchmarkDotNetArguments\bin\Release\net461\5e1c23e7-7d7a-445f-a530-12b4e2027258.bat
// ***** Done, took 00:00:20 (20.03 sec)   *****
// Found 1 benchmarks:
//   MyBenchmark.DoNothing: Job-EWOGGX(Arguments=/p:a=b\)

// **************************
// Benchmark: MyBenchmark.DoNothing: Job-EWOGGX(Arguments=/p:a=b\)
// *** Execute ***
// Launch: 1 / 1
// Execute: C:\src\Experiments\BenchmarkDotNetArguments\bin\Release\net461\5e1c23e7-7d7a-445f-a530-12b4e2027258.exe --benchmarkName "BenchmarkDotNetArguments.MyBenchmark.DoNothing" --job "Arguments=/p:a=b\" --benchmarkId 0 in
// BeforeAnythingElse

System.FormatException: Input string was not in a correct format.
   at System.Number.StringToNumber(String str, NumberStyles options, NumberBuffer& number, NumberFormatInfo info, Boolean parseDecimal)
   at System.Number.ParseInt32(String s, NumberStyles style, NumberFormatInfo info)
   at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args)
// AfterAll
ExitCode != 0
// Benchmark Process 60508 has exited with code -1
No more Benchmark runs will be launched as NO measurements were obtained from the previous run!

(...)

I was able to pause the porgram before it compiled the .notcs file and add the following line before it parses the arguments:

for(int i = 0; i < args.Length; ++i) System.Console.WriteLine("{0}: '{1}'", i, args[i]);

This outputs the following,

0: '--benchmarkName'
1: 'BenchmarkDotNetArguments.MyBenchmark.DoNothing'
2: '--job'
3: 'Arguments=/p:a=b" --benchmarkId 0'

which confirms that the arguments are not being passed correctly. There's probably some escaping needed somewhere along the toolchain.

A workaround is to add a summy parameter that does not end with a backslash.

[adamsitnik]

Hi @aaubry

Thank you for the detailed bug report.

Would you be interested in sending a PR with a fix?

It should be a matter of fixing the Escape method which is called from here:

https://github.com/dotnet/BenchmarkDotNet/blob/e4d37d03c0b1ef14e7bde224970bd0fc547fd95a/src/BenchmarkDotNet/Running/BenchmarkId.cs#L32

[aaubry]

I'll try. Thanks for indicating the location where the arguments are escaped.

[ghost]

The above PR fixes the backslash issue. Please review and lemme know if it has to be modified. @aaubry @adamsitnik

[postmeback]

Any update on this ??

[adamsitnik]

Any update on this ??

@postmeback It's still up for grabs. You can fork the repo, create a branch, work on a fix and send a PR. If you don't there is no update.

[JobaDiniz]

Is this still happening? I admit I'm pretty new to the codebase, but I tried to run a simple test and it did not throw FormatException.

 public class Issues
    {
        [Fact]
        public void Issue_1536()
        {
            var summary = BenchmarkRunner.Run<MyBenchmark>(DefaultConfig.Instance.AddJob(Job.Default.WithArguments(new[] { new MsBuildArgument(@"/p:a=b\"), })));
        }
    }

BenchmarkDotNet=v0.13.5.20230613-develop, OS=Windows 11 (10.0.22621.1702/22H2/2022Update/SunValley2)
11th Gen Intel Core i7-1185G7 3.00GHz, 1 CPU, 8 logical and 4 physical cores
  [Host]     : .NET Framework 4.8.1 (4.8.9139.0), X64 RyuJIT VectorSize=256 [AttachedDebugger]
  Job-EWOGGX : .NET Framework 4.8.1 (4.8.9139.0), X64 RyuJIT VectorSize=256

Arguments=/p:a=b\  

Method	Mean	Error	StdDev
Foo	100.0 μs	0.02 μs	0.02 μs

[timcassell]

I'm also not able to repro, even on 0.12.1 (the latest version at the time of posting). @aaubry Is it still an issue?