Jobs can mess up builds with --keepFiles

[timcassell]

trafficstars

Jobs with provided Ids pass along those Ids when modifications are made. CsProj is always generated with the same filename BenchmarkDotNet.Autogenerated.csproj. When --keepFiles is specified, the generated directory uses the job Id instead of a guid.

Combine those factors, and when you try something like this, the builds will be incorrect.

DefaultConfig.Instance
    .AddJob(Job.ShortRun.WithRuntime(RuntimeMoniker.Net70.GetRuntime())) // Id is ShortRun
    .AddJob(Job.ShortRun.WithRuntime(RuntimeMoniker.NativeAot70.GetRuntime())) // Id is ShortRun
    .KeepBenchmarkFiles(true);

Orthogonally, I think the API is bad. Id is generally expected to be a unique identifier (or at least semi-unique), which is clearly not the case here. Ideally, the Id would be generated always internally, and we would only expose setting a Name rather than Id. But of course that would be a hefty breaking change.