Async GlobalSetup does not work with InProcess toolchain

[mstefarov]

https://github.com/dotnet/BenchmarkDotNet/issues/521 added support for async GlobalSetup, but it appears that InProcess toolchain does not wait for the async setup method to finish. Simple reproducer:

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System.Threading.Tasks;

namespace AsyncGlobalSetupRepro
{
    [InProcess]
    public class MyBenchmarkClass
    {
        private string _foo;

        [GlobalSetup]
        public async Task MyAsyncSetup()
        {
            await Task.Delay(1000);
            _foo = "hello";
        }

        [Benchmark]
        public int MyBenchmark() => _foo.Length; // NullReferenceException!
    }

    public class Program
    {
        public static void Main(string[] args)
        {
            BenchmarkRunner.Run(typeof(MyBenchmarkClass));
        }
    }
}

It looks like MyAsyncSetup gets invoked as a regular method here, and then JITting starts immediately without waiting for it to finish: https://github.com/dotnet/BenchmarkDotNet/blob/e7ff4cefcc8d429205a21a76045e82688ee063c5/src/BenchmarkDotNet/Engines/EngineFactory.cs#L28

[corrieriluca]

Hi, I experience the same problem here ! ✋

When using BDN with dotnet/crank, the --inProcess flag is passed to the BDN executable, and my benchmark throws a NullReferenceException. By testing my benchmarks directly with BDN (not through Crank) I realized that my async GlobalSetup method was not awaited/executed when the --inProcess flag is toggled.

[Turnerj]

Yeah, just hit this myself. Looks like #1968 will fix this so hopefully that gets merged in soon.