Error in Orleans.TestingHost.TestCluster

[sithwin]

Hi,

Is it possible to do performance test with NBench and Orleans?

I tested base on Olearns sample project HelloWorl but I'm having following exception.

NBench.NBenchException: Error occurred during $SiloTests.HelloWorldSiloTests+Benchmark SETUP. ---> System.MissingMethodException: No parameterless constructor defined for this object. at System.RuntimeTypeHandle.CreateInstance(RuntimeType type, Boolean publicOnly, Boolean noCheck, Boolean& canBeCached, RuntimeMethodHandleInternal& ctor, Boolean& bNeedSecurityCheck) at System.RuntimeType.CreateInstanceSlow(Boolean publicOnly, Boolean skipCheckThis, Boolean fillCache, StackCrawlMark& stackMark) at System.Activator.CreateInstance(Type type, Boolean nonPublic) at System.Activator.CreateInstance(Type type) at NBench.Sdk.ReflectionBenchmarkInvoker.InvokePerfSetup(BenchmarkContext context) at NBench.Sdk.ReflectionBenchmarkInvoker.InvokePerfSetup(Int64 runCount, BenchmarkContext context) at NBench.Sdk.Benchmark.PreRun() --- End of inner exception stack trace --- NBench.NBenchException: Error occurred during $SiloTests.HelloWorldSiloTests+Benchmark RUN. ---> System.NullReferenceException: Object reference not set to an instance of an object. at NBench.Sdk.ReflectionBenchmarkInvoker.InvokeRun(BenchmarkContext context) at NBench.Sdk.Benchmark.WarmUp() --- End of inner exception stack trace --- NBench.NBenchException: Error occurred during $SiloTests.HelloWorldSiloTests+Benchmark CLEANUP. ---> System.NullReferenceException: Object reference not set to an instance of an object. at NBench.Sdk.ReflectionBenchmarkInvoker.InvokePerfCleanup(BenchmarkContext context) at NBench.Sdk.Benchmark.PostRun() --- End of inner exception stack trace ---

Pls give me the idea how to work with Orleans and NBench. Thanks!

[Aaronontheweb]

hi there @sithwin

Could you post a brief code sample of the spec you were trying to run?

[sithwin]

Hi @Aaronontheweb ,

Sorry for the lack of information. The solution name is HellowWorld.SingleProcess and SiloTests is the test project. You could download the whole solution here.

I installed NBench in the test project and tried to run the runner.

Here is my code: `public class HelloWorldSiloTests : IClassFixture<OrleansSiloFixture> { private readonly OrleansSiloFixture _fixture;

    public HelloWorldSiloTests(OrleansSiloFixture fixture)
    {
        _fixture = fixture;
    }

    private IGrainFactory GrainFactory => _fixture.Cluster.GrainFactory;


    private Counter _counter;

    [PerfSetup]
    public void Setup(BenchmarkContext context)
    {
        _counter = context.GetCounter("TestCounter");
    }

    [PerfBenchmark(Description = "Test to ensure that a minimal throughput test can be rapidly executed.",
        NumberOfIterations = 3, RunMode = RunMode.Throughput,
        RunTimeMilliseconds = 1000, TestMode = TestMode.Test)]
    [CounterThroughputAssertion("TestCounter", MustBe.GreaterThan, 10000000.0d)]
    [MemoryAssertion(MemoryMetric.TotalBytesAllocated, MustBe.LessThanOrEqualTo, ByteConstants.ThirtyTwoKb)]
    [GcTotalAssertion(GcMetric.TotalCollections, GcGeneration.Gen2, MustBe.ExactlyEqualTo, 0.0d)]
    public void Benchmark()
    {
        _counter.Increment();
    }

    [PerfCleanup]
    public void Cleanup()
    {
        // does nothing
    }

    [Fact]
    public async Task SiloSayHelloTest()
    {
        // The Orleans silo / client test environment is already set up at this point.

        long id = new Random().Next();
        const string greeting = "Bonjour";

        IHello grain = GrainFactory.GetGrain<IHello>(id);

        // This will create and call a Hello grain with specified 'id' in one of the test silos.
        string reply = await grain.SayHello(greeting);

        Assert.NotNull(reply);
        Assert.Equal($"You said: '{greeting}', I say: Hello!", reply);
    }

    [Fact]
    public async Task SiloSayHelloArchiveTest()
    {
        long id = new Random().Next();
        const string greeting1 = "Bonjour";
        const string greeting2 = "Hei";

        IHelloArchive grain = GrainFactory.GetGrain<IHelloArchive>(id);

        // This will directly call the grain under test.
        await grain.SayHello(greeting1);
        await grain.SayHello(greeting2);

        var greetings = (await grain.GetGreetings()).ToList();

        Assert.Contains(greeting1, greetings);
        Assert.Contains(greeting2, greetings);
    }
}

/// <summary>
/// Class fixture used to share the silos between multiple tests within a specific test class.
/// </summary>
public class OrleansSiloFixture : IDisposable
{
    public TestCluster Cluster { get; }

    public OrleansSiloFixture()
    {
        GrainClient.Uninitialize();

        var options = new TestClusterOptions(initialSilosCount: 2);
        options.ClusterConfiguration.AddMemoryStorageProvider("Default");
        options.ClusterConfiguration.AddMemoryStorageProvider("MemoryStore");
        Cluster = new TestCluster(options);

        if (Cluster.Primary == null)
        {
            Cluster.Deploy();
        }
    }

    /// <summary>
    /// Clean up the test fixture once all the tests have been run
    /// </summary>
    public void Dispose()
    {
        Cluster.StopAllSilos();
    }
}`

Please let me know what wrong with my code.

Thanks a lot!

[Horusiath]

AFAIK, NBench test runner requires perf test class to have a single parameterless constructor.