Adapter does not detect multiple tests when using arrays in testcase

[CharliePoole]

Test Explorer only shows one test, given the following method.

    [TestCase(5, new int[] { 2, 3 })]
    [TestCase(5, new int[] { 1, 2, 2 })]
    public void Class1Test(int expectedTotal, int[] vals)
    {
        var calc = new Calculator();
        int total = calc.AddNums(vals);
        Assert.AreEqual(expectedTotal, total);
    }

Changing the first int in one of the test cases causes Explorer to show two test:

    [TestCase(5, new int[] { 2, 3 })]
    [TestCase(6, new int[] { 1, 2, 2 })]
    public void Class1Test(int expectedTotal, int[] vals)
    {
        var calc = new Calculator();
        int total = calc.AddNums(vals);
        Assert.AreEqual(expectedTotal, total);
    }

Used "install-package NUnitTestAdapter.WithFramework" to install version 0.96 on Visual Studio 2012 Update 3.

NUnit gui shows two test in both of the above scenarios.

For more info see https://bugs.launchpad.net/nunit-vs-adapter/+bug/1207946

[OsirisTerje]

This is really not a bug, but more the way it should work. There is only one test here - the test method - with multiple test cases given by the TestCase list. In the Test Explorer the one test is given, and in the Test Result window below the test list, -2- results are shown, one for each test case. So both test cases are run.

NUnit adds the parameters to the test name which is used for the test list. This is probably unfortunate, and creates this confusion (changing from 5 to 6 on the first parameter). It would have been better if the test name was equal to the method name only plus the signature of the formal parameters of the test method.

[CharliePoole]

Using NUnit terminology, there are indeed two tests, since NUnit calls each testcase, method, class, namespace or assembly a "test." TestExplorer looks at tests differently, which is the source of the confusion. When viewing the example in the NUnit Gui, two tests are shown.

I think this needs to be fixed, although the fix can't be done in the VS adapter. We'll have to figure out how to show bugs like this in GitHub. Under Launchpad,. we could add the nunit framework project to the bug. The true fix would be in NUnit first and would then be verified using the adapter. Maybe we need a status that says this requires a change in NUnit. OTOH, we don't want to have to keep seeing the same bug listed when it's merely waiting for another project to fix it. Any idea about how to track this?

Charlie

On Tue, Nov 19, 2013 at 10:17 PM, Terje Sandstrom [email protected]:

This is really not a bug, but more the way it should work. There is only one test here - the test method - with multiple test cases given by the TestCase list. In the Test Explorer the one test is given, and in the Test Result window below the test list, -2- results are shown, one for each test case.

NUnit adds the parameters to the test name which is used for the test list. This is probably unfortunate, and creates this confusion (changing from 5 to 6 on the first parameter). It would have been better if the test name was equal to the method name only plus the signature of the formal parameters of the test method.

— Reply to this email directly or view it on GitHubhttps://github.com/nunit/nunit-vs-adapter/issues/5#issuecomment-28866302 .

[OsirisTerje]

We should also look at this together with the Generic inheritance issue, where the tests in the base class comes out as External instead of inside the correct project. They will have a mangled name, which is probably the reason for them being "out-of-project", but it doesn't look good.

Tracking: We should have linking of issues, but that does not exist on GitHub. Could we add a new label type "Link:" with values for each project ? And in the description for the bug, at the top, add the correct Issue # it is linked to. If it waits, it is impeded, which is ok - we could add that as a status, but if it is just waiting for the other fix and don't require any more action, then it should just be closed.

[CharliePoole]

The problem is that even when we don't think we will need any more action, we may be wrong and we have to at least test it. In this case, the NUnit fix is to ensure that every test has a unique name. That should fix the adapter problem, of course, but will we like what we see once we get it? Maybe we could close it and make a note in the NUnit bug to check also under the adapter.

I'm not familiar with the Generic inheritance issue.

Charlie

On Tue, Nov 19, 2013 at 10:39 PM, Terje Sandstrom [email protected]:

We should also look at this together with the Generic inheritance issue, where the tests in the base class comes out as External instead of inside the correct project. They will have a mangled name, which is probably the reason for them being "out-of-project", but it doesn't look good.

Tracking: We should have linking of issues, but that does not exist on GitHub. Could we add a new label type "Link:" with values for each project ? And in the description for the bug, at the top, add the correct Issue # it is linked to. If it waits, it is impeded, which is ok - we could add that as a status, but if it is just waiting for the other fix and don't require any more action, then it should just be closed.

— Reply to this email directly or view it on GitHubhttps://github.com/nunit/nunit-vs-adapter/issues/5#issuecomment-28866988 .

[IoanaKera]

Hi, I have the same problem. Is this issue something you can fix in the next Nunit version or is ot already fixed? Thank you Ioana

[CharliePoole]

This was confusingly labelled as closed:wontfix but not actually closed in github. I changed the label to status:pending and assigned it to the V3.0 milestone. The reason is that we believe the issue will automatically be fixed, without further work, as soon as we have a version of the adapter that uses NUnit 3.0. Of course, if it isn't we may have to do something else.