RemoteException When Running Tests via Xamarin Studio on OS X

[topgenorth]

If the Nunit Test Adapter Nuget package is added to a C# project file (.CSPROJ), then tests in the test assembly will fail with a "RemotingException: Unix transport error. See the following screenshot:

This gist has some information from the Xamarin Studio logs about the issue.

Steps to Duplicate:

1. Create a new C# Class Library project in Visual Studio 2013 or Xamarin Studio 5.5.4 or higher.
2. Add the NUnit Test Adapter or NUnit Test Adapter with Framework package from NuGet.
3. Create a simple NUnit TestFixture and Test.
4. Run the test in Visual Studio - notice the test runs fine.
5. Save everything.
6. Open the solution and project in Xamarin Studio 5.5.4 (build 15).
7. Try to run the Nunit tests. You should receive the RemoteException shown in the previous screenshot.

A sample project is available for download here: https://dl.dropboxusercontent.com/u/24991646/XamarinBugzillaScreenshots/NUnitTestAdapterException.zip

[CharliePoole]

Are you expecting the Visual Studio Adapter to somehow work with Xamarin Studio?

[topgenorth]

Is there a way to have the VS Adapter "turn off" in Xamarin Studio? XS doesn't really need it, and it would make sharing the CSPROJ files between the two IDE's easier.

[CharliePoole]

@OsirisTerje: You will have to answer this one.

I notice that the adapter assembly and the nunit util, core and core.interfaces assemblies are going into the lib directory rather than tools. I thought lib was only for assemblies that are referenced by the tests.

[topgenorth]

Interesting - if you add the NUnit VS Adapter through Tools > Extensions and Updates... in Visual Studio then there is no problem - one can run tests in either XS or VS using the same CSPROJ.

If you add the NuGet package then you cannot run the tests in XS - you will experiencing the RemoteException.

Perhaps the simplest solution is to just document this? i.e. the NuGet package is not compatible with XS but the Extension (which I assume is a VSIX) is?

[CharliePoole]

Yes, exactly. Now that we know this, we should at least document it but ideally fix it.

[OsirisTerje]

That makes sense [VSIX], because these are not picked up the same way, as nuget adapter packages are. The nuget dll's should have been under tools, as Charlie said, I don't remember exactly why it was under Libs, but there was some issue at that time. Could you try to manually move the dll's from the project that references them - and un-reference them too ofc, and move them to a seperate folder under the solution root. What I find strange however, is that they are marked as non-copy local, so they are not present in the output directory, so why and how are they picked up?
Anyway, if you try the move first - if that works, it can be made to work by us just moving it under tools instead of libs.

[topgenorth]

Installing the Test Adapter as an extension instead of a package works just as well for me, so that is what I'm going to go with. As handy as NuGet is, I don't have a compelling reason to choose it over the VSIX. Thanks for looking at this.

[CharliePoole]

@topgenorth If you have time to try the experiment @OsirisTerje suggested it would help us to resolve the problem for other folks.

[topgenorth]

@OsirisTerje : I just gave this a try. What I did was, in Xamarin Studio 5.5.4, I removed the references to the assemblies in the old lib directory. Then I moved all the assemblies from lib to the tools directory, opened up Xamarin Studio, and then re-referenced all the DLLs.

Then I tried to run the tests in Xamarin Studio - the problem still exists.

[CharliePoole]

Thanks for trying!

On Mon, Dec 15, 2014 at 10:29 AM, Tom Opgenorth [email protected] wrote:

@OsirisTerje https://github.com/OsirisTerje : I just gave this a try. What I did was, in Xamarin Studio 5.5.4, I removed the references to the assemblies in the old lib directory. Then I moved all the assemblies from lib to the tools directory, opened up Xamarin Studio, and then re-referenced all the DLLs.

Then I tried to run the tests in Xamarin Studio - the problem still exists.

— Reply to this email directly or view it on GitHub https://github.com/nunit/nunit-vs-adapter/issues/57#issuecomment-67040897 .

[OsirisTerje]

Ok, thanks!
I probably wasn't clear enough, but you should NOT try to reference the dll's. Just copy them into a folder below the root. Then the Xamarin test will not find them, as they are NOT referenced, but the VS tester will find them because it scans for the dll's. I think the issue is related to the fact that they are referenced and then being part of what the assembly will try to load, and then when being tried to load it will fail.

[mrward]

Had a quick look at this. The problem seems to be that the assemblies the NUnitTestAdapter NuGet adds to the project as references all have Local Copy set to true if you install the NuGet package with Xamarin Studio. This is not the case if you add the NUnitTestAdapter NuGet package to the project with Visual Studio. This is because the NUnitTestAdapter NuGet package is using a PowerShell script (install.ps1) to set Copy Local to false for these assemblies. Xamarin Studio currently does not support running PowerShell scripts, at least not out of the box.

So right now we have a manual workaround of setting Local Copy to false in Xamarin Studio. Or you can remove the assembly references.

To test this I did the following:

1. In Xamarin Studio 5.8 (build 443) create a new NUnit library project.
2. Add the NUnitTestAdapter NuGet package (1.2).
3. Run the tests and see the "Unix transport error". The error in the Xamarin Studio's IDE log file is that NUnit.Core.RemoteTestRunner doesn't implement interface NUnit.Core.TestRunner. This then causes Xamarin Studio's test runner to crash.
4. Build - Clean to remove the assemblies from the output directory.
5. For the assemblies added by the NUnitTestAdapter NuGet package (nunit.core, nunit.core.interfaces, nunit.util and NUnit.VisualStudio.TestAdapter) set Local Copy to false.
6. Re-run the tests and now they work.

If the NUnitTestAdapter NuGet package is added to the project in Visual Studio first then you should be able to share the projects between Xamarin Studio and Visual Studio.

If the assemblies need to be referenced in the project and cannot just be in the tools directory another way to fix this is to might be to have the references defined in a separate MSBuild .targets file in the NuGet package. The Copy Local value could be set correctly in this separate MSBuild .targets file which would be imported into the project. That should work for both Xamarin Studio and Visual Studio, but not sure if it is worth doing since I think the common scenario would be to add the NUnitTestAdapter NuGet package to a project whilst using Visual Studio. In this case the project can be shared with Xamarin Studio and the test runner will work with both IDEs.

[kzu]

For inspiration on how to do this, you can look at the xunit implementation of the VS test adapter.

[CharliePoole]

@OsirisTerje How do you want to deal with this issue? Document only?

[OsirisTerje]

If we can make the nuget package work as a solution package and not as a Project package it should also solve this issue, if I have understood the above correctly. I Guess that is the best approach, so we could raise another issue on V3.0 to work as a solution package, and just add a reference to this.
The dll's don't really need to be referenced, so we could also check that route.