Visual Studio Test Adapter (NUnit 2) Crashes, Will Not Run or Debug Tests

[CharliePoole]

@DarthGregarious commented on Fri Mar 10 2017

I tried to run tests in Visual Studio 2015 for the first time since the latest adapter extension update. In one of my solutions, it just crashes. I am using NUnit 2.6.4 with the latest Visual Studio adapter and targeting .NET 4.5.

When debugging a test, it starts to run, but it stops abruptly when it says "loading symbols for Mono.Cecil..."

I found an older issue that suggested adding an ".addins" file to the output directory, so I tried that but it didn't help.

Whether "Running" or "Debugging" a test, "Test Output" includes: Dependent Assembly of C:\projects...Tests.dll not found. Can be ignored if not a NUnit project.

Debug Output:

---

@DarthGregarious commented on Fri Mar 10 2017

I should add that I tried a smaller solution, and it works just fine - running and debugging. It still has the "Cannot find or open the PDB file." errors but it does not have the "Dependent Assembly..." warning. I also used the separate NUnit desktop program with no problems.

---

@DarthGregarious commented on Fri Mar 10 2017

NUnit VS Adapter 2.1.0.7 executing tests is started Loading tests from C:\projects\...Tests.dll Dependent Assembly of C:\projects\...Tests.dll not found. Can be ignored if not a NUnit project. NUnit VS Adapter 2.1.0.7 executing tests is finished

[OsirisTerje]

@DarthGregarious Which framework version are you targeting ? Can you please check with the nuget or vsix package enclosed with issue #142 , so that we can see if it is related to Mono.Cecil versions.

[OsirisTerje]

The "Cannot find or open PDB file" issue is the same as we had in the NUnit3 adaper, https://github.com/nunit/nunit3-vs-adapter/issues/276, so I created a corresponding here #144 , and has done the same fix, pushing soon. This issue can be the same, but the exception points to the mono.cecil pdbs, should they have been included too?

[andkvo]

@OsirisTerje I downloaded the VSIX from #142 and tried to debug a test in Visual Studio 2015. It did the same thing. A pile of "Mono.Cecil.AssemblyResolutionException" (looks like one per test) culminating in the abrupt crash.

Let me know if you need me to try anything else.

[OsirisTerje]

Which framework are you targeting? Have you checked that your own dll's have corresponding pdb files ? When it starts the test run, it writes out the version number, you should see 2.1.0.8, did you see that, or 2.1.0.7 ?

[andkvo]

It says 2.1.0.8 now.

[andkvo]

Debug output says "symbols loaded" for all of my DLLs.

[andkvo]

I'm currently targeting .NET 4.5 but based on other issues I read, I tried adjusting it all the way up to 4.6.1 without success.

[OsirisTerje]

You're having something "interesting" in your code base, it is worthwhile to try to get to the root of this. My standard repro, which checks mainly that it works on 3.5, and 4.6, works as it should. Do you have any possibility to extract a small repro of this issue and upload it? There might be something you do that triggers this behaviour and I am very curious about what that can be.

[initram]

I think this could be the same issue that I reported in #147. I got the same error message: dependant assembly not found.

[andkvo]

You might be on to something there. I removed everything except a very simple Assert(true, Is.True)` test from my test project and I was able to debug the test. I added my base "UnitTests" class back and inherited from that, and the NUnit Adapter crashed again. The base class is not in another DLL though.

~~The base class has some set up code. When it is commented out, debugging works. When it is not, it crashes. I am going to keep working through line-by-line until I can isolate the problem.~~

Thanks for the tip!

[initram]

What I did was to download the source of the adapter to build it and get pdb files. With that you can debug the unit test and set break on all exceptions (remember to find and uncheck "only my code" debug setting). With this you can step through the adapter code when it fails.

[andkvo]

OK, I'll give that a shot.

~~I found that the line that's killing the adapter is coming from another DLL: our proprietary ORM library. I know it does all kinds of assembly loading, so it could be something in there causing a problem...~~

[andkvo]

@OsirisTerje I think this project reproduces the problem. Git history of attempting to reproduce is included.

break-nunit-adapter.zip

[andkvo]

The basic setup:

1. A "main" test library (the one you're trying to debug). At least one test extends a base class from the secondary test library (see below).
2. A secondary test library (contains a base class for one of the tests in main that extends a base class in the precompiled shared test library, below)
3. A precompiled shared test library (contains an abstract base class for one of the tests in the secondary library)

[andkvo]

The result: you cannot debug any test in the "main" test library.

[OsirisTerje]

Thanks! That repro was great! With it I can repro what I think is what you mean, let me walk through it:

1. Look at the tests (5) and (6), 5, is run, which is in the OTherTestLibrary, whereas 6, in the TestLibraryMain is crashing. Does this match what you see?

2. If I try to debug the crashed test (6), that fails too, and in the Output debug window I get an extra error which is an AssemblyResolutionException in Mono.Cecil (7).

3. I found that I didnt need the 3rd library (2) (SimulatedPrecompiledCommonLibrary, it failed just as well without that one.

Just to make sure there was nothing other special here, I also created another test repro, with two projects, and a test in both, and the error repro'ed there too.

The interesting thing is that if the Assert in the BaseTest is removed, then it all works.

I know we have had some inheritance issues earlier, but since this work with the 2.0 adapter and not the 2.1.1 adapter, adn we do see a failure reported by Mono.Cecil on loading some assembly, there must be a specific bug here.

I tried the same thing with NUnit3, and there it all worked. It did however, reported the inherited test twice, but that might be ok since there are in fact two instances here.

[andkvo]

Yes, I can confirm everything you stated. I see the same failures, the same exception. I also removed the SimulatedPrecompiledCommonLibrary and still saw a failure.

[initram]

Have you tried the "fix" from my case #147, to see if it solves the issue? It sounds like a very similar issue, if not the same.

[OsirisTerje]

@initram I think you found the issue there, thank you!!! , Mono.Cecil doesn't search the right directories. Based on your finding, I added the directories for the types found into an assembly resolver that the resolve method uses. I pushed up the changes to a branch named Issue143, and will create a pull-request - we'll wait with any merging until it is checked out.

I've enclosed the vsix and the nuget package for testing here. (Btw, I am only using the nuget package for testing, so the vsix is untested) But with the new nuget adapter, I could run all the tests, it also found the intermediate test, and I could debug them.

NUnitTestAdapter-2.1.2-a001 (2).zip

[OsirisTerje]

@DarthGregarious Can you confirm if this case now works on your machine too? I want to be sure about that before attacking the #150

[andkvo]

Yes, I am able to run and debug my tests again using the adapter that you provided here. I believe the issue I described here has been fixed by your changes.

The adapter you provided does not address the "break on exception" problem described in #150 .

[M-Eber]

I have fallen into this discussion. I have a very simple looking test project that uses Moq, NUnit 2.6.4, and the 4.6.2 .NET library. The thing that is weird is we have 3 projects: Project 1 contains a microkernel of our primary tool to avoid to need to load the UI Project 2 contains some test initializations and 2 tests Project 3 contains all our NUnit tests for the application (1800+ tests)

I'm on Visual Studio 2015 Update 3. I'm running it on Windows 10 with all updates applied. I'm logged in with a standard ID and Visual Studio is run under a different Admin ID. The admin ID has no network access allowed as per the latest DoD workstation requirements.

When I installed the NUnit plugin only two of the tests from the second DLL were displayed and nothing from the 3rd DLL. I downloaded the 2.1.2-a001 zip file and installed the vsix file. (The admin login has no network access allowed so Nuget is out) After installing your zip file I'm still experiencing the failure to expose the tests from the 3rd DLL.

[OsirisTerje]

@M-Eber

• We really need a repro project for this. Can you extract out a small repro of your Project3 ?