nunit3-vs-adapter icon indicating copy to clipboard operation
nunit3-vs-adapter copied to clipboard

Published 20 hours ago • verified publisher icon nunit

Regression in handling C++ files

Open phatcher opened this issue 8 years ago • 5 comments

I recently upgraded from 3.4.1 to 3.6.0 and now my native tests are failing in VSTS.

Previously the adapter ignored things it couldn't handle, now it raises an error

Here's the trace execution for 3.4.1 ...

2016-12-08T11:03:58.8952513Z Information: NUnit Adapter 3.4.1.0: Test execution started
2016-12-08T11:03:58.8952513Z 
2016-12-08T11:03:58.8972513Z Information: Running all tests in C:\a\1\s\RM\NativeClient.NativeTest.dll
2016-12-08T11:03:58.8972513Z 
2016-12-08T11:03:59.0432508Z Warning: Assembly not supported: C:\a\1\s\RM\NativeClient.NativeTest.dll
2016-12-08T11:03:59.0432508Z 
2016-12-08T11:03:59.0472514Z Information: NUnit Adapter 3.4.1.0: Test execution complete
2016-12-08T11:03:59.0472514Z

and for 3.6.0

2016-12-08T11:21:00.0859369Z Information: NUnit Adapter 3.6.0.0: Test execution started
2016-12-08T11:21:00.0859369Z 
2016-12-08T11:21:00.2279398Z Information: Running all tests in C:\a\1\s\RM\NativeClient.NativeTest.dll
2016-12-08T11:21:00.2279398Z 
2016-12-08T11:21:00.2729402Z ##[error]Error: Error: Unable to get runner for this assembly. Check installation, including any extensions.
2016-12-08T11:21:00.2789405Z ##[error]
2016-12-08T11:21:00.2789405Z ##[error]Error: BadImageFormatException: Format of the executable (.exe) or library (.dll) is invalid.
2016-12-08T11:21:00.2789405Z ##[error]
2016-12-08T11:21:00.2789405Z Warning: Exception System.BadImageFormatException, Exception thrown executing tests
2016-12-08T11:21:00.2799405Z 
2016-12-08T11:21:00.2799405Z Warning: Format of the executable (.exe) or library (.dll) is invalid.
2016-12-08T11:21:00.2799405Z 
2016-12-08T11:21:00.2799405Z Information: NUnit Adapter 3.6.0.0: Test execution complete

We don't need the test adapter for this one, but the test step in VSTS is searching the packages folder, so I've asked them as well

phatcher avatar Dec 09 '16 11:12 phatcher

I've noticed the same issue on 3.7.0. We have a large project that has some VS native unit tests and some nunit tests. The test adapter fails the build step whenever it encounters any of the the native test dlls with the error: "Error: Unable to get runner for this assembly. Check installation, including any extensions." and the exception "BadImageFormatException: Format of the executable (.exe) or library (.dll) is invalid."

dkalbertson avatar Feb 16 '17 15:02 dkalbertson

@dkalbertson Would that be any unmanaged assembly or is it necessary to install some framework in order to reproduce the result you have?

CharliePoole avatar Feb 17 '17 01:02 CharliePoole

@CharliePoole It's an unmanaged test assembly that was built using the Visual Studio 2015 Visual C++ Native Unit Test project template. It's being executed by a Microsoft Team Services build client. I've pasted the error content from test build step logs below. You can see that the native tests actually ran and passed (from FrameGrabberIVVTests.dll). However, then, NUnit Adapter 3.7.0.0 tried to examine the same native assembly and then threw and errored out. As @phatcher noted, reverting back to 3.4.1 resolved it in the mean time. Thanks for taking a look. Let me know if I can give you any more information. Error log snippet copied below...

2017-02-15T22:59:54.0005358Z Starting test execution, please wait... 2017-02-15T22:59:54.0630390Z Warning: Using Isolation mode to run tests as required by effective Platform:X64 and .Net Framework:Framework40 settings for test run. Use the /inIsolation parameter to suppress this warning. 2017-02-15T22:59:54.9068322Z Passed BaslerCameraTests_WidthAndHeight 2017-02-15T22:59:54.9068322Z Passed BaslerCameraTests_SyncMode 2017-02-15T22:59:54.9068322Z Passed BaslerCameraTests_LinePeriod 2017-02-15T22:59:54.9068322Z Passed BaslerCameraTests_ResetGain 2017-02-15T22:59:54.9068322Z Passed BaslerCameraTests_Exposure 2017-02-15T22:59:54.9068322Z Passed BaslerCameraTests_CameraLinkMode 2017-02-15T22:59:54.9068322Z Passed BaslerCameraTests_BinningMode 2017-02-15T22:59:54.9068322Z Passed DalsaCameraTests_LinePeriod 2017-02-15T22:59:54.9068322Z Passed DalsaCameraTests_ResetGain 2017-02-15T22:59:54.9068322Z Passed DalsaCameraTests_Exposure 2017-02-15T22:59:54.9068322Z Passed DalsaCameraTests_CameraLinkMode 2017-02-15T22:59:54.9068322Z Passed DalsaCameraTests_SyncMode 2017-02-15T22:59:54.9068322Z Passed DalsaCameraTests_WidthAndHeight 2017-02-15T22:59:54.9068322Z Passed DalsaCameraTests_BinningMode 2017-02-15T22:59:54.9068322Z Passed BaslerSerialProtocolTests_ParseSerialFrame_InvalidCheckBlock 2017-02-15T22:59:54.9068322Z Passed BaslerSerialProtocolTests_CalculateBlockCheck_ShouldSucceed 2017-02-15T22:59:54.9068322Z Passed BaslerSerialProtocolTests_ParseSerialFrame_BufferNull 2017-02-15T22:59:54.9068322Z Passed BaslerSerialProtocolTests_BuildFrameTypeFormat_GetNumBytesForAddress_ShouldSucceed 2017-02-15T22:59:54.9068322Z Information: NUnit Adapter 3.7.0.0: Test execution started 2017-02-15T22:59:54.9068322Z 2017-02-15T22:59:54.9068322Z Information: Running all tests in E:\CIAgent3_work\2\s\Source\InVivoVue\FrameGrabberIVVTests\bin\x64\Release\FrameGrabberIVVTests.dll 2017-02-15T22:59:54.9068322Z 2017-02-15T22:59:54.9537096Z ##[error]Error: Error: Unable to get runner for this assembly. Check installation, including any extensions. 2017-02-15T22:59:54.9537096Z ##[error] 2017-02-15T22:59:54.9537096Z ##[error]Error: BadImageFormatException: Format of the executable (.exe) or library (.dll) is invalid. 2017-02-15T22:59:54.9537096Z ##[error] 2017-02-15T22:59:54.9537096Z Warning: Exception System.BadImageFormatException, Exception thrown executing tests 2017-02-15T22:59:54.9537096Z 2017-02-15T22:59:54.9537096Z Warning: Format of the executable (.exe) or library (.dll) is invalid. 2017-02-15T22:59:54.9537096Z 2017-02-15T22:59:54.9537096Z Information: NUnit Adapter 3.7.0.0: Test execution complete

dkalbertson avatar Feb 17 '17 01:02 dkalbertson

@CharliePoole I'm not sure if it's helpful, but I've copied the log snippet from the build step that succeeded after I reverted to version 3.4.1 below...

2017-02-17T01:38:13.3700619Z Starting test execution, please wait... 2017-02-17T01:38:13.4325639Z Warning: Using Isolation mode to run tests as required by effective Platform:X64 and .Net Framework:Framework40 settings for test run. Use the /inIsolation parameter to suppress this warning. 2017-02-17T01:38:14.8701099Z Passed BaslerCameraTests_WidthAndHeight 2017-02-17T01:38:14.8701099Z Passed BaslerCameraTests_SyncMode 2017-02-17T01:38:14.8701099Z Passed BaslerCameraTests_LinePeriod 2017-02-17T01:38:14.8701099Z Passed BaslerCameraTests_ResetGain 2017-02-17T01:38:14.8701099Z Passed BaslerCameraTests_Exposure 2017-02-17T01:38:14.8701099Z Passed BaslerCameraTests_CameraLinkMode 2017-02-17T01:38:14.8857354Z Passed BaslerCameraTests_BinningMode 2017-02-17T01:38:14.8857354Z Passed DalsaCameraTests_LinePeriod 2017-02-17T01:38:14.8857354Z Passed DalsaCameraTests_ResetGain 2017-02-17T01:38:14.8857354Z Passed DalsaCameraTests_Exposure 2017-02-17T01:38:14.8857354Z Passed DalsaCameraTests_CameraLinkMode 2017-02-17T01:38:14.8857354Z Passed DalsaCameraTests_SyncMode 2017-02-17T01:38:14.8857354Z Passed DalsaCameraTests_WidthAndHeight 2017-02-17T01:38:14.8857354Z Passed DalsaCameraTests_BinningMode 2017-02-17T01:38:14.8857354Z Passed BaslerSerialProtocolTests_ParseSerialFrame_InvalidCheckBlock 2017-02-17T01:38:14.8857354Z Passed BaslerSerialProtocolTests_CalculateBlockCheck_ShouldSucceed 2017-02-17T01:38:14.8857354Z Passed BaslerSerialProtocolTests_ParseSerialFrame_BufferNull 2017-02-17T01:38:14.8857354Z Passed BaslerSerialProtocolTests_BuildFrameTypeFormat_GetNumBytesForAddress_ShouldSucceed 2017-02-17T01:38:14.8857354Z Information: NUnit Adapter 3.4.1.0: Test execution started 2017-02-17T01:38:14.8857354Z 2017-02-17T01:38:14.8857354Z Information: Running all tests in E:\CIAgent3_work\2\s\Source\InVivoVue\FrameGrabberIVVTests\bin\x64\Release\FrameGrabberIVVTests.dll 2017-02-17T01:38:14.8857354Z 2017-02-17T01:38:14.9169864Z Warning: Assembly not supported: E:\CIAgent3_work\2\s\Source\InVivoVue\FrameGrabberIVVTests\bin\x64\Release\FrameGrabberIVVTests.dll 2017-02-17T01:38:14.9169864Z 2017-02-17T01:38:14.9169864Z Information: Running all tests in E:\CIAgent3_work\2\s\Source\InVivoVue\IvvUtilitiesIVVTests\bin\Release\IvvUtilitiesIVVTests.dll 2017-02-17T01:38:14.9326119Z 2017-02-17T01:38:15.2451219Z Information: NUnit3TestExecutor converted 1 of 1 NUnit test cases 2017-02-17T01:38:15.2451219Z 2017-02-17T01:38:15.3388749Z Passed BaslerSerialProtocolTests_ParseSerialFrame_BufferTooSmall 2017-02-17T01:38:15.3388749Z Passed BaslerSerialProtocolTests_BuildSerialFrame_ShouldSucceed 2017-02-17T01:38:15.3388749Z Passed BaslerSerialProtocolTests_CalculateBlockCheck_ShouldFail 2017-02-17T01:38:15.3388749Z Passed BaslerSerialProtocolTests_ParseSerialFrame_InvalidStartByte 2017-02-17T01:38:15.3388749Z Passed BaslerSerialProtocolTests_BuildSerialFrame_BufferNull 2017-02-17T01:38:15.3545004Z Passed BaslerSerialProtocolTests_BuildSerialFrame_BufferTooSmall 2017-02-17T01:38:15.3545004Z Passed BaslerSerialProtocolTests_BuildFrameTypeFormat_Write16BitUseBCC 2017-02-17T01:38:15.3545004Z Passed BaslerSerialProtocolTests_ParseSerialFrame_InvalidEndByte 2017-02-17T01:38:15.3545004Z Passed BaslerSerialProtocolTests_BuildFrameTypeFormat_Read32BitNoBCC 2017-02-17T01:38:15.7920144Z Information: NUnit Adapter 3.4.1.0: Test execution complete 2017-02-17T01:38:15.7920144Z 2017-02-17T01:38:15.8232654Z Passed TestMethod

dkalbertson avatar Feb 17 '17 02:02 dkalbertson

The way I'm getting around it now is to have separate test steps for the C# and C++ assemblies and then put a custom path in the C++ test step "Path to Custom Test Adapters" so it doesn't find the NUnit test adapter.

That way we can upgrade to 3.6/3.7 for the rest

phatcher avatar Feb 17 '17 09:02 phatcher