testfx
testfx copied to clipboard
microsoft
Possible issue with Microsoft Fakes
A project that uses MSTest.Sdk/3.7.0 and EnableMicrosoftTestingExtensionsFakes set to true has issues execution tests that use fakes. Partial trace output:
Message:
Test method TestMethod1 threw exception:
Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationException: Failed to resolve profiler path from COR_PROFILER_PATH and COR_PROFILER environment variables.
Stack Trace:
IntelliTraceInstrumentationProvider.ResolveProfilerPath()
IntelliTraceInstrumentationProvider.Initialize()
UnitTestIsolationRuntime.InitializeUnitTestIsolationInstrumentationProvider()
ShimRuntime.CreateContext()
Unfortunately, I was not able to create a project to reproduce the issue.
Some additional observations:
- Test project targets .NET Framework 4.8.
- Running EXE directly results in the same error.
- Running the tests using vstest.console.exe has no issues.
- Converting back to non-MSTest.Sdk works as expected.
Looking at the sample the <PlatformTarget>AnyCPU</PlatformTarget> portion of the csproj is causing issues.
When specified, the <RuntimeIdentifier> property is not automatically resolved via the .NET SDK. As a result, NuGet will not restore the necessary native Fakes dlls to the bin folder.
Removing <PlatformTarget>AnyCPU</PlatformTarget> or for instance explicitly setting <RuntimeIdentifier>win-x64</RuntimeIdentifier> makes the NuGet native dll files get copied to the bin folder (such as FakesInstrumentationProfiler_x64.config, Microsoft.QualityTools.Testing.Fakes.dll, Microsoft.QualityTools.Testing.Fakes.Instrumentation_x64.dll, Microsoft.Testing.Extensions.Fakes.dll) and the tests run successfully.
The reason why this is working with vstest.console (non-MSTest.SDK) is that it dynamically loads the Fakes libraries from C:\ProgramFiles, whereas the testing platform is designed to be self-contained and all of the libraries needed to run Fakes are expected to be in the bin folder instead.
@drognanar Thanks for the investigation. Would it be possible to update the Fakes extension doc page?
@evangelink it makes sense to add an FAQ entry as the same issue impacts code coverage, i.e. in this configuration the coverage file generated by running testProject.exe --coverage will be empty, since the code coverage native binaries are not copied into the bin folder.
@drognanar, @Evangelink, looks like the SDK-style projects have problems with .NET 4.8 targeting. If you change the target framework in the sample project to net8.0 it works.
@drognanar, @Evangelink, removing <PlatformTarget>AnyCPU</PlatformTarget> results in another issue: you cannot build for multiple platforms anymore. The reason is that fake assemblies are generated in FakesAssemblies folder. So if I build x64 first, the generated assemblies will be x64, and subsequent build of ARM64 or x86 will fail.
The .NET 48 issue is a NuGet issue. If adding the code coverage extension adds files to the bin folder, but adding the fakes extension does not, then this would be an issue on the fakes extension nuget package instead.
On the other hand the <PlatformTarget></PlatformTarget> is a feature that was not previously supported as the fakes compilation did not support multi architecture solutions and had one path to the fakes dlls regardless of the csproj configuration. I'm not aware of a workaround, but to create multiple projects.
A fix for <PlatformTarget> is currently in the backlog, but currently there were several issues being worked on having a higher priority.
Closing this as this bug currently falls under low priority category. Should the priority change or should there be more asks about adding this feature to fakes, we'll be happy to revisit this task.