.net 6 web api assembly not shown on the coverage list

[andy250]

I have a .net6 C# project containing multiple projects - web apis and class libraries. I am unable to get coverlet to include the main web api assembly in the assemblies list (so no coverage reported at all). While trying to create a minimal repro for this case I was able to nail one other issue which I reported here https://github.com/coverlet-coverage/coverlet/issues/1380.

This allowed me to see coverage for some more class libraries (assemblies) in the report which originally didn't work - I just removed the default params of type System.IO.SearchOption. However the web api project doesn't show up and there are no warnings. The commands I am using:

coverlet .\bin\Debug\net6.0\MyService.UnitTests.dll --target "dotnet"--targetargs "test --no-build"

and also

dotnet test --collect:"XPlat Code Coverage" --settings:coverlet.runsettings --diag:TestResults\test.log and converting the output file to HTML report with

the coverlet.runsettings file:

<?xml version="1.0" encoding="utf-8" ?>
<RunSettings>
    <DataCollectionRunSettings>
        <DataCollectors>
            <DataCollector friendlyName="XPlat Code Coverage">
                <Configuration>
                    <Format>cobertura</Format>
      		    <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies>
                    <Include>[MyService.WebApi*]*</Include>
                    <Exclude>[MyService.UnitTests.*]*</Exclude>
                </Configuration>
            </DataCollector>
        </DataCollectors>
    </DataCollectionRunSettings>
</RunSettings>

There is one error that I can see in the test log:

TpTrace Warning: 0 : 24644, 1, 2022/09/14, 10:07:36.768, 98429180171, vstest.console.dll, TestPluginDiscoverer: Failed to get types from assembly 'Microsoft.Diagnostics.NETCore.Client, Version=0.2.5.801, Culture=neutral, PublicKeyToken=31bf3856ad364e35'. Error: System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types.
Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the file specified.
   at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
   at System.Reflection.Assembly.GetTypes()
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginDiscoverer.GetTestExtensionsFromAssembly[TPluginInfo,TExtension](Assembly assembly, Dictionary`2 pluginInfos, String filePath)
System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. The system cannot find the file specified.
File name: 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'
 ---> System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.
File name: 'Microsoft.Bcl.AsyncInterfaces, Culture=neutral, PublicKeyToken=null'
   at System.Reflection.RuntimeAssembly.InternalLoad(ObjectHandleOnStack assemblyName, ObjectHandleOnStack requestingAssembly, StackCrawlMarkHandle stackMark, Boolean throwOnFileNotFound, ObjectHandleOnStack assemblyLoadContext, ObjectHandleOnStack retAssembly)
   at System.Reflection.RuntimeAssembly.InternalLoad(AssemblyName assemblyName, RuntimeAssembly requestingAssembly, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, AssemblyLoadContext assemblyLoadContext)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
   at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginCache.CurrentDomainAssemblyResolve(Object sender, AssemblyResolveEventArgs args)
   at Microsoft.VisualStudio.TestPlatform.PlatformAbstractions.PlatformAssemblyResolver.AssemblyResolverEvent(Object sender, Object eventArgs)
   at System.Runtime.Loader.AssemblyLoadContext.GetFirstResolvedAssemblyFromResolvingEvent(AssemblyName assemblyName)
   at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingEvent(AssemblyName assemblyName)
   at System.Runtime.Loader.AssemblyLoadContext.ResolveUsingResolvingEvent(IntPtr gchManagedAssemblyLoadContext, AssemblyName assemblyName)

However installing this assembly Microsoft.Bcl.AsyncInterfaces from nuget (both into test project and into SUT project) doesn't make a difference.

Also a lot of these this in datacollector log:

TpTrace Error: 0 : 2976, 1, 2022/09/14, 10:07:37.566, 98437162154, datacollector.dll, The type "Coverlet.Collector.DataCollection.CoverletCoverageCollector" defined in "coverlet.collector.dll" does not have ExtensionUri attribute.

I cannot share the code and creating a minimal repro has not been successful so far (I spent reasonable amount of time trying to make it).

[andy250]

Just to check if theres is sth fundamentally wrong with my solution setup I tried running AltCover - correctly got all assemblies (including the main web api assembly) in the coverage report.

[Cyberboss]

I am also running into this issue. See this run: https://github.com/tgstation/tgstation-server/actions/runs/3074603866

Check the artifact linux-integration-test-coverage-Release-System-MariaDB. Although this runs code from it, the web assembly Tgstation.Server.Host is not included

[Cyberboss]

Here's a copy of my data collector log. This is with both PreserveCompilationContext and CopyLocalLockFileAssemblies set to true.

test.datacollector.22-09-19_10-47-46_42281_1.log

TpTrace Warning: 0 : 20400, 1, 2022/09/19, 10:47:52.529, 1792507324109, datacollector.dll, [coverlet]Unable to instrument module: S:\workspace\tgstation-server\tests\Tgstation.Server.Tests\bin\Debug\net6.0\Tgstation.Server.Host.dll
Coverlet.Core.Exceptions.CecilAssemblyResolutionException: AssemblyResolutionException for 'Microsoft.Extensions.Logging.Abstractions, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. Try to add <PreserveCompilationContext>true</PreserveCompilationContext> to test projects </PropertyGroup> or pass '/p:CopyLocalLockFileAssemblies=true' option to the 'dotnet test' command-line
 ---> Mono.Cecil.AssemblyResolutionException: Failed to resolve assembly: 'Microsoft.Extensions.Logging.Abstractions, Version=6.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'
   --- End of inner exception stack trace ---
   at Coverlet.Core.Instrumentation.NetstandardAwareAssemblyResolver.TryWithCustomResolverOnDotNetCore(AssemblyNameReference name) in /_/src/coverlet.core/Instrumentation/CecilAssemblyResolver.cs:line 213
   at Coverlet.Core.Instrumentation.NetstandardAwareAssemblyResolver.Resolve(AssemblyNameReference name) in /_/src/coverlet.core/Instrumentation/CecilAssemblyResolver.cs:line 125
   at Mono.Cecil.MetadataResolver.Resolve(TypeReference type)
   at Mono.Cecil.ModuleDefinition.Resolve(TypeReference type)
   at Mono.Cecil.TypeReference.Resolve()
   at Mono.Cecil.Mixin.CheckedResolve(TypeReference self)
   at Mono.Cecil.MetadataBuilder.GetConstantType(TypeReference constant_type, Object constant)
   at Mono.Cecil.MetadataBuilder.AddConstant(IConstantProvider owner, TypeReference type)
   at Mono.Cecil.MetadataBuilder.AddField(FieldDefinition field)
   at Mono.Cecil.MetadataBuilder.AddFields(TypeDefinition type)
   at Mono.Cecil.MetadataBuilder.AddType(TypeDefinition type)
   at Mono.Cecil.MetadataBuilder.AddTypes()
   at Mono.Cecil.MetadataBuilder.BuildTypes()
   at Mono.Cecil.MetadataBuilder.BuildModule()
   at Mono.Cecil.MetadataBuilder.BuildMetadata()
   at Mono.Cecil.ModuleWriter.<>c.<BuildMetadata>b__2_0(MetadataBuilder builder, MetadataReader _)
   at Mono.Cecil.ModuleDefinition.Read[TItem,TRet](TItem item, Func`3 read)
   at Mono.Cecil.ModuleWriter.BuildMetadata(ModuleDefinition module, MetadataBuilder metadata)
   at Mono.Cecil.ModuleWriter.Write(ModuleDefinition module, Disposable`1 stream, WriterParameters parameters)
   at Mono.Cecil.ModuleWriter.WriteModule(ModuleDefinition module, Disposable`1 stream, WriterParameters parameters)
   at Mono.Cecil.ModuleDefinition.Write(Stream stream, WriterParameters parameters)
   at Coverlet.Core.Instrumentation.Instrumenter.InstrumentModule() in /_/src/coverlet.core/Instrumentation/Instrumenter.cs:line 335
   at Coverlet.Core.Instrumentation.Instrumenter.Instrument() in /_/src/coverlet.core/Instrumentation/Instrumenter.cs:line 145
   at Coverlet.Core.Coverage.PrepareModules() in /_/src/coverlet.core/Coverage.cs:line 135

[Cyberboss]

I think I've resolved the issue in my case by including a nuget reference to Microsoft.Extensions.Logging.Abstractions. This project is already referenced further down the dependency tree, though.

Neither CopyLocalLockFileAssemblies nor PreserveCompilationContext were required afterwards

[MarcoRossignoli]

Sometime Cecil is not able to find out where the references are and we have some fallback(but not complete i.e. UX frameworks shipped inside SDK). The simplest solution is to "copy" the missing library close to the test module(I know suboptimal). PreserveCompilationContext and CopyLocalLockFileAssemblies are attempt to delegate that "copy" to msbuild but doesn't work in all scenario unfortunately.

I'm thinking to a new option that could solve these edge case where you can specify a list of paths to search to.

Anyway if you're using dotnet test you can use the built-in dynamic coverage --collect:"Code Coverage"(On Windows (x86, x64 and Arm64), Linux (x64) and macOS (x64)) or the dotnet tool https://learn.microsoft.com/en-us/dotnet/core/additional-tools/dotnet-coverage shipped by MS. These tools doesn't use static instrumentation but dynamic that doesn't need to resolve reference to work.

[axelheer]

To reproduce this, just add an extension method with the following signature to the web project:

public static string Narf(this TagBuilder tag, TagRenderMode renderMode = TagRenderMode.Normal)

Note, using the following signature instead works:

public static string Narf(this TagBuilder tag, TagRenderMode renderMode)

InstrumentMethod does some things, which leads to Mono.Cecil resolving a bunch of additional assemblies, which doesn't work at all (in the end)...

Maybe I'll invest a few more hours for an actual fix, but not today.

[github-actions[bot]]

This issue is stale because it has been open for 3 months with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 9 months since being marked as stale.