Exception trying to instrument a .NET Framework assembly that uses SqlClient

[sharpjs]

trafficstars

Package: all — collector, msbuild, and console Version: 3.1.2 Target: net48

Description

In a .NET Framework project using a type from the System.Data.SqlClient namespace, coverlet attempts to instrument an assembly with that name. That fails with an AssemblyResolutionException because in .NET Framework, that namespace is provided by System.Data.dll instead. The instrumentation failure results in zero coverage being reported for the project.

Steps to Reproduce

Minimal reproduction here: https://github.com/ogsys/CoverletBugRepro

Sample Error

TryWithCustomResolverOnDotNetCore for System.Data.SqlClient, Version=0.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Unable to instrument module: MyLib.Tests\bin\Debug\net48\MyLib.dll
Coverlet.Core.Exceptions.CecilAssemblyResolutionException: AssemblyResolutionException for 'System.Data.SqlClient, Version=0.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Try to add <PreserveCompilationContext>true</PreserveCompilationContext> to test projects </PropertyGroup> or pass '/p:CopyLocalLockFileAssemblies=true' option to the 'dotnet test' command-line ---> Mono.Cecil.AssemblyResolutionException: Failed to resolve assembly: 'System.Data.SqlClient, Version=0.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
   --- End of inner exception stack trace ---
   at Coverlet.Core.Instrumentation.NetstandardAwareAssemblyResolver.TryWithCustomResolverOnDotNetCore(AssemblyNameReference name) in /_/src/coverlet.core/Instrumentation/CecilAssemblyResolver.cs:line 213
   at Coverlet.Core.Instrumentation.NetstandardAwareAssemblyResolver.Resolve(AssemblyNameReference name) in /_/src/coverlet.core/Instrumentation/CecilAssemblyResolver.cs:line 125
   at Mono.Cecil.MetadataResolver.Resolve(TypeReference type)
   at Mono.Cecil.ModuleDefinition.Resolve(TypeReference type)
   at Mono.Cecil.TypeReference.Resolve()
   at Mono.Cecil.Mixin.CheckedResolve(TypeReference self)
   at Mono.Cecil.MetadataBuilder.GetConstantType(TypeReference constant_type, Object constant)
   at Mono.Cecil.MetadataBuilder.AddConstant(IConstantProvider owner, TypeReference type)
   at Mono.Cecil.MetadataBuilder.AddParameter(UInt16 sequence, ParameterDefinition parameter, ParamTable table)
   at Mono.Cecil.MetadataBuilder.AddParameters(MethodDefinition method)
   at Mono.Cecil.MetadataBuilder.AddMethod(MethodDefinition method)
   at Mono.Cecil.MetadataBuilder.AddMethods(TypeDefinition type)
   at Mono.Cecil.MetadataBuilder.AddType(TypeDefinition type)
   at Mono.Cecil.MetadataBuilder.AddTypes()
   at Mono.Cecil.MetadataBuilder.BuildTypes()
   at Mono.Cecil.MetadataBuilder.BuildModule()
   at Mono.Cecil.MetadataBuilder.BuildMetadata()
   at Mono.Cecil.ModuleWriter.<>c.<BuildMetadata>b__2_0(MetadataBuilder builder, MetadataReader _)
   at Mono.Cecil.ModuleDefinition.Read[TItem,TRet](TItem item, Func`3 read)
   at Mono.Cecil.ModuleWriter.BuildMetadata(ModuleDefinition module, MetadataBuilder metadata)
   at Mono.Cecil.ModuleWriter.Write(ModuleDefinition module, Disposable`1 stream, WriterParameters parameters)
   at Mono.Cecil.ModuleWriter.WriteModule(ModuleDefinition module, Disposable`1 stream, WriterParameters parameters)
   at Mono.Cecil.ModuleDefinition.Write(Stream stream, WriterParameters parameters)
   at Coverlet.Core.Instrumentation.Instrumenter.InstrumentModule() in /_/src/coverlet.core/Instrumentation/Instrumenter.cs:line 335
   at Coverlet.Core.Instrumentation.Instrumenter.Instrument() in /_/src/coverlet.core/Instrumentation/Instrumenter.cs:line 145
   at Coverlet.Core.Coverage.PrepareModules() in /_/src/coverlet.core/Coverage.cs:line 135
Test run for D:\Projects\OGsys\CoverletBugRepro\MyLib.Tests\bin\Debug\net48\MyLib.Tests.dll (.NETFramework,Version=v4.8)
Microsoft (R) Test Execution Command Line Tool Version 17.1.0
Copyright (c) Microsoft Corporation.  All rights reserved.
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Ascending
Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 10 ms - MyLib.Tests.dll (net48)

Calculating coverage result...
  Generating report 'D:\Projects\OGsys\CoverletBugRepro\coverage.json'
+--------+------+--------+--------+
| Module | Line | Branch | Method |
+--------+------+--------+--------+

+---------+------+--------+--------+
|         | Line | Branch | Method |
+---------+------+--------+--------+
| Total   | 0%   | 0%     | 0%     |
+---------+------+--------+--------+
| Average | 0%   | 0%     | 0%     |
+---------+------+--------+--------+

[MarcoRossignoli]

Thanks for reporting this.

Package: all — collector, msbuild, and console

if you run the dotnet tool you don't need collector/msbuild dependency. But it won't change the result unfortunately.

Current unique workaround is to copy the missing lib close to the test container(System.Data.SqlClient) we have some scenario where we're not able to locate/resolve all assemblies locations, Cecil needs to load all dependencies to instrument the dlls and sometime they're not where are expected to be, and this is more frequent on netXX scenarios.

This is the current state for this known issue https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/KnownIssues.md#failed-to-resolve-assembly-during-instrumentation

[sharpjs]

Unfortunately, the workarounds fail in this case.

There is a System.Data.SqlClient NuGet package. For .NET Framework, the package DLL is just a set of TypeForwardedTo attributes pointing to the framework-provided implementation (in System.Data.dll). As I understand it, the .NET Framework builds of the package exist to simplify netstandard and cross-targeting projects: one can just put the <PackageReference> in one's .csproj and everything works, even on .NET Framework. .NET-Framework-only projects generally do not need or use the package, as SqlClient is baked in to the framework itself.

Adding the package to MyLib.csproj in the repro project:

  <ItemGroup>
    <PackageReference Include="System.Data.SqlClient" Version="4.8.3" />
  </ItemGroup>

...results in the coverlet run hanging until cancelled with CTRL-C.

The same behaviour occurs without the <PackageReference> if I manually copy the DLL from $HOME\.nuget\packages\system.data.sqlclient\4.8.3\lib\net461 to the location of the test DLL. Coverlet hangs until cancelled with CTRL-C.

Note that the none of the built DLLs reference System.Data.SqlClient.dll, so I am unsure why coverlet is trying to resolve it. Perhaps there is an incorrect assumption somewhere (maybe in one of coverlet's dependencies) that because coverlet is running in .NET Core/5+, then SqlClient types should be resolved from their .NET Core/5+ location. This would be an incorrect assumption in the case of a .NET Framework DLL.

Here are the DLLs in the output folder along with their references.

MyLib.Tests.dll:
- mscorlib
- MyLib
- nunit.framework
- System.Data

MyLib.dll:
- mscorlib
- System.Data

nunit.framework.dll:
- mscorlib
- System
- System.Core
- System.Web
- System.Windows.Forms
- System.Xml
- kernel32.dll
- Kernel32.dll
- libc
- user32.dll

NUnit3.TestAdapter.dll:
- mscorlib
- nunit.engine
- nunit.engine.api
- System
- System.Core
- System.Xml
- System.Xml.Linq
- testcentric.engine.metadata

nunit.engine.dll:
- mscorlib
- nunit.engine.api
- nunit.engine.core
- System
- System.Runtime.Remoting
- System.Xml
- testcentric.engine.metadata

nunit.engine.core.dll:
- mscorlib
- nunit.engine.api
- System
- System.Runtime.Remoting
- System.Web
- System.Xml
- testcentric.engine.metadata

nunit.engine.api.dll:
- mscorlib
- System.Xml

testcentric.engine.metadata.dll:
- mscorlib

Microsoft.VisualStudio.CodeCoverage.Shim.dll:
- mscorlib

[naveenxy]

just install system.data.sqlclient package in your test project, works for me..

[Bertk]

coverlet.console runs successfully using the latest coverlet.console version and updating MyLib.csproj, MyLib.Tests.csproj.

C:\GitHub\bug-repo\CoverletBugRepro-main>coverlet MyLib.Tests\bin\Debug\net48\MyLib.Tests.dll --target "dotnet" --targetargs "test MyLib.Tests\MyLib.Tests.csproj" --verbosity detailed
Instrumented module: 'MyLib.Tests\bin\Debug\net48\MyLib.dll'
  Determining projects to restore...
  Restored C:\GitHub\bug-repo\CoverletBugRepro-main\MyLib.Tests\MyLib.Tests.csproj (in 310 ms).
  Restored C:\GitHub\bug-repo\CoverletBugRepro-main\MyLib\MyLib.csproj (in 310 ms).
  MyLib -> C:\GitHub\bug-repo\CoverletBugRepro-main\MyLib\bin\Debug\net48\MyLib.dll
  MyLib.Tests -> C:\GitHub\bug-repo\CoverletBugRepro-main\MyLib.Tests\bin\Debug\net48\MyLib.Tests.dll
Test run for C:\GitHub\bug-repo\CoverletBugRepro-main\MyLib.Tests\bin\Debug\net48\MyLib.Tests.dll (.NETFramework,Version=v4.8)
Microsoft (R) Test Execution Command Line Tool Version 17.8.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Ascending
Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 15 ms - MyLib.Tests.dll (net48)

Calculating coverage result...
Hits file:'C:\Users\bertk\AppData\Local\Temp\MyLib_251b6694-c256-477d-9176-8910c5247221' not found for module: 'MyLib'
  Generating report 'C:\GitHub\bug-repo\CoverletBugRepro-main\coverage.json'
+--------+------+--------+--------+
| Module | Line | Branch | Method |
+--------+------+--------+--------+
| MyLib  | 0%   | 100%   | 0%     |
+--------+------+--------+--------+

+---------+------+--------+--------+
|         | Line | Branch | Method |
+---------+------+--------+--------+
| Total   | 0%   | 100%   | 0%     |
+---------+------+--------+--------+
| Average | 0%   | 100%   | 0%     |
+---------+------+--------+--------+


C:\GitHub\bug-repo\CoverletBugRepro-main>

MyLib.csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net48</TargetFramework>
    <LangVersion>10</LangVersion>
    <Nullable>enable</Nullable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.Data.SqlClient" Version="5.1.5" />
  </ItemGroup>

</Project>

Foo.cs

using System;
using Microsoft.Data.SqlClient;

namespace MyLib;

public static class Foo
{
    public static void Bar(SortOrder order = SortOrder.Ascending)
    {
        Console.WriteLine(order);
    }
}

MyLib.Tests.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net48</TargetFramework>
    <LangVersion>10</LangVersion>
    <Nullable>enable</Nullable>
    <RootNamespace>MyLib</RootNamespace>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.9.0" />
    <PackageReference Include="NUnit" Version="4.0.1" />
    <PackageReference Include="NUnit3TestAdapter" Version="4.5.0" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\MyLib\MyLib.csproj" />
  </ItemGroup>

</Project>

[sharpjs]

Unfortunately, @Bertk, your example uses the wrong SqlClient. Your code references the Microsoft.Data.SqlClient NuGet package, but the problem occurs with the System.Data.SqlClient namespace that ships with .NET Framework. They are not the same.

The problem is still present in the latest Coverlet with System.Data.SqlClient. I have updated the minimal repro with latest dependencies so that you can verify this for yourself.

Please reopen the issue.

[Bertk]

System.Data.SqlClient has been replaced by Microsoft.Data.SqlClient

Please tell me why are you still using System.Data.SqlClient?

[sharpjs]

Large legacy app. It uses libraries (including Entity Framework 6) that do not support the newer SqlClient.