Microsoft.CodeCoverage 18.0.0 contains system.text.json.dll v6, which gets copied to the output folder

[jeroenjanssen-cpp]

I am getting issues with reqnroll (navigating to steps) which I traced back to a version 6 of System.Text.Json.dll being used (causing serialization issues since v8 is expected)

It seems this v6 of System.Test.Json.dll is getting copied to my output folder from the Microsoft.CodeCoverage 18.0.0 package.

The previous (17.4.1) package worked ok and does not seem to contain the System.Text.Json.dll

[nohwnd]

@engyebrahim please have a look.

[engyebrahim]

Hi @jeroenjanssen-cpp, Coverage settings can be specified in a dedicated coverage settings file, which is a subset of the .runsettings file containing only code coverage-related settings. This file can be provided in either XML or JSON format. The JSON format is supported starting from version 18.0 so it is expected to have the System.Text.Json.dll in the Microsoft.CodeCoverage 18.0.0 package.

[jeroenjanssen-cpp]

I have some actual logging information on the actual failure:

I have some more information on the actual failure:

```txt
Info: OnActivityStarted: Starting Visual Studio Extension...
Info: CreateProjectScope: Initializing project: xxxTest
Info: OnSettingsInitialized: Project settings initialized: .NETCoreApp,Version=v8.0,Reqnroll:2.4.1,x64
Warning: AndDiscoveryProviderSucceed: Error during binding discovery. 
Command executed:
  xxx C:\Program Files\dotnet\dotnet.exe exec c:\users\xxx\appdata\local\microsoft\visualstudio\17.0_09a6698c\extensions\tgizja0l.t4q\Connectors\Reqnroll-Generic-net8.0\reqnroll-vs.dll discovery xxxTest.dll xxx\reqnroll.json
Exit code: 0
Message: 
'M' is an invalid start of a value. Path: $ | LineNumber: 0 | BytePositionInLine: 0.
Method not found: 'Int32 System.Text.Encodings.Web.TextEncoder.FindFirstCharacterToEncodeUtf8(System.ReadOnlySpan`1<Byte>)'.
Unable to deserialize discovery options:  Method not found: 'Int32 System.Text.Encodings.Web.TextEncoder.FindFirstCharacterToEncodeUtf8(System.ReadOnlySpan`1<Byte>)'. ({
  "assemblyFile": "xxx.xSBDTest.dll",
  "configFile": "xxx\\reqnroll.json",
  "connectorFolder": "c:\\users\\xxx\\appdata\\local\\microsoft\\visualstudio\\17.0_09a6698c\\extensions\\tgizja0l.t4q\\Connectors\\Reqnroll-Generic-net8.0"
})
Error System.MissingMethodException: Method not found: 'Int32 System.Text.Encodings.Web.TextEncoder.FindFirstCharacterToEncodeUtf8(System.ReadOnlySpan`1<Byte>)'.
   at System.Text.Json.JsonWriterHelper.NeedsEscaping(ReadOnlySpan`1 value, JavaScriptEncoder encoder)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo.DeterminePropertyName()
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo.GetPolicies(Nullable`1 ignoreCondition, Nullable`1 declaringTypeNumberHandling)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.Initialize(Type parentClassType, Type declaredPropertyType, Type runtimePropertyType, ConverterStrategy runtimeClassType, MemberInfo memberInfo, Boolean isVirtual, JsonConverter converter, Nullable`1 ignoreCondition, Nullable`1 parentTypeNumberHandling, JsonSerializerOptions options)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo.CreateProperty(Type declaredPropertyType, Type runtimePropertyType, MemberInfo memberInfo, Type parentClassType, Boolean isVirtual, JsonConverter converter, JsonSerializerOptions options, Nullable`1 parentTypeNumberHandling, Nullable`1 ignoreCondition)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo.AddProperty(MemberInfo memberInfo, Type memberType, Type parentClassType, Boolean isVirtual, Nullable`1 parentTypeNumberHandling, JsonSerializerOptions options)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo.CacheMember(Type declaringType, Type memberType, MemberInfo memberInfo, Boolean isVirtual, Nullable`1 typeNumberHandling, Boolean& propertyOrderSpecified, Dictionary`2& ignoredMembers)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo..ctor(Type type, JsonConverter converter, Type runtimeType, JsonSerializerOptions options)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo..ctor(Type type, JsonSerializerOptions options)
   at System.Text.Json.JsonSerializerOptions.<InitializeForReflectionSerializer>g__CreateJsonTypeInfo|112_0(Type type, JsonSerializerOptions options)
   at System.Text.Json.JsonSerializerOptions.GetClassFromContextOrCreate(Type type)
   at System.Text.Json.JsonSerializerOptions.GetOrAddClass(Type type)
   at System.Text.Json.JsonSerializer.GetTypeInfo(JsonSerializerOptions options, Type runtimeType)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
   at ReqnrollConnector.JsonSerialization.DeserializeDiscoveryOptions(String json, ILogger log) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\JsonSerialization.cs:line 77


DiscoveryOptions { DebugMode = False, AssemblyFile = xxxTest.dll, ConfigFile = xxx\reqnroll.json, ConnectorFolder = c:\users\xxx\appdata\local\microsoft\visualstudio\17.0_09a6698c\extensions\tgizja0l.t4q\Connectors\Reqnroll-Generic-net8.0 }
Loading xxxTest.dll
xxxTest, Version=99.99.0.0, Culture=neutral, PublicKeyToken=null loaded
Microsoft.Extensions.DependencyModel.DependencyContext loaded
Loading System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Loading System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Loading System.Runtime.Loader, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Loading System.Collections.Immutable, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Loading System.Text.Json, Version=8.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
Resolved with Microsoft.Extensions.DependencyModel.Resolution.AppBaseCompilationAssemblyResolver from xxx\System.Text.Json.dll
Found runtime library:System.Text.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
Loading netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
Loading System.Text.Encodings.Web, Version=8.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
Loading System.Text.Encodings.Web, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
Loading System.Memory, Version=4.0.1.1, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
Resolved with Microsoft.Extensions.DependencyModel.Resolution.AppBaseCompilationAssemblyResolver from xxx\System.Memory.dll
Found runtime library:System.Memory, Version=4.0.1.1, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
Loading netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
Loading Microsoft.Bcl.AsyncInterfaces, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
Resolved with Microsoft.Extensions.DependencyModel.Resolution.AppBaseCompilationAssemblyResolver from xxx\Microsoft.Bcl.AsyncInterfaces.dll
Found runtime library:Microsoft.Bcl.AsyncInterfaces, Version=8.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
Loading netstandard, Version=2.1.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
Loading System.Runtime.CompilerServices.Unsafe, Version=4.0.4.1, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Loading Reqnroll.VisualStudio.ReqnrollConnector.Models, Version=2025.1.0.0, Culture=neutral, PublicKeyToken=611ce36403091019
System.Text.Json.JsonException: 'M' is an invalid start of a value. Path: $ | LineNumber: 0 | BytePositionInLine: 0.
 ---> System.Text.Json.JsonReaderException: 'M' is an invalid start of a value. LineNumber: 0 | BytePositionInLine: 0.
   at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan`1 bytes)
   at System.Text.Json.Utf8JsonReader.ConsumeValue(Byte marker)
   at System.Text.Json.Utf8JsonReader.ReadFirstToken(Byte first)
   at System.Text.Json.Utf8JsonReader.ReadSingleSegment()
   at System.Text.Json.Utf8JsonReader.Read()
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   --- End of inner exception stack trace ---
   at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, JsonReaderException ex)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo`1 jsonTypeInfo, Nullable`1 actualByteCount)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo`1 jsonTypeInfo)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
   at ReqnrollConnector.JsonSerialization.DeserializeObjectRunnerResult(String json, ILogger log) in D:\a\Reqnroll.VisualStudio\Reqnroll.VisualStudio\Connectors\Reqnroll.VisualStudio.ReqnrollConnector.Generic\JsonSerialization.cs:line 108
Warning: AndDiscoveryProviderSucceed: The project bindings (e.g. step definitions) could not be discovered. Navigation, step completion and other features are disabled. 
  Please check the error message above and report to https://github.com/reqnroll/Reqnroll.VisualStudio/issues if you cannot fix.

This is a .NET 8 project using reqnroll

The .NET 6 version of System.Text.Json.dll seems to be coming from the Microsoft.CodeCoverage 18.0.0 package, see .nuget\packages\microsoft.codecoverage\18.0.0\build\netstandard2.0:

in this case it would probably not have been a problem if the package included a .NET 8 version of this package

it seems the CopyTraceDataCollectorArtifacts target at https://github.com/microsoft/vstest/blob/81d120f85fbdddc7b981e910a127bd6ef913cd42/src/package/Microsoft.CodeCoverage/Microsoft.CodeCoverage.targets#L17 copies the codecoverage files the output folder in case of publishing, overwriting any newer (e.g. 8.x) version of System.Text.Json.dll

[jvgarner]

This also appears to be causing issues with my build pipeline. Self-hosted Azure Devops with build agents, target framework net8.0. I get the following exception in my test task:

Data collector 'Code Coverage' message: Data collector caught an exception of type 'System.IO.FileNotFoundException': 'Could not load file or assembly 'System.Text.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The system cannot find the file specified.'.

Reverting to 17.14.1 resolves the issue for me.

[engyebrahim]

@jeroenjanssen-cpp Can you please add steps for repro? and if you can attach a sample project that would be great!

[Youssef1313]

@jeroenjanssen-cpp Also, how are you invoking the tests please? Are you doing first dotnet publish then dotnet test/vstest <path to dll>?

[jeroenjanssen-cpp]

Sorry, I didn't have time yet to extract a reproduction case, I will try to do that this week

please note that the command that is executed xxx C:\Program Files\dotnet\dotnet.exe exec c:\users\xxx\appdata\local\microsoft\visualstudio\17.0_09a6698c\extensions\tgizja0l.t4q\Connectors\Reqnroll-Generic-net8.0\reqnroll-vs.dll discovery xxxTest.dll xxx\reqnroll.json

is something that reqnroll does when you try to navigate to a step implementation from a feature file. so it tries to load the test assembly (and uses whatever assemblies are in the output folder)

I was under the impression that dotnet v6 is not supported anymore, is there a specific reason that System.Test.Json.dll v6 is included in the Microsoft.CodeCoverage nuget? (and not v8)

[MaceWindu]

Is it possible to disable Microsoft.CodeCoverage? we just need to run tests without any coverage and cannot do it anymore

[nohwnd]

@fhnaseer to help, and review the decisions taken in the Code Coverage package, with shipping the dll.

[MaceWindu]

Workaround (if you don't need CodeCoverage)

<Target Name="Save_PublishDir" BeforeTargets="CopyTraceDataCollectorArtifacts">
	<CreateProperty Value="$(PublishDir)">
		<Output TaskParameter="Value" PropertyName="PublishDir_Backup"/>
	</CreateProperty>
	<CreateProperty Value="$(PublishDir)\_ignore\">
		<Output TaskParameter="Value" PropertyName="PublishDir"/>
	</CreateProperty>
</Target>
<Target Name="Restore_PublishDir" AfterTargets="CopyTraceDataCollectorArtifacts">
	<RemoveDir Directories="$(PublishDir)" />
	<CreateProperty Value="$(PublishDir_Backup)">
		<Output TaskParameter="Value" PropertyName="PublishDir"/>
	</CreateProperty>
</Target>

Save_PublishDir target will override target folder for codecoverage binaries and Restore_PublishDir will restore it after target CopyTraceDataCollectorArtifacts and remove copied files (as we publish it to azure artifacts and don't want extra bytes)

[fhnaseer]

System.Text.Json was added in code coverage librariesto support testconfig.json (MTP). We are using 6 version due to compatibility reasons.

@nohwnd I believe that copying of MS.CC is handled by vstest. Is it possible to pass some arg in the command line to skip copying step?

[Laniusexcubitor]

As @jeroenjanssen-cpp already mentioned, the copying is done by the CopyTraceDataCollectorArtifacts target in Microsoft.CodeCoverage.targets, which will automatically be integrated into the build as transient reference when Microsoft.NET.Test.Sdk is referenced and it will be executed when publishing.