Test execution failure on Linux

[chris1248]

Description

I have a .net core project with some unit tests. The unit test framework fails on Linux.

Steps to reproduce

Given this project file:

<Project Sdk="Microsoft.NET.Sdk">
	<Import Project="..\company.globals.props" />
	<PropertyGroup>
		<TargetFramework>netcoreapp3.1</TargetFramework>
		<OutputPath>$(BuildDir)</OutputPath>
		<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
	</PropertyGroup>

	<ItemGroup>
		<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.5.0" />
		<PackageReference Include="MSTest.TestAdapter" Version="2.1.0" />
		<PackageReference Include="MSTest.TestFramework" Version="2.1.0" />
		<PackageReference Include="coverlet.collector" Version="1.2.0" />
	</ItemGroup>
</Project>

When run this on Red Hat Enterprise Linux 8.2, from the command line using .net core. I get a runtime exception and the test execution fails. Execute the following command from the command line: dotnet test ./foo.csproj -p:Configuration=Debug

Expected behavior

This runs absolutely fine on windows:

Test run for C:\Users\me\source\repos\foo\builds\bin\x64\Debug\netcoreapp3.1\foo.dll(.NETCoreApp,Version=v3.1)
Microsoft (R) Test Execution Command Line Tool Version 16.6.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

A total of 1 test files matched the specified pattern.

Test Run Successful.
Total tests: 28
     Passed: 28
 Total time: 4.6982 Seconds

Actual behavior

Test run for /home/admin/code/foo/builds/bin/x64/Debug/netcoreapp3.1/foo.dll(.NETCoreApp,Version=v3.1)
Microsoft (R) Test Execution Command Line Tool Version 16.3.0-dev+55e7e45431c9c05656c999b902686e7402664573
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

A total of 1 test files matched the specified pattern.  

An exception occurred while invoking executor 'executor://mstestadapter/v2': Protocol error

Test Run Failed.

Environment

On Linux I have:

root (/home/admin/code/foo) $ dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   3.1.105
 Commit:    99c9707ea8

Runtime Environment:
 OS Name:     rhel
 OS Version:  8
 OS Platform: Linux
 RID:         rhel.8-x64
 Base Path:   /usr/lib64/dotnet/sdk/3.1.105/

Host (useful for support):
  Version: 3.1.5
  Commit:  65cd789777

.NET Core SDKs installed:
  3.1.105 [/usr/lib64/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.App 3.1.5 [/usr/lib64/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.1.5 [/usr/lib64/dotnet/shared/Microsoft.NETCore.App]
...

On Windows (v10) I have:

.NET Core SDK (reflecting any global.json):
 Version:   3.1.301
 Commit:    7feb845744

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.18362
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.1.301\

Host (useful for support):
  Version: 3.1.5
  Commit:  65cd789777

.NET Core SDKs installed:
  2.1.4 [C:\Program Files\dotnet\sdk]
  2.1.202 [C:\Program Files\dotnet\sdk]
  2.1.400 [C:\Program Files\dotnet\sdk]
  2.1.602 [C:\Program Files\dotnet\sdk]
  2.1.700 [C:\Program Files\dotnet\sdk]
  3.1.201 [C:\Program Files\dotnet\sdk]
  3.1.301 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.19 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.19 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.19 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Operating system, Build version of vstest.console, Package version of MSTest framework and adapter

[chris1248]

The thing is, I have a few other projects in my particular project, and all of them fail in Linux with the same error:

An exception occurred while invoking executor 'executor://mstestadapter/v2': Protocol error

[nohwnd]

Install Red Hat 8, and the same dotnet SDK is you do, and it works just fine. Are you able to repro this on multiple machines?

[btull89]

I'm seeing this same issue with Vagrant CentOS 7. dotnet version is 3.1.401.

I run dotnet test in windows for the same project and it works fine. It fails in Vagrant CentOS 7.

[btull89]

This appears to be some kind of directory issue. By using the -r command if I write my tests results to /home/vagrant/repos/my-micro/My.Micro.Test/test-results it fails. Writing to /home/vagrant/test/test-results succeeds.

[nohwnd]

This should be a TestHost related error, which version of Microsoft.NET.Test.SDK is your project using? Currently there is 16.7.1 and 16.8.0-preview.

[btull89]

16.7.1 and 16.8.0-preview both have the same error outcome. I am also using Microsoft (R) Test Execution Command Line Tool Version 16.7.0.

In addition to Protocol Error I'm also getting another error message:

An exception occurred while invoking executor 'executor://mstestadapter/v2': Protocol error
Results File: /home/vagrant/repos/my-micro/My.Micro.Test/test-results/_localhost_2020-08-27_13_16_38.trx

Test Run Failed.
Additionally, path to test adapters can be specified using /TestAdapterPath command. Example  /TestAdapterPath:<pathToCustomAdapters>.
/usr/share/dotnet/sdk/3.1.401/Microsoft.TestPlatform.targets(32,5): error MSB4181: The "Microsoft.TestPlatform.Build.Tasks.VSTestTask" task returned false but did not log an error. [csproj path redacted]

[nohwnd]

could you run it with --diag:logs/log.txt and share the log files that are written into logs directory please?

[btull89]

Attached are the requested logs files.

logs.zip

[nohwnd]

@btull89 Thanks, the log says it is caused by Protocol error, probably nothing we can do about that:

TpTrace Error: 0 : 4460, 4, 2020/09/29, 14:59:41.481, 784228795599, testhost.dll, BaseRunTests.RunTestInternalWithExecutors: 
  An exception occurred while invoking executor executor://mstestadapter/v2. 
  System.IO.IOException: Protocol error
   at System.IO.FileSystem.CreateDirectory(String fullPath)
   at System.IO.Directory.CreateDirectory(String path)
   at Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Utilities.FileUtility.CreateDirectoryIfNotExists(String directory)
   at Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Utilities.DeploymentUtilityBase.CreateDeploymentDirectories(IRunContext runContext)
   at Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.TestDeployment.Deploy(IEnumerable`1 tests, IRunContext runContext, IFrameworkHandle frameworkHandle)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestExecutionManager.RunTests(IEnumerable`1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle, TestRunCancellationToken cancellationToken)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.MSTestExecutor.RunTests(IEnumerable`1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.RunTestsWithSources.InvokeExecutor(LazyExtension`2 executor, Tuple`2 executorUriExtensionTuple, RunContext runContext, IFrameworkHandle frameworkHandle)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RunTestInternalWithExecutors(IEnumerable`1 executorUriExtensionMap, Int64 totalTests).
TpTrace Error: 0 : 4460, 4, 2020/09/29, 14:59:41.482, 784229718049, testhost.dll, An exception occurred while invoking executor 'executor://mstestadapter/v2': Protocol error

Sorry to not have any specific solution, but searching for this turns up with a whole range of possible causes, such as no space on the disk, or missing permissions when mounting a volume in VirtualBox https://www.google.com/search?q=cannot+create+directory+protocol+error+dotnet

[danmincu]

We had a similar problem where the issue end up being that vstest was creating a temporary folder containing ":" character because the code is trying to use the current date time to get the temporary name. The folder was shared from Virtual Box into Windows so basically Windows was denying the creation of the respective folder. Protocol error : '/test-results/Deploy_ 2022-06-01 19:16:31' Possible related to https://github.com/microsoft/testfx/issues/678

[Evangelink]

@danmincu If you are able to reproduce the issue, could you please let us know if the fix in the related PR fixed this issue for you? Thanks

[danmincu]

In my case the fix was to run the tests into a Linux folder and to copy the *.trx file in the folder that was mounted on Windows avoiding the cross OS folder name creation problem. However, this commit should fix it https://github.com/microsoft/testfx/pull/1119/files I'll be able to test it soon after I upgrade the testfx framework.

[Evangelink]

Thank you! I will keep this one open until you confirm it's good.

[Evangelink]

Hey @danmincu. I don't know if you had time to confirm things are good now? I'll move forward and close this ticket. If you have a repro of something that still fails, please either ping here or create a new issue.