docs icon indicating copy to clipboard operation
docs copied to clipboard

--parallel flag, migrated from dotnet vstest, is missing in dotnet test documentation

Open uladz-zubrycki opened this issue 2 years ago • 2 comments

Issue description

According to the dotnet vstest documentation page, the command is superseded by the dotnet test command, which can now be used to run assemblies.

Which, I believe, was done as a result of https://github.com/microsoft/vstest/issues/1453 request in a scope of https://github.com/microsoft/vstest-docs/pull/196.

dotnet vstest is able to run tests from the assemblies in parallel if --parallel flag is provided. Per the referenced RFC, there was an intention to support the same behavior for the dotnet test command as well.

Success criteria (as per user ask)

  1. The "dotnet test" command can be used in place of the "dotnet vstest" command.
  2. The "dotnet test" command can accept a list of dlls
  3. Using "dotnet test" and targeting a list of dlls, there is an argument for "--parallel" which will run the tests from the dlls in parallel.
  4. Using "dotnet test" and targeting a dll, the inclusion or exclusion of the "--no-build" argument will have no impact on the behavior of execution.

My local testing confirms that --parallel flag is respected and tests are run in parallel; test from the assemblies are run sequentially otherwise. Note, that I'm talking parallel execution of the assemblies themselves and not the internal assembly-level of tests parallelization, which is handled by NUnit and its adapter for me.

However, there is no documentation for that feature at the dotnet test page and in the output of the dotnet test --help command, which is confusing.

  1. Is parallel execution of tests from assemblies supported by the dotnet test command officially, and it's safe to rely on that, or it's an implementation detail exposed accidentally?
  2. Should the documentation for dotnet test be updated to reflect that?

Target framework

  • [x] .NET Core
  • [ ] .NET Framework
  • [ ] .NET Standard
dotnet --info output
.NET SDK:
 Version:   7.0.304
 Commit:    7e794e2806

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.22621
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\7.0.304\

Host:
  Version:      7.0.7
  Architecture: x64
  Commit:       5b20af47d9

.NET SDKs installed:
  3.1.100 [C:\Program Files\dotnet\sdk]
  3.1.426 [C:\Program Files\dotnet\sdk]
  6.0.100 [C:\Program Files\dotnet\sdk]
  6.0.118 [C:\Program Files\dotnet\sdk]
  7.0.302 [C:\Program Files\dotnet\sdk]
  7.0.304 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.18 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.18 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.32 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.18 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.7 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found
dotnet test --version output 17.6.3.22601

uladz-zubrycki avatar Jun 23 '23 11:06 uladz-zubrycki