sdk icon indicating copy to clipboard operation
sdk copied to clipboard

`dotnet test --filter` appears to have a hidden character count limitation for tests with long names

Open DrEsteban opened this issue 1 year ago • 0 comments

Describe the bug

It's a bit hard to describe, but dotnet test --filter seems to have a hidden max character length at which it will no longer match characters that make up the test name. Hopefully the repro steps below help clear it up.

NOTE: The filter string examples below might look a little weird, but according to the docs you need to essentially URL-escape special characters in order for them to be interpreted.

To Reproduce

  1. Check-out my example repo: https://github.com/DrEsteban/dotnet-test-filter-bug-example

  2. Open a command prompt to the repo location

  3. Execute the following command, which works: dotnet test --filter "(DisplayName~TestProject1.UnitTest1.ThisIsAnExtremelyLongTestName_SoICanDemonstrateAStrangeBehaviorInDotnetTest_Filter&DisplayName~string1%3a %221234567890%22%2c string2%3a %221234567890%22%2c string3%3a %221234567890%22%2c string4%3a %221234567890%22%2c string5%3a %221234567890%22%2c string6%3a %221234567890%22%2c string7%3a %221234567890%22%2c string8%3a %221234567890%22%2c string9%3a %221234567890%22%2c string10%3a %221234567890%22%2c string11%3a %221234567890%22%2c string12%3a %221234567890%22%2c string13%3a %221234567890%22%2c string14%3a %22123456%22%2c string15%3a %221234)" image

  4. Execute the following command, which does not work: dotnet test --filter "(DisplayName~TestProject1.UnitTest1.ThisIsAnExtremelyLongTestName_SoICanDemonstrateAStrangeBehaviorInDotnetTest_Filter&DisplayName~string1%3a %221234567890%22%2c string2%3a %221234567890%22%2c string3%3a %221234567890%22%2c string4%3a %221234567890%22%2c string5%3a %221234567890%22%2c string6%3a %221234567890%22%2c string7%3a %221234567890%22%2c string8%3a %221234567890%22%2c string9%3a %221234567890%22%2c string10%3a %221234567890%22%2c string11%3a %221234567890%22%2c string12%3a %221234567890%22%2c string13%3a %221234567890%22%2c string14%3a %22123456%22%2c string15%3a %2212345)" image

Note the only difference between those 2 filter strings is the single character 5 at the end. So, there seems to be a test name length at which the --filter parameter is no longer able to search against. This is further evidenced by:

  1. Execute the following command, which does not work: dotnet test --filter "(DisplayName~TestProject1.UnitTest1.ThisIsAnExtremelyLongTestName_SoICanDemonstrateAStrangeBehaviorInDotnetTest_Filter&DisplayName~string16%3a %221234567890%22)" image

This appears to ultimately indicate that the character limit doesn't exist on the filter string, but rather on the test name itself.

E.g. TestProject1.UnitTest1.ThisIsAnExtremelyLongTestName_SoICanDemonstrateAStrangeBehaviorInDotnetTest_Filter(string1: "1234567890", string2: "1234567890", string3: "1234567890", string4: "1234567890", string5: "1234567890", string6: "1234567890", string7: "1234567890", string8: "1234567890", string9: "1234567890", string10: "1234567890", string11: "1234567890", string12: "1234567890", string13: "1234567890", string14: "123456", string15: "1234~567890", string16: "1234567890")~

^ None of the strikethrough portion above will match against the test name if used in the filter string.

Further technical details

  • Include the output of dotnet --info
.NET SDK (reflecting any global.json):
 Version:   6.0.203
 Commit:    a20feadf6d

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.22000
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\6.0.203\

global.json file:
  Not found

Host:
  Version:      6.0.9
  Architecture: x64
  Commit:       163a63591c

.NET SDKs installed:
  1.1.14 [C:\Program Files\dotnet\sdk]
  6.1.423 [C:\Program Files\dotnet\sdk]
  7.0.202 [C:\Program Files\dotnet\sdk]
  6.0.203 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 3.1.24 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.29 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.16 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 1.0.16 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 1.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.24 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.29 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.16 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.24 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.26 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.29 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.16 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Download .NET:
  https://aka.ms/dotnet-download

Learn about .NET Runtimes and SDKs:
  https://aka.ms/dotnet/runtimes-sdk-info

DrEsteban avatar Sep 23 '22 00:09 DrEsteban