sdk
sdk copied to clipboard
`dotnet test --filter` appears to have a hidden character count limitation for tests with long names
Describe the bug
It's a bit hard to describe, but dotnet test --filter seems to have a hidden max character length at which it will no longer match characters that make up the test name. Hopefully the repro steps below help clear it up.
NOTE: The filter string examples below might look a little weird, but according to the docs you need to essentially URL-escape special characters in order for them to be interpreted.
To Reproduce
-
Check-out my example repo: https://github.com/DrEsteban/dotnet-test-filter-bug-example
-
Open a command prompt to the repo location
-
Execute the following command, which works:
dotnet test --filter "(DisplayName~TestProject1.UnitTest1.ThisIsAnExtremelyLongTestName_SoICanDemonstrateAStrangeBehaviorInDotnetTest_Filter&DisplayName~string1%3a %221234567890%22%2c string2%3a %221234567890%22%2c string3%3a %221234567890%22%2c string4%3a %221234567890%22%2c string5%3a %221234567890%22%2c string6%3a %221234567890%22%2c string7%3a %221234567890%22%2c string8%3a %221234567890%22%2c string9%3a %221234567890%22%2c string10%3a %221234567890%22%2c string11%3a %221234567890%22%2c string12%3a %221234567890%22%2c string13%3a %221234567890%22%2c string14%3a %22123456%22%2c string15%3a %221234)"
-
Execute the following command, which does not work:
dotnet test --filter "(DisplayName~TestProject1.UnitTest1.ThisIsAnExtremelyLongTestName_SoICanDemonstrateAStrangeBehaviorInDotnetTest_Filter&DisplayName~string1%3a %221234567890%22%2c string2%3a %221234567890%22%2c string3%3a %221234567890%22%2c string4%3a %221234567890%22%2c string5%3a %221234567890%22%2c string6%3a %221234567890%22%2c string7%3a %221234567890%22%2c string8%3a %221234567890%22%2c string9%3a %221234567890%22%2c string10%3a %221234567890%22%2c string11%3a %221234567890%22%2c string12%3a %221234567890%22%2c string13%3a %221234567890%22%2c string14%3a %22123456%22%2c string15%3a %2212345)"
Note the only difference between those 2 filter strings is the single character 5 at the end. So, there seems to be a test name length at which the --filter parameter is no longer able to search against. This is further evidenced by:
- Execute the following command, which does not work:
dotnet test --filter "(DisplayName~TestProject1.UnitTest1.ThisIsAnExtremelyLongTestName_SoICanDemonstrateAStrangeBehaviorInDotnetTest_Filter&DisplayName~string16%3a %221234567890%22)"
This appears to ultimately indicate that the character limit doesn't exist on the filter string, but rather on the test name itself.
E.g. TestProject1.UnitTest1.ThisIsAnExtremelyLongTestName_SoICanDemonstrateAStrangeBehaviorInDotnetTest_Filter(string1: "1234567890", string2: "1234567890", string3: "1234567890", string4: "1234567890", string5: "1234567890", string6: "1234567890", string7: "1234567890", string8: "1234567890", string9: "1234567890", string10: "1234567890", string11: "1234567890", string12: "1234567890", string13: "1234567890", string14: "123456", string15: "1234~567890", string16: "1234567890")~
^ None of the strikethrough portion above will match against the test name if used in the filter string.
Further technical details
- Include the output of
dotnet --info
.NET SDK (reflecting any global.json):
Version: 6.0.203
Commit: a20feadf6d
Runtime Environment:
OS Name: Windows
OS Version: 10.0.22000
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\6.0.203\
global.json file:
Not found
Host:
Version: 6.0.9
Architecture: x64
Commit: 163a63591c
.NET SDKs installed:
1.1.14 [C:\Program Files\dotnet\sdk]
6.1.423 [C:\Program Files\dotnet\sdk]
7.0.202 [C:\Program Files\dotnet\sdk]
6.0.203 [C:\Program Files\dotnet\sdk]
.NET runtimes installed:
Microsoft.AspNetCore.App 3.1.24 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.29 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.16 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 1.0.16 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.24 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.29 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.16 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.24 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.26 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.29 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.16 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.2 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Download .NET:
https://aka.ms/dotnet-download
Learn about .NET Runtimes and SDKs:
https://aka.ms/dotnet/runtimes-sdk-info