arcade
arcade copied to clipboard
dotnet
Should ConditionalClass and similar attributes in XUnitExtensions filter tests out automatically?
ConditionalClass and similar attributes add a failing trait to the tests that should conditionally not run: https://github.com/dotnet/arcade/blob/1ccd352f12e845d89e5b5a9b800e2be88f8a6d98/src/Microsoft.DotNet.XUnitExtensions/src/Discoverers/ConditionalClassDiscoverer.cs#L30
It might make sense to automatically filter those out without relying on the test settings to exclude the failing trait (i.e. in a .runsettings file).
This is a problem which affects running tests in SqlClient and has done for many years. If there is a chance it can be fixed that would be very useful and allow tests to be run from inside visual studio which has never been useful before.
@dotnet/sqlclientdevteam may want to weigh in and identify if they consider this something important to them.
I don't know about the "failing" terminology on this issue. But we (SqlClient) do see tests run in the VS GUI that are filtered out (Skipped) in command-line tests runs by ConditionalFacts that exclude them based on environment or other logic. It would be really nice if the VS GUI behaved the same.
@David-Engel can you please try again once you get an arcade update with Microsoft.DotNet.XUnitExtensions version 10.0.0-beta.25175.1 or later? I think I fixed the issue with ConditionalFacts not working in VS with https://github.com/dotnet/arcade/pull/15667
And just to make this issue more accurate: ConditionalFact/ConditionalTheory do not use the "failing" category, only ConditionalClass does.