Missing tests and pending tests and tests not run when running theory in .NET5.0

[asgerhallas]

Given below test setup, in net5.0,

When running all test case once

• commenting out one of the MyDataAttributes
• rerunning all test cases again

Then the commented out test case will be missing from the list (resharper) or marked as not run (vs runner). When run again, it will re-appear, but often as pending (sometimes a rebuild is necessary).

This does not happen in netcoreapp3.1.

public class UnitTest1
{
    [Theory, MyDataProvider(1, 8, "B")]
    [MyData(1, "A")]
    [MyData(2, "A")]
    [MyData(3, "A")]
    [MyData(4, "A")]
    [MyData(5, "A")]
    [MyData(6, "A")]
    [MyData(7, "A")]
    [MyData(8, "A")]
    public void Test1(int value, string expected) 
    { 
    }

    public class MyDataProviderAttribute : DataAttribute
    {
        public MyDataProviderAttribute(int from, int count, string expected)
        {
            From = @from;
            Count = count;
            Expected = expected;
        }

        public int From { get; }
        public int Count { get; }
        public string Expected { get; }

        public override IEnumerable<object[]> GetData(MethodInfo testMethod)
        {
            var overrides = testMethod.GetCustomAttributes<MyDataAttribute>();

            return Enumerable.Range(From, Count)
                .Select(x => overrides.SingleOrDefault(y => Equals(y.Value, x)) is {} theOverride
                    ? new object[] {theOverride.Value, theOverride.Expected}
                    : new object[] {x, Expected});
        }
    }

    [AttributeUsage(AttributeTargets.Method, AllowMultiple = true)]
    public class MyDataAttribute : Attribute
    {
        public MyDataAttribute(object value, string expected)
        {
            Value = value;
            Expected = expected;
        }

        public object Value { get; }
        public string Expected { get; }
    }
}

[asgerhallas]

[bradwilson]

Using:

• Visual Studio 2022 (17.5.5)
• .NET 6
• xunit 2.5.0-pre.20 (from MyGet)
• xunit.runner.visualstudio 2.5.0-pre.14 (from MyGet)

I am not able to reproduce this. I don't know if this is something was previous broken and now fixed in the intervening time since the issue was opened, but let me know if you are still able to repro this. (A reminder that we do not support Resharper's test runner; that support is provided by JetBrains.)

https://github.com/xunit/visualstudio.xunit/assets/16944/61532ab2-45a1-4463-a1de-efbb4280af04

[bradwilson]

I just realized in your video that your test runner is the Resharper runner, so I'm closing this as External and suggest if you can still repro it, you open an issue against Resharper.