testfx icon indicating copy to clipboard operation
testfx copied to clipboard

runsettings are ignored during test discovery (data driven tests)

Open rdstevens-gh opened this issue 1 month ago • 1 comments

Describe the bug

During test discovery of DynamicData tests (using the ITestDataSource), the <EnvironmentVariables> section of the selected .runsettings file is ignored.

Steps To Reproduce

  1. Implement the following:
    internal class ExampleTestDataSource : Attribute, ITestDataSource
    {
        public IEnumerable<object?[]> GetData(MethodInfo methodInfo)
        {
            yield return new object[] { Environment.GetEnvironmentVariable("TEST_ENV") };
        }

        public string? GetDisplayName(MethodInfo methodInfo, object?[]? data)
        {
            if (data != null)
            {
                var value = (string)data[0];
                return $"{methodInfo.Name} - {value}";
            }
            return null;
        }
    }

    [TestClass]
    public class MyTests
    {
        [TestMethod]
        [ExampleTestDataSource]
        public void MyTest(string name)
        {
            Assert.IsTrue(!string.IsNullOrEmpty(name));
        }
    }
  1. Create a .runnsettings file with the following section:
<RunSettings>
	<RunConfiguration>
		<EnvironmentVariables>
			<TEST_ENV>This is a string</TEST_ENV>
		</EnvironmentVariables>
	</RunConfiguration>
</RunSettings>
  1. Configure TestExplorer to use your .runsettings file.

  2. Observe that the test name (in test explorer) does not include the "This is a string" token.

  3. Observe that MyTest fails.

Expected behavior

.Runsettings to be used during test discovery.

Actual behavior

.runsettings is ignored during test discovery.

rdstevens-gh avatar Nov 26 '25 11:11 rdstevens-gh

The issue here is in VSTestBridge. The RunSettingsEnvironmentVariableProvider implementation doesn't consider TESTINGPLATFORM_EXPERIMENTAL_VSTEST_RUNSETTINGS.

Youssef1313 avatar Nov 26 '25 19:11 Youssef1313