testfx
testfx copied to clipboard
runsettings are ignored during test discovery (data driven tests)
Describe the bug
During test discovery of DynamicData tests (using the ITestDataSource), the <EnvironmentVariables> section of the selected .runsettings file is ignored.
Steps To Reproduce
- Implement the following:
internal class ExampleTestDataSource : Attribute, ITestDataSource
{
public IEnumerable<object?[]> GetData(MethodInfo methodInfo)
{
yield return new object[] { Environment.GetEnvironmentVariable("TEST_ENV") };
}
public string? GetDisplayName(MethodInfo methodInfo, object?[]? data)
{
if (data != null)
{
var value = (string)data[0];
return $"{methodInfo.Name} - {value}";
}
return null;
}
}
[TestClass]
public class MyTests
{
[TestMethod]
[ExampleTestDataSource]
public void MyTest(string name)
{
Assert.IsTrue(!string.IsNullOrEmpty(name));
}
}
- Create a .runnsettings file with the following section:
<RunSettings>
<RunConfiguration>
<EnvironmentVariables>
<TEST_ENV>This is a string</TEST_ENV>
</EnvironmentVariables>
</RunConfiguration>
</RunSettings>
-
Configure TestExplorer to use your .runsettings file.
-
Observe that the test name (in test explorer) does not include the "This is a string" token.
-
Observe that MyTest fails.
Expected behavior
.Runsettings to be used during test discovery.
Actual behavior
.runsettings is ignored during test discovery.
The issue here is in VSTestBridge. The RunSettingsEnvironmentVariableProvider implementation doesn't consider TESTINGPLATFORM_EXPERIMENTAL_VSTEST_RUNSETTINGS.