Add support for passing in arguments for methods used to store dynamic data

[cdepano]

Summary

Hello team, hope you're all well. I'd like to request for a feature to be able to pass in arguments to methods used for storing dynamic data.

Background and Motivation

I was working on some complicated test scenarios with many variations and found myself having to write methods that are almost identical to each other for generating a list of list of N booleans:

For example:

private static IEnumerable<object[]> GenerateBoolListWith3Flags()
{
    bool[] boolArray = new bool[] { true, false };

    var testData = new List<object[]>();

    foreach (var flag0 in boolArray)
    {
        foreach (var flag1 in boolArray)
        {
            foreach (var flag2 in boolArray)
            {
                testData.Add(new object[] { flag0, flag1, flag2 });
            }
        }
    }

    return testData;
}

private static IEnumerable<object[]> GenerateBoolListWith6Flags()
{
    bool[] boolArray = new bool[] { true, false };

    var testData = new List<object[]>();

    foreach (var flag0 in boolArray)
    {
        foreach (var flag1 in boolArray)
        {
            foreach (var flag2 in boolArray)
            {
                foreach (var flag3 in boolArray)
                {
                    foreach (var flag4 in boolArray)
                    {
                        foreach (var flag5 in boolArray)
                        {
                            testData.Add(new object[] { flag0, flag1, flag2, flag3, flag4, flag5 });
                        }
                    }
                }
            }
        }
    }

    return testData;
}

Then use it as follows:

[TestMethod]
[DynamicData(nameof(GenerateBoolListWith3Flags), DynamicDataSourceType.Method)]
public void Test1(
    bool flag0,
    bool flag1,
    bool flag2)
{
    ...
}

[TestMethod]
[DynamicData(nameof(GenerateBoolListWith6Flags), DynamicDataSourceType.Method)]
public void Test2(
    bool flag0,
    bool flag1,
    bool flag2,
    bool flag3,
    bool flag4,
    bool flag5)
{
    ...
}

If I happen to have a bunch of tests with many different numbers of bool parameters for different test variations, then I currently am writing a bunch of GenerateBoolListWith*Flags methods.

I was hoping to refactor and combine the dynamic data source methods into one, but I realized I can only achieve this if I am able to pass in an argument

Proposed Feature

A new attribute such as DynamicDataMethodArgumentAttribute:

[TestMethod]
[DynamicData(nameof(GenerateBoolListWithNFlags), DynamicDataSourceType.Method)]
[DynamicDataMethodArgument(3)]
public void Test1(
    bool flag0,
    bool flag1,
    bool flag2)
{
    ...
}

[TestMethod]
[DynamicData(nameof(GenerateBoolListWithNFlags), DynamicDataSourceType.Method)]
[DynamicDataMethodArgument(6)]
public void Test2(
    bool flag0,
    bool flag1,
    bool flag2,
    bool flag3,
    bool flag4,
    bool flag5)
{
    ...
}

private static IEnumerable<object[]> GenerateBoolListWithNFlags(int n)
{
    bool[] boolArray = new bool[] { true, false };

    var testData = new List<object[]>();

    // Generate test data with n elements per row

    return testData;
}

I understand this may be a very specific usecase, but I think it may also be useful if we somehow want to change up the dynamic data returned depending on passed arguments.

Please let me know if there are any workarounds, other suggestions, or if this is feasible. Thank you!

Alternative Designs

n/a

[Evangelink]

Hi @cdepano,

Thanks for the suggestion! It indeed looks like an interesting feature.

We need to investigate how we want to implement it (e.g. how many arguments, what type, does it fit the ITestDataSource mode etc).

[jnyrup]

NUnit has a similar inline approach to this, with [Values] and [Combinatorial].

With those features Test2 can in NUnit be written as:

[Test, Combinatorial]
public void Test2(
    [Values] bool flag0,
    [Values] bool flag1,
    [Values] bool flag2,
    [Values] bool flag3,
    [Values] bool flag4,
    [Values] bool flag5)
{
    ...
}

[Evangelink]

I think @cdepano was referring to something like ParameterizedSourceExampleFixture in NUnit doc https://docs.nunit.org/articles/nunit/writing-tests/attributes/testcasesource.html