Support DataSourceAttribute in .Net Core.

[magol]

Description

We have following code

[TestMethod]
[DeploymentItem("MyTests.Configuration.xml")]
[DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML", "|DataDirectory|\\MyTests.Configuration.xml", "MyTestCase", DataAccessMethod.Sequential)]
public void GetFiles_OnRootDirectory_CallsApiCorrectly()
{
    var foo = TestContext.DataRow["MyFooData"]);
}

with following xml file

<?xml version="1.0" encoding="utf-8" ?>
<Rows>
  <MyTestCase>
    <MyFooData>true</MyFooData>
  </MyTestCase>
  <MyTestCase>
    <MyFooData>false</MyFooData>
  </MyTestCase>

  <OtherTestCase>
    <MyFooData>false</MyFooData>
  </OtherTestCase>
</Rows>

Is TestContext.DataRow expected to come to MsTest v2, or is it any other way to do the same thing?

Steps to reproduce

1. Install .NET Core 2.0.0-preview2-006497
2. Target the test project to ,NET Core 2.0
3. Use MSTest.TestAdapter version 1.2.0-beta
4. Try to build

Expected behavior

The compile past

Actual behavior

I get following build error

error CS1061: 'TestContext' does not contain a definition for 'DataRow' and no extension method 'DataRow' accepting a first argument of type 'TestContext' could be found (are you missing a using directive or an assembly reference?)

Environment

.NET Core 2.0.0-preview2-006497 Visual Studio 15.3.0 preview 7.0 Windows 10

[AbhitejJohn]

@magol: Apologies for the delay. The .Net Core version of the adapter does not support DataSource yet. I'll update the title and mark this as a request for adding in that support. As a workaround, you could update to the latest beta and use DynamicData instead.

[magol]

@AbhitejJohn Thanks for the answer. What is you time framme to add this feature? When can I expert to be able to use this?

[AbhitejJohn]

@magol: Don't have one yet. Put this temporarily in S123 where it would most likely be addressed.

[nvianhd]

I hope MS still support DataRow on Dot Net Core

[AbhitejJohn]

@nvianhd: It still does and much more!

[magol]

@AbhitejJohn What is the status on this? The S123 milestone have past due by about 1 month.

[AbhitejJohn]

@magol : Sorry, we had a few other things that came up and this took the back seat. Tagging @sudiptadmsft and @pvlakshm to see when we can bring this in.

[omkarmore83]

I am waiting on this too.. @AbhitejJohn any ETA?

[rhyous]

Trying to move my Unit Tests to .NET Core 2.0. Of course, I have many row tests. I'll have a look at Dynamic data. But the problem is all the existing row tests use DataSourceAttribute.

[alex182]

this would be great to have working

[magol]

Any progress on this?

[AbhitejJohn]

Sorry, I haven't been working on the framework for a while now. Tagging @cltshivash @pvlakshm .

[alenjalex-zz]

I am also waiting on this.

[sirkris]

More than a year since the initial post and still waiting. This is unacceptable, as it makes it impossible to make use of data-driven unit tests in .NET Core. Please fix this! I'm gonna have to hack in a really sloppy workaround for my test project to work in the meantime....

[cilerler]

As @AbhitejJohn already mentioned above DynamicData is way to go at this point. Here is an example for CSV reading

private static string[] SplitCsv(string input)
{
    var csvSplit = new Regex("(?:^|,)(\"(?:[^\"]+|\"\")*\"|[^,]*)", RegexOptions.Compiled);
    var list = new List<string>();
    foreach (Match match in csvSplit.Matches(input))
    {
        string value = match.Value;
        if (value.Length == 0)
        {
            list.Add(string.Empty);
        }

        list.Add(value.TrimStart(','));
    }
    return list.ToArray();
}

private static IEnumerable<string[]> GetData()
{
    IEnumerable<string> rows = System.IO.File.ReadAllLines(@"Resources\NameAddressCityStateZip.csv").Skip(1);
    foreach (string row in rows)
    {
        yield return SplitCsv(row);
    }
}

[TestMethod]
[DynamicData(nameof(GetData), DynamicDataSourceType.Method)]
//x [DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", @"Resources\NameAddressCityStateZip.csv", "NameAddressCityStateZip#csv", DataAccessMethod.Sequential)]
public void TestMethod1(string input, string expected)
{
    // Arrange
    //x string input = _testContext.Properties["Data"].ToString(); //x _testContext.DataRow["Data"].ToString();
    //x string expected = _testContext.Properties["Expected"].ToString(); //x _testContext.DataRow["Expected"].ToString();
    var parser = _serviceProvider.GetService<Parser>();

    // Act
    string actual = parser.MultiParser(input, ModeType.NameAddressCityStateZipCountry).ToString();

    // Assert
    Assert.AreEqual(expected, actual);
}

[rhyous]

@cilerler It is NOT that easy.

Is the DynamicDataAttribute available in other .NET platforms yet?

We have a bunch of code that compiles to multiple .Net versions. The code and unit tests are shared between platforms. So if I change the unit tests to use DynamicData, I break them for .Net Framework. So basically, I cannot share unit test code.

At best I could add a compiler directive to all my unit tests so in .NET Framework it uses the old code and in .NetCore it uses the new code. Actually, I already did this in my open source projects, only the unit tests are simply not run in .NetCore as it doesn't support the DataSourceAttribute.

Not to mention that the GetData() method you have above 1) doesn't take into account Xml row tests or Excel row tests, and 2) is far too naive to be used by product code. We need to get this working for Csv, Xml, and Excel. I have a CsvParser, but I have to figure out what Xml parser to use or write my own, figure out what Excel parser to use or write my own, etc. Now I have to bring in either new code, which requires approval and sprint time, etc, or new open source projects, which means a security team involved to vet the open source project and paperwork and process.

So again, if it were that easy, we would have done it already.

[sirkris]

@rhyous is spot-on. Relying on DynamicData instead of just implementing this (is there some pressing reason not to?) is not a viable long-term solution to this problem, in my opinion.

[cilerler]

@rhyous

1. The very basic example I provided above is for demonstration purpose only to let others see how easy it is to implement it. I literally didn't expect no one to use it for production.
2. As you already mentioned above you have compiler directives which means it should not break your code base. (you can't expect everyone having a need to support multiple frameworks, many developers are developing fresh new projects and the method I presented may help them to continue with the latest stuff)
3. The situation is pretty simple writing your own parser VS waiting for years to get it from Microsoft. (Well, I usually prefer to write my own at this point in order to be able to use the latest stuff)

Therefore please take it into the consideration that it could be very easy workaround where things are not complicated as it in your projects.

[rhyous]

Wow. Let's ignore that unhelpful response and talk about implementation of a single solution in the right place, which is in this project, instead having customers implement many one-off solutions.

It would be better to implement this once than to have many users implement their own hacks. It looks like this feature is tagged with up-for-grabs so anyone can implement it. That is greateness of open source.

So if DynamicDataAttribute is the future, why not implement DataSourceAttribute in this project as a child of DynamicDataAttribute.

We could probably write an attribute that inherits from DynamicDataAttribute (except the class is sealed, but that is easy to change, it is an open source project after all) and simply have child class that acts exactly like the DynamicDataAttribute but has the signature of the DataSourceAttribute and we enhance the DataSourceAttribute to populate the GetData needs.

Assuming the posters here decide to fork testfx, implement this ourselves, and do a pull request. We could use the source for DataSourceAttribute and associated classes. Can someone at Microsoft comment on if that code can be made available?

[sebainones]

I'm facing the same situation. I was trying to move my Tests to .Net Core but for me is strictly necessary to use DataRow as my information comes only from the DB. So, without this, I wouldn't migrate to .Net Core. @AbhitejJohn It's been more than a year since the post. Is there any ETA or at least do you have on your product backlog. I want to know if it makes sense to wait or if I should be considering other options. Thanks

[Justin-Lloyd]

+1 on the above. Unit testing does not look particularly well supported by Microsoft.

[hansmbakker]

DataSourceAttribute / DataRow would be required for https://github.com/Microsoft/Recognizers-Text to move to .Net Core for tests.

[AbhitejJohn]

@AbhitejJohn It's been more than a year since the post. Is there any ETA or at least do you have on your product backlog.

@sebainones : I haven't been getting much time since I've moved off of working on this area. @cltshivash should be able to help answer if this is on the immediate backlog. What @cilerler has added seems to be good start to unblock folks. It does create an API diff between .NetCore and .NetFramework but the DynamicData schematics was what we were hoping we could converge on.

[mpettus]

Any update on this? I could really use DataRow in my project. Thanks.

[Sl1ckR1ck]

Waiting for this too before continuing porting to .NET Core. No point to start writing reading CSV, DB, etc code if this is coming soon?

I see that @nohwnd assigned this work, so i guess its being implemented?

[nohwnd]

@Sl1ckR1ck it is on my work list, but I did not even get a chance to read through this thread properly. Swamped with other issues. Sorry! 😔

[sebainones]

@nohwnd and @AbhitejJohn I would like to help (contribute) with this. Could you please tell me how could I help to move forward with this? Let me know if I can review some code or even do some coding. (I'm pretty new about contributing but I would love to)

[nohwnd]

@sebainones Thank you for showing interest in this, I think it would be nice to sum up few examples of what is the desired functionality is, and what is missing from core so we have a clear way of going forward. I will in the meantime try to figure out if we can take the existing Framework code and port it to core. I don't know what the procedure here is, and I did not see the code to be already publicly available in this repo.

[sebainones]

Desired Functionality

1. The example provided by @magol when he initially opened the issue. That's an XML as DataSource

2 This is my scenario using a Table (or View) as DataSource

    [TestClass]
    public class MyTestClass
    {
        static TestContext testContext;

        [ClassInitialize]
        public static void ClassInitialize(TestContext context)
        {        
            testContext = context;
        }

        [TestMethod]
        [DeploymentItem(@".\bin\EntityFramework.SqlServer.dll")]
        [DataTestMethod]
        [DataSource("System.Data.SqlClient", "ConnectionString", "tableName", DataAccessMethod.Sequential)]
        public void RunMyDataDrivenTests()
        {          
             var someFiled= testContext.DataRow["SomeColumnFromMyTable"];
        }

What is missing from .Net Core?

As the title originally said : "TestContext' does not contain a definition for 'DataRow' " So, if your create a MSTest Test Project with .NET Core when you try to access the DataRow property you will get an error because TestContext' does not contain a definition for DataRow

If you go to the .Net Framework TestContext Class you will get this:

namespace Microsoft.VisualStudio.TestTools.UnitTesting
{
    //
    // Summary:
    //     Used to store information that is provided to unit tests.
    public abstract class TestContext
    {
        protected TestContext();

         //
        // Summary:
        //     Gets the current data row when test is used for data driven testing.
        public abstract DataRow DataRow { get; }
    }

By contrast, if you go to the .NET Core TestContext Class you will get this:

namespace Microsoft.VisualStudio.TestTools.UnitTesting
{
    //
    // Summary:
    //     TestContext class. This class should be fully abstract and not contain any members.
    //     The adapter will implement the members. Users in the framework should only access
    //     this via a well-defined interface.
    public abstract class TestContext
    {
        protected TestContext();
 
     //There is NO DataRow property!
    }

Note: I have just copied the relevant parts of the code.

@nohwnd: Please let me know if you need that I put those examples in a github repo or something. In addition, I would like to help in taking the existing Framework code and port it to core (If that's possible).

[nohwnd]

@sebainones great. 👍

In addition, I would like to help in taking the existing Framework code and port it to core (If that's possible).

I am currently investigating where to get that code and how to make it open-source if it was not before.