Added ADO.NET importing/exporting functionality to DataFrame

[andrei-faber]

See https://github.com/dotnet/machinelearning/issues/5972 This universal integration with ADO.NET allows importing and exporting data to a large number of SQL-compatible databases. In addition to this, this PR adds methods to load data from any IEnumerable<T> collections and to export data to System.Data.DataTable

[dnfadmin]

All CLA requirements met.

[andrei-faber]

It seems that build fails because I added a reference to the System.Data.SQLite NuGet package (it's only used for integration tests), and the build doesn't use public NuGet repos; I wonder if it's possible to add System.Data.SQLite to the private repo?

[michaelgsharp]

@andrei-faber I believe that should be just fine. Let me talk to some people on my end and I'll get back to you.

[michaelgsharp]

/azp run

[azure-pipelines[bot]]

Azure Pipelines successfully started running 2 pipeline(s).

[michaelgsharp]

@andrei-faber The NuGet package has been added and I have requeued the pipelines.

[andrei-faber]

@michaelgsharp some strange version conflict in there. Maybe it's better to use libraries with the same version number for System.Data.SQLite.Core in Microsoft.ML.Tests.csproj and System.Data.SQLite in Microsoft.Data.Analysis.Tests.csproj For instance use System.Data.SQLite 1.0.112.2 (if it's possible to get it)

[michaelgsharp]

@andrei-faber So part of the issue is that we are still building for .net core 2, which the sql package doesn't support. I am in the process of removing that, should be done by next week (have a couple of things do to first), that should resolve this issue. So we should be able to get this in next week.

I did get the version stuff figured out, I didn't get the package moved correctly.

[andrei-faber]

@michaelgsharp great, thanks. Is there anything I can do to help?

[codecov[bot]]

Codecov Report

Merging #5975 (cf05e79) into main (ff3b1b9) will increase coverage by 0.01%. The diff coverage is 86.36%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5975      +/-   ##
==========================================
+ Coverage   68.65%   68.67%   +0.01%     
==========================================
  Files        1202     1203       +1     
  Lines      250769   250974     +205     
  Branches    26190    26209      +19     
==========================================
+ Hits       172166   172344     +178     
- Misses      71785    71805      +20     
- Partials     6818     6825       +7     

Flag	Coverage Δ
Debug	68.67% <86.36%> (+0.01%)	:arrow_up:
production	63.16% <79.31%> (+0.01%)	:arrow_up:
test	88.87% <100.00%> (+0.01%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/Microsoft.Data.Analysis/Extensions.cs	73.33% <73.33%> (ø)
src/Microsoft.Data.Analysis/DataFrame.IO.cs	80.76% <80.00%> (+0.88%)	:arrow_up:
...Microsoft.Data.Analysis.Tests/DataFrame.IOTests.cs	98.97% <100.00%> (+0.07%)	:arrow_up:

... and 5 files with indirect coverage changes