Can't find dependency of Mysql.Data 8

[johannesegger]

Description

Mysql.Data 8.x.x has a dependency on Google.Protobuf. When settings the resolution path of SqlDataProvider to the directory where Mysql.Data is stored the type provider is unable to find Google.Protobuf.

Repro steps

1. Create a new project with the following files:

type SisSqlDataProvider =
    SqlDataProvider< 
        ConnectionString = ...,
        DatabaseVendor = Common.DatabaseProviderTypes.MYSQL,
        ResolutionPath = "path\to\MySql.Data\lib\net452">

Expected behavior

No type provider error.

Actual behavior

Error from type provider:

error FS3033: The type provider 'FSharp.Data.Sql.SqlTypeProvider' reported an error: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.Details: Could not load file or assembly 'Google.Protobuf, Version=3.5.1.0, Culture=neutral, PublicKeyToken=a7d26565bac4d604' or one of its dependencies. The system cannot find the file specified.

Known workarounds

Use Mysql.Data 6 because it doesn't have a dependency on a 3rd party library.

Related information

• Used database: Mysql
• Operating system: Windows 10
• Branch: stable, v1.1.41
• .NET Runtime, CoreCLR or Mono Version: dotnet --version is 2.1.104
• Performance information, links to performance testing scripts

[piaste]

Suggested workaround: I find it easier to manage compile-time dependencies manually, i.e. point the ResolutionPath to a folder in the project and janitor the DLLs within it on my own (and git add --force to check them into source control).

So my src/{project}/DbLayer/DataSources/CompileTimeLibs/ folder contains SqlProvider.dll, Npgsql.dll, as well as every DLL that Npgsql references (currently System.Data.Common.dll and System.Threading.Tasks.Extensions.dll). Regular package management (nuget / paket) takes care of those dependencies at runtime.

[johannesegger]

I like your work-around, thanks. However regarding usability I would love if it just worked. I'm glad that we now have two possible work-arounds, so feel free to close this at any time. Thanks again.

[Thorium]

The problem here is that we load the dll via reflection (both in compile and runtime), and .NET needs to load the reference assemblies also when loading with reflection. But there is no easy way to say in .NET that please load also all the other dependant Nuget-packages and search from those paths when doing reflection loading. Loading the first dll .NET will trigger a ResolveEvent for related dlls, and then we try to add some common locations like the execution/bin path and the resolution path, but we really don't know the correct nuget-package-caches-path or anything like that. And if the reference assembly is different version, then we have to do a runtime-binding redirect to the existing assembly (which is not optimal either, but users do expect that any version of their system.data.common.dll or whatever will just work).

I'm all in for any better solution, if you have any ideas how could we do that.

[Thorium]

The reason for reflection is that we don't want to load all the possible database connection drivers for every database, and we don't want to deal possible dependency conflicts between those. We don't want a huge and complex dependency hierarchy for SQLProvider package. We just want whatever version of the driver for the database that the developer is currently using. This did work well, but then came the .NET Core and npm-style download-the-whole-internet dependency-hierarchies for the drivers.

BTW, for MySql I do recommend using MySqlConnector over MySQL.Data, because its performance, if you don't have any special reason to use the official driver.

[johannesegger]

Thanks for the detailed explanation. I'm completely with you on everything you wrote, maybe just have the user define more of these resolution paths or maybe even a resolution function, would that work?

The docs for the MySql drivers say MySqlConnector has less features, so although I probably don't use the special features of MySql.Data I was a bit afraid of MySqlConnector not working properly... I think I could give it a try at least.

Thanks again.

[piaste]

The reason for reflection is that we don't want to load all the possible database connection drivers for every database, and we don't want to deal possible dependency conflicts between those.

Hmm. So (a) if SqlProvider didn't use reflection, compile-time dependency chains would Just Work® and (b) limiting the dependency graph is the only reason to use reflection?

Just thinking out loud: would it be possible to split off each provider into a separate NuGet package, each one dependent on its database drivers? Say:

1. Each provider is just a regular class implementing ISqlProvider and doesn't depend on the TP architecture. It gets made into a NuGet package with explicit dependencies, and can then be written without the need for reflection.

2. The main type provider package gets built with references to each provider sub-package (and indirect references to their respective database connectors), but those references are not defined in the NuGet packages, so they don't get pulled when a user pulls the main package.

3. In the main package, the only point where the main package invokes the types in the sub-packages is in ProviderBuilder.createProvider, so here you'll get a "runtime error" (actually design-time) if you specify a vendor for which you didn't pull the corresponding NuGet sub-package and its dependencies. Otherwise, the reference to (say) SqlProvider.MySql.MySqlConnector will be resolved normally without reflection, and will pull in its own dependencies.

[johannesegger]

I now tried MySqlConnector and I ran into a similar problem. This time System.Threading.Tasks.Extensions couldn't be loaded. I think this is normally loaded from the GAC, but I'm quite sure that I don't have it there.

[Thorium]

Yes, that's a separate Nuget package.

[johannesegger]

Yeah, I know, I just wanted to say that it has the same problem as Mysql.Data.

[Thorium]

This is fixed