Support HDF5 1.12

[Achilles1515]

HDF5 1.12 was released earlier this year.

Is there a plan in place/work already being done to update this library (and HDF.PInvoke.1.10) to the new version?

[gheber]

'No' on both counts. (Other priorities.) Which 1.12 features are important for .NET? Would it be valuable to organize an HDF5 w/ .NET pow-wow? Does the (currently) Windows-only HDF.PInvoke have a place in a .NET Standard world? We (The HDF Group) are not .NET experts and want to hear from the community. If someone else wants to take the lead, you have an attentive audience.

[hokb]

In a perfect world this repo would host the managed source files + official binaries for all 'major' platforms targeted by .NET today. This would be: Windows, Linux, MacOS, each for x86, x64 and ARM runtimes, where applicable. To provide a nuget package, multitargeting all those platforms with official HDF5 binaries would be a very valuable contribution.

[Apollo3zehn]

This is already done with https://github.com/HDFGroup/HDF.PInvoke.1.10 (except ARM) until version 1.10.6.

[hokb]

You mean, all binaries are included and it builds a multitargeting nuget package which we can consume for building and publishing .NET Framework + .NET Core (/5) projects? How could I miss that !?

[Apollo3zehn]

Yes, it is a .NET Standard 2.0 package. IIRC it contains win-x86, win-x64, linux-x64 and osx-x64 binaries. OSX binaries are built using Travis CI and the rest is built by AppVeyor. This process is triggered in a separate branch. When the native binaries are ready, a push to the main branch triggers the actual build of the final nuget package.

The build process is not perfect but it works. I have tested the package manually on Windows, Ubuntu and CentOS (to ensure the native libs are resolved correctly). I have no Mac available, so no manual tests there.

[hokb]

Excellent! Thank you!

[hokb]

We might add a note to the documentation about some restrictions implied by this packaging approach:

• AnyCPU targets are not supported. Users must specify the platform configuration (x86, x64) explicitly. If AnyCPU targets may work then only because of the fallback to x64. But this will fail in more complicated setups and/or when switching targets.
• .NET Framework publish (click once) not supported.
• Transitive references not supported (if you load the package into a DLL and consume the DLL from, let's say: NET461 project). It is kind of working for transitive references in .NET Core apps and with PackageReference. I don't know how reliable, though. This may be improved with some 'buildTransitive/' trickery in the package.
• One must use PackageReference. packages.config will not work.

[hokb]

PS: the term 'not supported' may sound a little too harsh. In fact, it means that the consumer of the HDF.PInvoke.Netstandard nuget package in such 'unsupported' scenarios must handle the selection and distribution of the native dependencies to the runtime folder of her app on her own. But still, the provisioning of the native libraries for multiple platforms in form of a single package will be very useful even then! This wraps-up the situation: https://github.com/NuGet/docs.microsoft.com-nuget/issues/2070#issuecomment-767706280

[Apollo3zehn]

Thanks for the summary. Please note that the nuget package I was referring to is https://www.nuget.org/packages/HDF.PInvoke.1.10/ (not https://www.nuget.org/packages/HDF.PInvoke.NETStandard/). The confusion comes from the unfortunate situation that we were not able to contact the owner of the "HDF.PInvoke.NETStandard" package and thus we were forced to create another one.

I guess all these problems can be solved with msbuild targets but I am no expert with this an do not have test setups available for .NET Framework and / or packages.config. I thought packages.config was obsolete, but it seems I am wrong.

In case you need the package to support more scenarios, would you be able to create such msbuild targets to copy the files to the right place? Or is it sufficient for you if we simply document the problems you mentioned?

[hokb]

Thanks for the clarification! I don't have a one-for-all solution and I doubt it exists today. What was easy in a .NET Framework world becomes at least unspecified in a portable .NET world. The main open question for me is: when / how to specify the final execution configuration. In the .NET Core philosophy this is determined at publish time or even later. The documentation on the topic is still very much "evolving" and it seems there is no official recommendation as of today. https://github.com/NuGet/Home/issues/6645 For now we will fall back to this very simple approach: restrict/assume projects to x64 and distribute all native binaries as project items <None> to go into the output folder - all 3 platforms next to each other.

The confusion comes from the unfortunate situation that we were not able to contact the owner of the "HDF.PInvoke.NETStandard" package and thus we were forced to create another one.

@gheber you may consider registering HDF.* as a registered prefix for your organisation. This could help to prevent from such confusion in the future (and maybe even for existing projects): https://docs.microsoft.com/en-us/nuget/nuget-org/id-prefix-reservation

Or is it sufficient for you if we simply document the problems you mentioned?

As a package consumer I would appreciate a corresponding note on this.

[Achilles1515]

@Apollo3zehn FYI, I just put in a PR to the HDF.PInvoke.1.10 project to address @hokb 's desires to have proper .NET Framework support, as .NET Framework 4.6.1+ is .NET Standard 2.0 compliant and should be supported.

AnyCPU builds are supported, as well as packages.config

https://github.com/HDFGroup/HDF.PInvoke.1.10/pull/19

[Apollo3zehn]

@hokb there is now a new version of HDF.PInvoke.1.10 which includes a targets file to support (hopefully all) .NET Framework configurations. Credits go to @Achilles1515.