recipes icon indicating copy to clipboard operation
recipes copied to clipboard

Add h5py

Open talmo opened this issue 1 year ago • 6 comments

  • Addresses https://github.com/emscripten-forge/recipes/issues/1352
  • Based off of https://github.com/emscripten-forge/recipes/pull/685
  • Updated recipe to current rattler-build syntax
  • Updated to h5py=3.11.0
  • Using this monkey patch for removing -R flags from emcc calls.
  • ~Assuming hdf5=1.12.2. I think we have 1.12.3 available in emscripten-forge, but the older version is getting pulled in by the CI and there is a feature flag in an env var that breaks the API if the version isn't set correctly.~
  • Added libhdf5=1.12.1 as a shared library rather than the static hdf5 one. This is based on the pyodide recipe.

talmo avatar Sep 29 '24 08:09 talmo

At this point I think I'm stuck. The package builds, but then we get loads of errors deep in HDF5. There are plenty of suspicious warnings earlier on, but unclear where it's coming from.

Maybe @DerThorsten or @bmaranville have some ideas?

talmo avatar Sep 29 '24 10:09 talmo

Perhaps the hdf5 package in this repo needs to be built as a shared library like libhdf5 over in Pyodide?

talmo avatar Sep 29 '24 11:09 talmo

At this point I think I'm stuck. The package builds, but then we get loads of errors deep in HDF5. There are plenty of suspicious warnings earlier on, but unclear where it's coming from.

Maybe @DerThorsten or @bmaranville have some ideas?

I might find some time to look into this, but I would also need to do some major debugging / experimentation

DerThorsten avatar Oct 01 '24 07:10 DerThorsten

Sorry, I can't be much help until I understand how this very specific build process works...

I went through the instructions for doing a local build, but after I build libhdf5 and then h5py, I get an error:

rattler-build build  --recipe recipes/recipes_emscripten/h5py/recipe.yaml  --target-platform=emscripten-wasm32 -c https://repo.mamba.pm/emscripten-forge -c conda-forge -c microsoft -m conda_build_config.yaml
...
Error building package: Failed to resolve dependencies: Cannot solve the request because of: No candidates were found for libhdf5 *

How do you tell the h5py build to use the locally-built libhdf5?

bmaranville avatar Oct 01 '24 23:10 bmaranville

I think the issue is that we're building two conda packages successfully (libhdf5 and h5py), but we're not telling the second one (h5py) where to find its dependency (libhdf5).

With conda-build I think you have to set the local build directory as an additional channel to look for locally-built packages, but I'm not so sure how to do that in rattler-build.

Alternatively, we could get libhdf5 merged in and uploaded to the repository so that h5py can find it, but it'd be nice to be able to test it first before uploading.

Maybe @DerThorsten or @wolfv can provide some quick assistance? I think the hard part is done (the packages individually are getting built!).

talmo avatar Oct 01 '24 23:10 talmo

@talmo I hope I will find some time to debug this, but cannot promise to do It this week!

DerThorsten avatar Oct 02 '24 06:10 DerThorsten

closing this since #2235 landed (#2235 is very much based on this PR) Thanks for your work and sorry for the long time it took to get h5py on emscripten-forge

DerThorsten avatar Apr 08 '25 12:04 DerThorsten