pyodide icon indicating copy to clipboard operation
pyodide copied to clipboard

Add package `pyart` and `netcdf4`

Open mgrover1 opened this issue 3 years ago • 11 comments

Description

Package for working with weather radar data. Repo is https://github.com/ARM-DOE/pyart

Also added netcdf4 (closes https://github.com/pyodide/pyodide/issues/1582)

Checklists

  • [x] Add a CHANGELOG entry
  • [x] Add / update tests

mgrover1 avatar Aug 24 '22 18:08 mgrover1

Thanks, @mgrover1. The file name has to be meta.yaml not meta.yml.

ryanking13 avatar Aug 25 '22 00:08 ryanking13

Thanks, @mgrover1. The file name has to be meta.yaml not meta.yml.

Sorry about that - I fixed the name issue!

mgrover1 avatar Aug 25 '22 00:08 mgrover1

Perhaps we ought to add a check that all directories in packages contain a meta.yaml file, aside from a list of known exceptions.

hoodmane avatar Aug 25 '22 01:08 hoodmane

You'll need to add more requirements of pyart (https://github.com/ARM-DOE/pyart/blob/main/requirements.txt).

Unfortunatly, netCDF4 is not included in Pyodide yet (#1582) so you'll need to add that too.

ryanking13 avatar Aug 25 '22 04:08 ryanking13

@ryanking13 - with the h5py and cftime recipe, we should be able to add netcdf4 here too... I created an additional meta + test file here to see if we can get this working

mgrover1 avatar Aug 25 '22 13:08 mgrover1

Tests seem to be passing - think this is ready for review! @ryanking13

mgrover1 avatar Aug 25 '22 15:08 mgrover1

Something definitely doesn't seem right because netcdf4 shouldn't even be passing the build yet, since it requires netcdf-c (libnetcdf). That's been on my list to add, but was hesitant since last I checked there were issues with libhdf5.

dopplershift avatar Aug 25 '22 17:08 dopplershift

Everything looks fine but the tests don't seem to be running correctly @ryanking13. In both test-packages-[browser]-no-numpy-dependents and test-packages-[browser] I see:

arm_pyart/test_pyart.py::pyart[firefox] SKIPPED (package 'arm_pyart'...) [ 54%]

Right, libhdf5 is disabled due to test failure after Emscripten update so netcdf4 is not building at all.

ryanking13 avatar Aug 26 '22 04:08 ryanking13

Ahhh okay - so we can hold off on this until that issue is resolved...

mgrover1 avatar Aug 26 '22 20:08 mgrover1

@mgrover1 We just re-enabled h5py, so your packages will not be skipped now. Could you please merge upstream main branch and re-run CI?

ryanking13 avatar Sep 13 '22 07:09 ryanking13

Okay, the build is now running and it is failing.

HDF5_DIR environment variable not set, checking some standard locations ..

Probably you need to add these into the meta.yaml of netcdf4 to tell the HDF5 installation directory:

requirements:
  host:
    - libhdf5
build:
  script: |
    export HDF5_DIR=${WASM_LIBRARY_DIR}
NETCDF4_DIR environment variable not set, checking standard locations..

And I think you'll also have to build netcdf C library, which is a dependency for netcdf4 Python library.

ryanking13 avatar Sep 13 '22 23:09 ryanking13

@mgrover1 I'm planning on working on netcdf-c one day, or someone smarter than me will do it, but for now you can try to use h5netcdf in pyart_arm instead. That only depends on h5py, which is already compiled for pyodide, and be enough for more cases where netcdf4 is used.

ocefpaf avatar Oct 06 '22 17:10 ocefpaf

I'll give that a try! Thanks @ocefpaf

mgrover1 avatar Oct 06 '22 19:10 mgrover1