pyodide icon indicating copy to clipboard operation
pyodide copied to clipboard

Add pygmt

Open weiji14 opened this issue 2 years ago • 3 comments

Description

Placeholder for adding PyGMT (a library for processing geospatial data and making maps) to Pyodide.

TODO, include 2 runtime C dependencies, xref https://github.com/conda-forge/pygmt-feedstock/blob/b7ecf50df60e8e15cbf5d47122a7beb4ce349d9c/recipe/meta.yaml#L24-L31:

  • [ ] gmt (https://github.com/GenericMappingTools/gmt)
  • [ ] netCDF4, xref #1582

Other Python dependencies:

  • [x] numpy >=1.19
  • [x] packaging
  • [x] pandas
  • [x] xarray (https://github.com/pyodide/pyodide/pull/2538)

Notes on testing this branch

  1. Follow https://pyodide.org/en/latest/development/new-packages.html, get pyodide development environment set-up
  2. git clone this branch
  3. cd packages/pygmt/
  4. python -m pyodide_build buildpkg meta.yaml
  5. cd ../../
  6. PYODIDE_PACKAGES="pygmt" make
  • May need to do sudo apt install gfortran f2c ccache build-essential and so on, ref https://pyodide.org/en/latest/development/building-from-sources.html#partial-builds
  1. ...

Checklists

  • [ ] Add a CHANGELOG entry
  • [ ] Add / update tests
  • [ ] Add new / update outdated documentation

weiji14 avatar May 08 '22 19:05 weiji14

Note to self, get libgmt packaged, and update/patch code on PyGMT to fix this error https://app.circleci.com/pipelines/github/pyodide/pyodide/4095/workflows/6cd9a4d7-2f51-4f0b-ae57-15cd7be70b42/jobs/45300?invite=true#step-102-701:

----------------------------- Captured stdout call -----------------------------
JavascriptException message:  Traceback (most recent call last):
  File "/lib/python3.10/asyncio/futures.py", line 201, in result
    raise self._exception
  File "/lib/python3.10/asyncio/tasks.py", line 232, in __step
    result = coro.send(None)
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 500, in eval_code_async
    await CodeRunner(
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 351, in run_async
    coroutine = eval(self.code, globals, locals)
  File "<exec>", line 1, in <module>
  File "/lib/python3.10/site-packages/pygmt/__init__.py", line 71, in <module>
    _begin()
  File "/lib/python3.10/site-packages/pygmt/session_management.py", line 16, in begin
    with Session() as lib:
  File "/lib/python3.10/site-packages/pygmt/clib/session.py", line 183, in __enter__
    self.create("pygmt-session")
  File "/lib/python3.10/site-packages/pygmt/clib/session.py", line 330, in create
    c_create_session = self.get_libgmt_func(
  File "/lib/python3.10/site-packages/pygmt/clib/session.py", line 282, in get_libgmt_func
    self._libgmt = load_libgmt()
  File "/lib/python3.10/site-packages/pygmt/clib/loading.py", line 47, in load_libgmt
    for libname in lib_fullnames:
  File "/lib/python3.10/site-packages/pygmt/clib/loading.py", line 107, in clib_full_names
    libnames = clib_names(os_name=sys.platform)  # e.g. libgmt.so, libgmt.dylib, gmt.dll
  File "/lib/python3.10/site-packages/pygmt/clib/loading.py", line 85, in clib_names
    raise GMTOSError(f"Operating system '{os_name}' not supported.")
pygmt.exceptions.GMTOSError: Operating system 'emscripten' not supported.

weiji14 avatar May 11 '22 13:05 weiji14

@weiji14 this would be awesome! The major challenge will likely be getting GMT in first and all its dependencies. From the pyodide contributing docs it seems that C dependencies should be added separately and then included as dependencies here. So the way forward is probably to try to get all of the GMT dependencies in first: https://github.com/conda-forge/gmt-feedstock/blob/main/recipe/meta.yaml The ones that usually cause problems are GDAL and ghostscript.

leouieda avatar May 12 '22 09:05 leouieda

@weiji14 this would be awesome!

Yes, the goal is to have an in-browser version of https://github.com/GenericMappingTools/try-gmt. No install needed to try out PyGMT :smile:

The major challenge will likely be getting GMT in first and all its dependencies. From the pyodide contributing docs it seems that C dependencies should be added separately and then included as dependencies here. So the way forward is probably to try to get all of the GMT dependencies in first: https://github.com/conda-forge/gmt-feedstock/blob/main/recipe/meta.yaml The ones that usually cause problems are GDAL and ghostscript.

No kidding, this definitely won't be easy, but let's take it one step at a time. There should be some smart people working on the GDAL port (e.g. the geopandas folks at https://github.com/pyodide/pyodide/issues/1569) since that's the basis of most geo-libraries. Ghostscript might be a tougher sell to port to WebAssembly, but hopefully doable (fingers crossed that PyQT doesn't creep in).

I don't have much dedicated time to push this forward, so if anyone wants to build on this, let me know and I can give you access to the branch at https://github.com/weiji14/pyodide/tree/pygmt to work on this (or you can fork a copy and work on it independently).

weiji14 avatar May 31 '22 19:05 weiji14