Missing requirements v1.5.5

[AndreLobato]

These requirements are missing, it was necessary to install before installing OpenDrift:

• xarray
• nc_time_axis
• coloredlogs

[AndreLobato]

Also, to install OilLibrary, I had to install:

• futures

And fix this library version:

• zope.sqlalchemy<1.2

[gauteh]

Hi, how did you install opendrift? These should be included both in environment.yml and in the conda package.

[AndreLobato]

I've cloned the repo, set the tag 1.5.5 and used pip install -r requirements.txt && pip install .

[knutfrode]

Hi,

No that won't work. You should follow this recipe: https://opendrift.github.io/install.html

[AndreLobato]

Well, by only supporting Conda installs, you're basic vendor-locking the package installation as not all Python developers are Conda users. With some small changes to requirements.txt could be installed with pip or python setup.py install without issues. Would have to keep requirements.txt and propagate to install_requires from setup.py file and everything should work.

The OilLibrary and opendrift_landmask_data would have to follow the same idea, for the opendrift_landmask_data I only had to install affine that wasn't declared.

[knutfrode]

Ok, good that you managed.

We had pip-instructions earlier, but left it out in favor of conda, as it was much more straightforward and identical on all platforms. But we should probably re-introduce the pip-instructions anyway. Are there any details you could share about the changes you needed to do - apart from what you already mentioned?

[AndreLobato]

Ok, I think I've managed to do a (near) stable build with pip, out of a Python official Docker image.

FROM python:3.8-buster

RUN apt update && \
    apt install -y libeccodes-dev libeccodes0 libproj-dev proj-bin libgeos-dev && \
    apt -y upgrade && \
    apt clean

RUN pip install -U pip Cython &&\
    pip install \
        # OilLibrary requires
        zope.sqlalchemy\<1.2\
        # opendrift-landmask-data requires
        future\
        # opendrift requires
        cfgrib \
        coloredlogs \
        dask \
        eccodes \
        ffmpeg \
        geojson \
        motuclient \
        nc_time_axis \
        pytest-benchmark\
        toolz\
        xarray \
        xhistogram \
        shapely --no-binary shapely &&\
    # We do this because we don't trust setup.py install_requires
    pip install -r https://raw.githubusercontent.com/OpenDrift/OilLibrary/master/requirements.txt &&\
    pip install -r https://raw.githubusercontent.com/OpenDrift/opendrift-landmask-data/master/requirements.txt

RUN git clone --depth 1 --branch v1.5.5 https://github.com/OpenDrift/opendrift.git /source/opendrift &&\
    cd /source/opendrift &&\
    pip install -r requirements.txt &&\
    pip install .

Fixed (see Edit): I've only had to skip 2 tests that gave segmentation fault, something between Cartopy, Matplotlib and multiprocessing, but I am not sure what, but all other tests from testall passed, although many warnings were raised:

• tests/integration/test_run.py:TestRun.test_plot_animation
• tests/unit/test_readers.py:TestReaders.test_map_background

For propagating the requirements to install_required I usually just .read().splitlines() the requirements file, but is a bit limited approach since you could have comments and some version notations that might not work.

Edit: Figured the issue with the 2 failed tests, turnsout that Shapely needs to be installed not from Pypi wheel, but built from source at install time using the current system GEOS library, that's why need to install Cython first then Shapely with --no-binary shapely option.

Note on GEOS installation: Debian stable (buster) only has GEOS v2.4 but Cartopy requres >v.3.3, which will be only avalable on apt on next Debian release, so it's possible to find some issue there, but at least all tests seems to pass as far OpenDrift is concerned.

I hope it helps

[Boorhin]

@gauteh you saved my day. I cannot stand Conda on Ubuntu, thanks!

[gauteh]

You probably meant @AndreLobato :) ?

[Boorhin]

You probably meant @AndreLobato :) ?

hahahahha yes, sorry :) just to say it worked

[knutfrode]

Did you manage to install it with the recipe of Andre, or did you have to make further tweaks?

We wish that installation with pip would be even smoother and same for all platforms, but will look out for updates towards this end in the future.

[Boorhin]

I followed the recipe, had to downgrade zope.sqlalchemy which was a bit annoying and didn't install shapely as I had done it already No tweak really on Ubuntu 20.04

[gauteh]

There will have to be some major changes to the oil-drift module in the future (but probably not near future), we will then hopefully get rid of several of the problematic dependencies, and maybe even make it easier to create a PIP package.

[Boorhin]

I come back to this as I am trying to deploy Opendrift in a cloud environment. The main annoyance is this SQLAlchemy module. One version working is 1.1.18. Then the Oil library work. pip install SQLAlchemy==1.1.18 The error is

ImportError: cannot import name 'SessionExtension' from 'sqlalchemy.orm.session' (/home/moreau_juli1/.local/lib/python3.8/site-packages/sqlalchemy/orm/session.py)

It would be good to change generation of SQLAlchemy as the 1.3 is fully python3 https://docs.sqlalchemy.org/en/13/intro.html#installation

It seems that it could just be this import command that needs to be changed. Then you can probably install everything with pip which would be a great convenience

[knutfrode]

In OpenDrift v 1.8.0 we have switched to the new version of NOAA OilLibrary (ADIOS), and therefore do not anymore need SQLAlchemy and several other troublesome packages:

https://github.com/OpenDrift/opendrift/blob/master/environment.yml

[Boorhin]

ok I am running 1.8 but oildrift and landmask requirements.txt are still SQLAlchemy I think, let me check

[knutfrode]

ok, yes requirements.txt has not yet been updated. I believe you can simply remove the OilLibrary from there, and possible need to insert pynucos>=2.12

[Boorhin]

made a pull request then