PyPI and/or conda-forge packages

[tbenthompson]

Would there be any interest in having a conda-forge package or PyPI package or both?

I notice that this was raised previously in issue #215. I think the big advantage of these package managers over the current git installation method would be automated dependency management especially when Devito is just one component of a larger system. Then Devito could be installed as a dependency of another package either via conda or requirements.txt. It's also just nice to fit into the standards of the Python community.

Another concrete use case: I'd like to be able to run Devito inside of a Google Colab notebook as part of an educational tutorial. It'd be nice to have just a simple !conda install -c conda-forge devito or !pip install devito at the top of the notebook that will install Devito and all its dependencies.

I've contributed a few packages to conda-forge before so I would be happy to set up a draft PR. Ultimately though, it would be best if one of the Devito maintainers could be involved so we could integrate the release process smoothly.

THANKS!

[mloubout]

devito is already a PyPI package and can be installed via pip installed devito

https://pypi.org/project/devito/

[tbenthompson]

Oh, wow. I thought I had done my research but I clearly hadn't...

Any interest in a conda-forge package then?

(quick edit: conda-forge packages tend to play more nicely when there are non-python dependencies like OpenMP!)

[mloubout]

I suppose so yes. We have it automatize via a GitHub action for PyPI so adding conda to it would be good

[tbenthompson]

pip install devito worked for me on Colab but I get some funky dependency problems where the end result is that pip runs through all the pytest versions back until 4.6.0. See below. This also happens with the coverage package. I'm guessing there's some kind of version disagreement. Just sharing in case it's interesting.

Collecting pytest<7.0,>=3.6
  Downloading pytest-6.2.4-py3-none-any.whl (280 kB)
     |████████████████████████████████| 280 kB 44.5 MB/s 
  Downloading pytest-6.2.3-py3-none-any.whl (280 kB)
     |████████████████████████████████| 280 kB 46.3 MB/s 
  Downloading pytest-6.2.2-py3-none-any.whl (280 kB)
     |████████████████████████████████| 280 kB 36.7 MB/s 
  Downloading pytest-6.2.1-py3-none-any.whl (279 kB)
     |████████████████████████████████| 279 kB 43.0 MB/s 
  Downloading pytest-6.2.0-py3-none-any.whl (279 kB)
     |████████████████████████████████| 279 kB 59.3 MB/s 
  Downloading pytest-6.1.2-py3-none-any.whl (272 kB)
     |████████████████████████████████| 272 kB 55.6 MB/s 
  Downloading pytest-6.1.1-py3-none-any.whl (272 kB)
     |████████████████████████████████| 272 kB 51.0 MB/s 
  Downloading pytest-6.1.0-py3-none-any.whl (272 kB)
     |████████████████████████████████| 272 kB 53.6 MB/s 
  Downloading pytest-6.0.2-py3-none-any.whl (270 kB)
     |████████████████████████████████| 270 kB 46.0 MB/s 
  Downloading pytest-6.0.1-py3-none-any.whl (270 kB)
     |████████████████████████████████| 270 kB 31.8 MB/s 
  Downloading pytest-6.0.0-py3-none-any.whl (270 kB)
     |████████████████████████████████| 270 kB 43.5 MB/s 
  Downloading pytest-5.4.3-py3-none-any.whl (248 kB)
     |████████████████████████████████| 248 kB 42.5 MB/s 
  Downloading pytest-5.4.2-py3-none-any.whl (247 kB)
     |████████████████████████████████| 247 kB 44.1 MB/s 
  Downloading pytest-5.4.1-py3-none-any.whl (246 kB)
     |████████████████████████████████| 246 kB 49.1 MB/s 
  Downloading pytest-5.4.0-py3-none-any.whl (247 kB)
     |████████████████████████████████| 247 kB 51.1 MB/s 
  Downloading pytest-5.3.5-py3-none-any.whl (235 kB)
     |████████████████████████████████| 235 kB 36.8 MB/s 
  Downloading pytest-5.3.4-py3-none-any.whl (235 kB)
     |████████████████████████████████| 235 kB 49.8 MB/s 
  Downloading pytest-5.3.3-py3-none-any.whl (235 kB)
     |████████████████████████████████| 235 kB 47.0 MB/s 
  Downloading pytest-5.3.2-py3-none-any.whl (234 kB)
     |████████████████████████████████| 234 kB 39.4 MB/s 
  Downloading pytest-5.3.1-py3-none-any.whl (233 kB)
     |████████████████████████████████| 233 kB 36.0 MB/s 
  Downloading pytest-5.3.0-py3-none-any.whl (233 kB)
     |████████████████████████████████| 233 kB 45.0 MB/s 
  Downloading pytest-5.2.4-py3-none-any.whl (227 kB)
     |████████████████████████████████| 227 kB 42.3 MB/s 
  Downloading pytest-5.2.3-py3-none-any.whl (227 kB)
     |████████████████████████████████| 227 kB 41.2 MB/s 
  Downloading pytest-5.2.2-py3-none-any.whl (227 kB)
     |████████████████████████████████| 227 kB 48.1 MB/s 
  Downloading pytest-5.2.1-py3-none-any.whl (226 kB)
     |████████████████████████████████| 226 kB 47.3 MB/s 
  Downloading pytest-5.2.0-py3-none-any.whl (226 kB)
     |████████████████████████████████| 226 kB 46.0 MB/s 
  Downloading pytest-5.1.3-py3-none-any.whl (224 kB)
     |████████████████████████████████| 224 kB 42.8 MB/s 
  Downloading pytest-5.1.2-py3-none-any.whl (224 kB)
     |████████████████████████████████| 224 kB 57.7 MB/s 
  Downloading pytest-5.1.1-py3-none-any.whl (223 kB)
     |████████████████████████████████| 223 kB 38.5 MB/s 
  Downloading pytest-5.1.0-py3-none-any.whl (223 kB)
     |████████████████████████████████| 223 kB 46.9 MB/s 
  Downloading pytest-5.0.1-py3-none-any.whl (221 kB)
     |████████████████████████████████| 221 kB 49.3 MB/s 
  Downloading pytest-5.0.0-py3-none-any.whl (221 kB)
     |████████████████████████████████| 221 kB 48.9 MB/s 
  Downloading pytest-4.6.11-py2.py3-none-any.whl (231 kB)
     |████████████████████████████████| 231 kB 36.7 MB/s 
  Downloading pytest-4.6.10-py2.py3-none-any.whl (231 kB)
     |████████████████████████████████| 231 kB 47.5 MB/s 
  Downloading pytest-4.6.9-py2.py3-none-any.whl (231 kB)
     |████████████████████████████████| 231 kB 46.0 MB/s 
  Downloading pytest-4.6.8-py2.py3-none-any.whl (230 kB)
     |████████████████████████████████| 230 kB 47.8 MB/s 
  Downloading pytest-4.6.7-py2.py3-none-any.whl (230 kB)
     |████████████████████████████████| 230 kB 44.6 MB/s 
  Downloading pytest-4.6.6-py2.py3-none-any.whl (230 kB)
     |████████████████████████████████| 230 kB 49.6 MB/s 
  Downloading pytest-4.6.5-py2.py3-none-any.whl (230 kB)
     |████████████████████████████████| 230 kB 46.8 MB/s 
  Downloading pytest-4.6.4-py2.py3-none-any.whl (229 kB)
     |████████████████████████████████| 229 kB 47.2 MB/s 
  Downloading pytest-4.6.3-py2.py3-none-any.whl (229 kB)
     |████████████████████████████████| 229 kB 46.9 MB/s 
  Downloading pytest-4.6.2-py2.py3-none-any.whl (229 kB)
     |████████████████████████████████| 229 kB 46.4 MB/s 
  Downloading pytest-4.6.1-py2.py3-none-any.whl (229 kB)
     |████████████████████████████████| 229 kB 46.6 MB/s 
  Downloading pytest-4.6.0-py2.py3-none-any.whl (229 kB)
     |████████████████████████████████| 229 kB 34.5 MB/s 

[mloubout]

Ok interesting thanks. I'll look into puting it in conda, any pointers for a quick start?

[FabioLuporini]

wth is going on with pytest here? 😮

[georgebisbas]

The google collab default python has already some packages installed, such as tensorflow, theano and others. These packages have their own dependencies for versions that devito may not be supporting. On the other hand devito may install dependency that depends upon already installed packages. Each of these dependencies have their own ones and this leads to a cyclic dependency graph of downgrading packages.

I would say either ignore existing ones and update them to match devito's requirements: e.g. with: !pip install --ignore-installed --no-cache-dir --upgrade devito or try to create some clean python env in collab if there is a way

[georgebisbas]

Grayskull can help: https://conda-forge.org/docs/maintainer/adding_pkgs.html#meta-yaml