Use Poetry for development

[PythonFZ]

Using setuptools has many downsides described elsewhere in much detail. Many of them can be solved by using Poetry. Therefore, I propose changing MDSuite to use poetry as well.

This would only require some small changes in the setup and no code changes at all. Some of the benefits would be:

• dependency lock file
• no MANIFEST.in / setup.py / requirements.txt / dev-requirements but only pyproject.toml
• better dependency resolving

Changes for the users:

• none, can still be installed via pip install . or pip install mdsuite

Changes for developers:

• use poetry install instead of pip install -e . for development installation
• all requirements are fixed in the poetry.lock file so the excat version is always known

I can make the changes but we should decide on it first @SamTov

[SamTov]

Are there any downsides / bugs. For example, conda environments can be built from requirements files which is no the worst thing to have the ability to do. I guess however, that poetry also interfaces somehow?

@christophlohrmann @Fratorhe

Maybe we can all throw an opinion at it.

Just to clarify my opinion, if there are no downsides or at least no unmanageable downsides and am pro.

[PythonFZ]

@laigner might have also some opinion on this topic. I don't excactly know how poetry and conda will interface but I'd assume it should work the same as with setuptools? Do you mean conda environments or conda packages? For the environments installing from pyproject.toml is an officially accepted PEP and not something poetry specific.

In my experience with ZnTrack there might be some open questions in the beginning just like there where when using setuptools the first time, but nothing that can not be resolved.

Feel free to ask any question that comes to mind.

[christophlohrmann]

I have never used poetry and did not have any problems with setuptools so far. So I do not have an opinion on which we should use

[laigner]

Because I was mentioned, my opinion on Poetry. It is really nice for a developer because you have less files and it is better in handling package dependencies but ... I had in the past as a user problems with the poetry shells (they were too often somehow broken). Pip install is the workaround here. Another issue I faced is the compatibility for m1 apple machines, conda has at the time a pretty broad support and arm wheels for packages which pip not support (my knowledge)

These are the two reasons why I still stick to the pip -e . install within conda environment but that is still doable beside poetry.

[SamTov]

Overall @PythonFZ and @laigner in your experience with ZnTrack did you find it a worthwhile thing to do? And what was the biggest impact you found?

[PythonFZ]

I think it is worthwhile for a few reasons:

• I like a single file over multiples and therefore, supporting the poetry project
• maintaining requirements in a single file compared to two/three files is much simpler and reduces mistakes, e.g. having something in dev and production.
• I really like the idea of a lock file for dependencys and think this avoids issues in the long term for development.
• setting up an environment is super easy for dev/test
• I think the dependency handling in general feels more reliable and might find issues early compared to setuptools

But there are also a few downsides that I found:

• learning curve, altough this should not be an issue for any of us
• can't use pip install -e which we used in the past, this can be a little annoying in the beginning
• using poetry and conda is not straight forward and needs to be documented, because poetry always tries to make its own environment.
• I don''t know how well it works with conda on mac but the upside here could be, that we can allow for a custom mdsuite[m1] easily that will do a tested install on mac m1.

[SamTov]

Let's give it a week to decide as I'd like to do a bit of research to see if I can see any clear issues.

[PythonFZ]

One more benefit of poetry is that it automatically installs dev requirements when making a dev installation via poetry install and does not require pip install -r dev-requirements.txt

[PythonFZ]

@SamTov If you want to make yourself familiar check out https://github.com/zincware/MDSuite/tree/528-use-poetry-for-development

It only took 15 mins and I wanted to test something out anyways.