deeprank2 icon indicating copy to clipboard operation
deeprank2 copied to clipboard

Conda migration

Open gcroci2 opened this issue 1 year ago • 2 comments

After PR #549, DSSP installation is done via conda. Now the goal is to further clean the installation of deeprank2 and deploy it on conda.

  • [x] Migrate all deps to conda (PR #565)
    • [x] Start with the pytorch geom optional deps
    • [x] Then the rest
    • [x] Deploy pdb2sql on conda
    • [x] Let's try to put >= everywhere in the deps and then only when necessary fix the versions
    • [x] Use the conda installation for both the CI build and the dockerfile
      • We use the PyPi release for now, but the idea is to not rely on pip dependencies anymore. We will have later two separate actions for testing the PyPI and the conda installations, or we will remove the PyPI installation entirely (we'll evaluate that).
  • [ ] Deploy on conda (after #565 will be merged and #570 will be solved)

gcroci2 avatar Jan 19 '24 14:01 gcroci2

I used grayskull to generate the meta.yml file, pointing to the latest PyPI installation of the package.

  • Reasons for not using the PyPI release but directly the Github repo (the latter is also possible with grayskull, see here:
    • It seems more convenient to use PyPI (e.g., repositories are usually larger than tarballs, draining shared CI time and bandwidth, repositories are not checksummed and thus using a tarball has a stronger guarantee that the download that is obtained to build from is in fact the intended package). This of course mean that we can't stop releasing there.
  • We are now investigating how to include deps that are on conda, but not on conda-forge. Relevant links:
    • https://conda-forge.org/docs/maintainer/conda_forge_yml/#channel_priority
    • https://github.com/conda/conda-build/issues/532#issuecomment-1815442500
  • Running the tests locally gives an error. Since we have deps not on conda-forge is expected, but the error seems related to something else. See here
  • Not related to the recipe, but more (I think) to the PyPI release, all the tests/ folders are included in the PyPI release

gcroci2 avatar Mar 01 '24 14:03 gcroci2

This issue is stale because it has been open for 30 days with no activity.

github-actions[bot] avatar Apr 12 '24 03:04 github-actions[bot]

Since the issue discussed in https://github.com/conda/conda-build/issues/532 remains unresolved and there has been no activity on it for a year, we have decided to revert to using pip for installation. We will continue to use conda only for packages that specifically require it.

gcroci2 avatar Jun 17 '24 13:06 gcroci2

Since the issue discussed in conda/conda-build#532 remains unresolved and there has been no activity on it for a year, we have decided to revert to using pip for installation. We will continue to use conda only for packages that specifically require it.

Does this mean this issue can be closed? Or is there still some action needed from this issue?

Before closing, maybe it would make sense to also explain in this issue why poetry wouldn't work either. Just in case in the future someone wants to reconsider moving to either conda or poetry, then the information of why it doesn't presently work is all consolidated in one place.

DaniBodor avatar Jun 18 '24 08:06 DaniBodor

Since the issue discussed in conda/conda-build#532 remains unresolved and there has been no activity on it for a year, we have decided to revert to using pip for installation. We will continue to use conda only for packages that specifically require it.

Does this mean this issue can be closed? Or is there still some action needed from this issue?

I am reverting things in #611

Before closing, maybe it would make sense to also explain in this issue why poetry wouldn't work either. Just in case in the future someone wants to reconsider moving to either conda or poetry, then the information of why it doesn't presently work is all consolidated in one place.

Good point, I'll add it here

gcroci2 avatar Jun 20 '24 09:06 gcroci2

A note about why we didn't used poetry.

Poetry is designed to work with pip dependencies, not conda packages. This is a significant limitation for this project since it relies on several conda packages from specific channels. Our current setup (a yml file for conda dependencies and pip for the rest) is straightforward and easy to understand. Users familiar with conda and pip will find this approach intuitive.

gcroci2 avatar Jun 20 '24 13:06 gcroci2

@gcroci2 , had you looked at uv at all as an alternative to the conda migration? I just came across it (same developers as ruff) and thought that that might be a nice solution. I'm just not sure whether it will ultimately have the same issues as poetry does (I tried scanning through the issues, but couldn't see anything about it).

DaniBodor avatar Nov 20 '24 15:11 DaniBodor

@gcroci2 , had you looked at uv at all as an alternative to the conda migration? I just came across it (same developers as ruff) and thought that that might be a nice solution. I'm just not sure whether it will ultimately have the same issues as poetry does (I tried scanning through the issues, but couldn't see anything about it).

Seems very interesting! We should definitely keep it in mind. But for this package, I don't think it solves the issue about the many conda deps we have (from what I've read it doesn't support conda installations) @DaniBodor

gcroci2 avatar Nov 20 '24 15:11 gcroci2