[pl_upgrades] OpenFold not configured to support compute capability 9.0

[reedharrison]

I have recently been trying to run OpenFold on NVIDIA H100 cards; however, I ran into problems from the main and pl_upgrades branches that prevented OpenFold from running.

The first issue is that I noticed that "setup.py" from the pl_upgrades branch is not configured to support compute capability 9.0 (the minimum compute capability required by H100 cards). This problem, seems to be an easy fix - just edit "setup.py" to include compute capability 9.0 (example).

The second issue is that I had some issues building a conda/mamba environment that was compatible with OpenFold. I experienced some bugs with pytorch version 2-2.2. It seemed one of the bugs in particular was not fixed until pytorch 2.4.1, so I had to identify an environment compatible with this version of pytorch. If interested, you can see an example of the "environment.yml" that I ended up using here. While this environment works well for me, I didn't do exhaustive testing of all options in the default OpenFold run script. I use a custom run script that doesn't use the new deepspeed evoformer attention option, for example.

[ulupo]

As for your second issue, would using the modified environment.yml from https://github.com/aqlaboratory/openfold/pull/496 help? See also https://github.com/aqlaboratory/openfold/issues/494

[reedharrison]

@ulupo , I'm not sure those changes to the environment.yml would have solved my problem, but I also don't have the bandwidth to test it right now. I suspect that it would not because the version of pytorch is not specified and the pytorch specific issue could still be encountered. I think if it is working for everyone else, then probably fine to ignore the first issue, though, as the issues I encountered might be more specific to the hardware I am working with.

The second issue is probably more important (and simpler?). setup.py just needs to be updated to compile for CC9.0 or OpenFold could not run on H100 cards.

[SundusNav]

I am getting this warning right now. I am running environment.yml from @reedharrison with compute capability (9,0) in setup.py but nothing seems to work.

_Warning: Package 'openfold.utirc' is absent from the packages configuration. !!

    ********************************************************************************
    ############################
    # Package would be ignored #
    ############################
    Python recognizes 'openfold.utils.kernel.csrc' as an importable package[^1],
    but it is absent from setuptools' `packages` configuration.

    This leads to an ambiguous overall configuration. If you want to distribute this
    package, please make sure that 'openfold.utils.kernel.csrc' is explicitly added
    to the `packages` configuration field.

    Alternatively, you can also rely on setuptools' discovery methods
    (for example by using `find_namespace_packages(...)`/`find_namespace:`
    instead of `find_packages(...)`/`find:`).

    You can read more about "package discovery" on setuptools documentation page:

    - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

    If you don't want 'openfold.utils.kernel.csrc' to be distributed and are
    already explicitly excluding 'openfold.utils.kernel.csrc' via
    `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
    you can try to use `exclude_package_data`, or `include-package-data=False` in
    combination with a more fine grained `package-data` configuration.

    You can read more about "package data files" on setuptools documentation page:

    - https://setuptools.pypa.io/en/latest/userguide/datafiles.html


    [^1]: For Python, any directory (with suitable naming) can be imported,
          even if it does not contain any `.py` files.
          On the other hand, currently there is no concept of package data
          directory, all directories are treated like packages.
    ********************************************************************************