BioSimSpace
BioSimSpace copied to clipboard
michellab
A easy way of setting up the development env in conda
Is your feature request related to a problem? Please describe.
I'm having a M1 mac, which makes it a bit difficult to follow the guide in https://biosimspace.org/install.html#for-developers to set up the developing environment. To be more exact, the conda cannot find the aarch64 version of MDAnalysis, so the install cannot complete.
On the one hand, this issue could probably be solved by add arch -x86_64 to a bunch of commands, but it is a bit messy.
The way that I usually deal with this is that I have a conda which runs on top of the transition layer (which makes everything happening inside this conda think it runs on x86). So as long as I'm using this conda, I don't need any extra caution and it would be an easy way out if we could just use conda to setup the development env.
Describe the solution you'd like The way that I'm doing it now is
# Install the dependencies
conda create -n biosimspace -c michellab/label/dev -c omnia -c conda-forge biosimspace python=3.7 --only-deps
conda activate biosimspace
# Install more dependencies
conda install gsl=2.7.0 conda
# pull the repo down
git clone https://github.com/michellab/BioSimSpace.git
cd BioSimSpace/python
python setup.py develop
This way works but it would install another conda inside the conda env which confuses IDE like pycharm.
To be more exact, python setup.py develop expect conda to be in ~/miniconda3_x86/envs/biosimspace/bin/conda and cannot handle conda in other places such as ~/miniconda3_x86/bin/conda. I know the reason for this is that I'm not using a standard way to set up the env. I think it would be the best if we could set up the conda BSS testing environment as
# pull the repo down
git clone https://github.com/michellab/BioSimSpace.git
# Install the dependencies
cd BioSimSpace
conda create -n biosimspace -f environments.yml
conda activate biosimspace
cd python
python setup.py develop
Hi there,
We've added support for M1 mac via the introduction of a ModuleStub class within BioSImSpace. This means that packages that are not available for a particular architecture, e.g. aarch64, don't trigger an ImportError, rather return an error to the user only when they try to use the package. In the setup.py we simply don't install dependencies if they are not available for a given architecture.
The issue is that the Sire conda-package isn't currently built for aarch64, since it's not yet available as a GitHub CI runner. As you've found, this makes it hard to set-up a conda development environment on an M1 mac. Hopefully support for aarch64 will be added in the near future.
Personally, I'd recommend installing Sire from source following the instructions here. This does work on the M1 chips, by default creating a conda environment at ~/sire.app. You can then install BioSimSpace into it using the setup.py, which would only install the dependencies that are available for aarch64.
When developing, I typically don't repeatedly re-install the package into the conda environment, rather use PYTHONPATH to point at my local source, e.g, when launching IPython, I'd do something like:
PATH=$HOME/sire.app/bin:$PATH PYTHONPATH=$HOME/Code/BioSimSpace/python $HOME/sire.app/bin/ipython
Just to note, you shouldn't be creating an environment with:
conda create -n biosimspace -c michellab/label/dev -c omnia -c conda-forge biosimspace python=3.7 --only-deps
We've now switched to using the conda-forge version of OpenMM, so the command would be:
conda create -n biosimspace -c conda-forge -c michellab/label/dev biosimspace python=3.7 --only-deps
(Let me know if this is incorrect anywhere in the docs and I'll update it.)
@lohedges Thanks for the explanation.
So on the M1 mac, I could install the Sire normally on M1 mac. The tricky bit is installing the BSS, which in the last time, that I have tried, it will fail if during the installation of MDAnalysis.
The ModuleStub don't quite work for me as I really need MDAnalysis for my work.
Thanks for the advice on channel specification.
I guess I will take it that for the moment, there isn't a plan to allow us to setup the env in this sense
# pull the repo down
git clone https://github.com/michellab/BioSimSpace.git
# Install the dependencies
cd BioSimSpace
conda create -n biosimspace -f environments.yml
conda activate biosimspace
cd python
python setup.py develop
If you need MDAnalysis for your work then you are currently stuck until they release a package for aarch64.
Once we have an aarch64 Sure package then you will be able to set up a conda development environment as you suggest, either using the only-deps option, or with a development environment file, which we could provide.
Cheers.
As mentioned, BioSimSpace shouldn't fail installing MDAnalysis on aarch64. However, you won't be able to use it, as no package exists. (It would tell you this when trying to use MDAnalysis functionality within BioSimSpace.)
@lohedges Sorry, I think there might be some misunderstanding here. Just to make sure that we are on the same page.
I wonder if you have tested that the current guide https://biosimspace.org/install.html#for-developers will work for the M1 mac? I have changed a lot of things on my Mac so just to be sure that it is the issue with my Mac.
For the x86, are you saying that there exists a solution which allows one to do
# pull the repo down
git clone https://github.com/michellab/BioSimSpace.git
# Install the dependencies
cd BioSimSpace
conda create -n biosimspace -f environments.yml
conda activate biosimspace
cd python
python setup.py develop
?
I don't have an M1 mac, but I believe @chryswoods has effectively followed the approach that you link to in the developers' guide successfully, i.e. installing Sire from source, then installing BioSimSpace and dependencies into the Sire miniconda.
If you want to use conda as a development environment on x86, you can do:
# Clone the repo.
git clone https://github.com/michellab/BioSimSpace.git
cd BioSimSpace
# Set-up a BioSimSpace conda environment.
conda create -n biosimspace -c conda-forge -c michellab/label/dev biosimspace --only-deps
conda activate biosimspace
# Build BioSimSpace within the environment.
cd python
python setup.py develop
Dependencies are only installed via conda when running python setup.py install, although it's still possible to skip installation be setting the BSS_SKIP_DEPENDENCIES environment variable.
@lohedges Thank you. It is kind of weird that it doesn't work on my computer.
Sorry for keep asking this question. I wonder if you have tested this?
# Clone the repo.
git clone https://github.com/michellab/BioSimSpace.git
cd BioSimSpace
# Set-up a BioSimSpace conda environment.
conda create -n biosimspace -c conda-forge -c michellab/label/dev biosimspace --only-deps
conda activate biosimspace
# Build BioSimSpace within the environment.
cd python
python setup.py develop
On my computer, I would need to do conda install gsl=2.7.0 conda otherwise the python setup.py develop won't work.
So if it works on your computer, I would know that it is the issue with my env.
Yes, I can confirm that it works for me on Linux. I'm really confused why you need to have an additional install command for gsl, since it is pulled in as a dependency of Sire, e.g. from the dependency resolution I get:
...
gsl conda-forge/linux-64::gsl-2.7-he838d99_0
...
Also, why do you need to update conda itself once the environment has been activated?
@lohedges Ok, then this might be a Mac issue or that the transition layer is not applied properly.
If I don't have gsl=2.7.0 then I got the error
Traceback (most recent call last):
File "setup.py", line 11, in <module>
import Sire.Base
File "/Users/zwu/miniconda3_x86/envs/biosimspace/lib/python3.7/site-packages/Sire/__init__.py", line 186, in <module>
from . import Config
File "/Users/zwu/miniconda3_x86/envs/biosimspace/lib/python3.7/site-packages/Sire/Config/__init__.py", line 7, in <module>
import Sire.Base
File "/Users/zwu/miniconda3_x86/envs/biosimspace/lib/python3.7/site-packages/Sire/Base/__init__.py", line 15, in <module>
from Sire.Base._Base import *
ImportError: dlopen(/Users/zwu/miniconda3_x86/envs/biosimspace/lib/python3.7/site-packages/Sire/Base/_Base.so, 0x0002): Library not loaded: @rpath/libgsl.25.dylib
Referenced from: /Users/zwu/miniconda3_x86/envs/biosimspace/lib/libSireCAS.2022.2.0.dylib
Reason: tried: '/Users/zwu/build/gromacs/2022/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/bin/.:@executable_path/../lib:@executable_path/../bundled/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/bin/../lib:@executable_path/../bundled/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/lib/python3.7/site-packages/Sire/Base/../../../../libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/bin/../lib:@executable_path/../bundled/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/lib/python3.7/site-packages/Sire/Base/../../../../libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/bin/../lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/bin/../lib/libgsl.25.dylib' (no such file), '/usr/local/lib/libgsl.25.dylib' (no such file), '/usr/lib/libgsl.25.dylib' (no such file)
If I don't install a separate conda in the env, then I got the error
Checking for dependencies that are already installed...
Traceback (most recent call last):
File "setup.py", line 92, in <module>
if not is_installed(dep, conda="%s/conda" % bin_dir):
File "setup.py", line 32, in is_installed
p = subprocess.Popen([conda, "list", dep], stdout=subprocess.PIPE)
File "/Users/zwu/miniconda3_x86/envs/biosimspace/lib/python3.7/subprocess.py", line 800, in __init__
restore_signals, start_new_session)
File "/Users/zwu/miniconda3_x86/envs/biosimspace/lib/python3.7/subprocess.py", line 1551, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/zwu/miniconda3_x86/envs/biosimspace/bin/conda': '/Users/zwu/miniconda3_x86/envs/biosimspace/bin/conda'
As Conda is usually installed in ~/miniconda3_x86/bin/conda
Hmm, I'm not exactly sure what your transition layer_is doing. Perhaps it's setting the paths incorrectly, or installing other packages into the environment that are downgrading dependencies of BioSImSpace. (It looks like you have a different gsl elsewhere.) I would go through things step by step, checking the version of gsl that is installed into your biosimspace environment at each step.
The second error is strange, since that section of the code shouldn't even be reached if you run python setup.py develop. It isn't for me, i.e. the dependencies will only be checked and installed when running python setup.py install.
For reference, I've just tested on an Intel macBook at it also works fine.
@lohedges With regard to gsl, the gsl installed during
is pkgs/main/osx-64::gsl-2.7.1-h0cefce6_0
It seems that only gsl-2.7.0 could be recognised by BSS.
Sorry, the conda dependency is not needed when one do python setup.py develop. I got this issue when I was doing python setup.py install and I didn't checked that this is still an issue with python setup.py develop.
Thanks, I'll check what's going on with the gsl version on macOS. From your error, it looks like a minor version number change of the package, i.e. 2.7.0 to 2.7.1 has resulted in a change to the dylib version number, which shouldn't happen.
I don't see the issue on our CI, which builds against Linux and Intel macOS. Also, looking at the conda-forge gsl page it looks like the package hasn't been updated for 9 months or so, which is way before the Sire package that you are using was built.
Could you look in ~/.conda/envs/biosimspace/libforlibgsl` and see what version is reported? On Linux I have:
/home/lester/.conda/envs/biosimspace-dev/lib/libgsl.so -> libgsl.so.25.1.0
I don't understand why your error states "no such" file for each of the matching libraries that it tries, e.g:
Reason: tried: '/Users/zwu/build/gromacs/2022/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/bin/.:@executable_path/../lib:@executable_path/../bundled/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/bin/../lib:@executable_path/../bundled/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/lib/python3.7/site-packages/Sire/Base/../../../../libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/bin/../lib:@executable_path/../bundled/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/lib/python3.7/site-packages/Sire/Base/../../../../libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/bin/../lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace/bin/../lib/libgsl.25.dylib' (no such file), '/usr/local/lib/libgsl.25.dylib' (no such file), '/usr/lib/libgsl.25.dylib' (no such file)
I think the issue is that you aren't prioritising the conda-forge channel for whatever reason. The defaults channel does have gsl 2.7.1. (See here.)
There is no guarantee that this will have the same dylib version, since it might, e.g., use a different build system, or target a different macOS version. I'd check your environment channel setup and make sure that you are creating the BioSimSpace development environment as follows:
conda create -n biosimspace -c conda-forge -c michellab/label/dev biosimspace --only-deps
Here the highest priority channel is conda-forge.
Interesting. I've not seen this problem on my machine (also an Apple M1). When developing on M1 you have to be careful which architecture you are using. By default, it is best to use the standard X86-64 miniconda and install Sire / BioSimSpace in that. This will install the X86-64 versions of everything, and will work identically to an installation on an older X86-64 Mac. The cost is a small performance hit (about 10-15%) as Apple's built-in Rosetta tool converts from X86-64 instructions to ARM64 instructions dynamically at runtime.
You can compile and install a native Apple M1 version of Sire / BioSimSpace. To do this, you have to download an ARM64 version of miniconda/minimamba and then use compile_sire.sh to compile Sire yourself. We don't yet ship this as a conda package, mainly because M1 is not supported yet on GitHub Actions (it should be soon).
The upside of using the M1 version is that it is faster. The downside is that you will miss conda packages (like MDAnalysis) that don't yet exist for M1 (hence why we have the ModuleStub interface, as Lester described above).
For your case, it sounds like the X86-64 version is the best to go with. I wonder though if the "no such file" errors that Lester is mentioning is because somehow an X86-64 library is being searched for from an ARM64 conda installation (or the other way around?). The error message in this case isn't helpful, as it tends to just be "no such" file, even though the file clearly exists. The error is because the X86-64 loader (in the X86-64 conda example) cannot find X86-86 instructions in the libgsl file that was installed for ARM64, or the ARM64 loader (in the ARM64 conda example) cannot find the ARM64 instructions in the libgsl file that was installed for X86-64.
To check, use;
file /Users/zwu/build/gromacs/2022/lib/libgsl.25.dylib
and also
file `which python`
to find the architectures of your files. I get
file ~/sire.app/lib/libgsl.dylib
/path/to/sire.app/lib/libgsl.dylib: Mach-O 64-bit dynamically linked shared library arm64
file `which python`
/path/to/sire.app/bin/python: Mach-O 64-bit executable arm64
which shows that my python and libgsl are both arm64.
Be careful with conda environments as I have found that multiple conda activates can leave old paths and library paths in your shell, meaning that the wrong python, wrong conda or wrong library paths are used, which can mess things up.
In general, I recommend always using a clean conda for development, e.g. just a minimamba or miniconda downloaded from conda-forge, and then do all development in that.
@lohedges Thank you. So I have attached the full log of setting up the env, where the conda is running on top of the transition layer.
(base) zwu@EXS-001525 ~ % conda create -n biosimspace_test -c conda-forge -c michellab/label/dev biosimspace --only-deps
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /Users/zwu/miniconda3_x86/envs/biosimspace_test
added / updated specs:
- biosimspace
- configargparse
- fkcombu
- lomap2
- mdanalysis
- mdtraj
- networkx
- nglview
- openff-toolkit-base
- parmed
- py3dmol
- pydot
- pygtail
- pypdb
- pytest
- python[version='>=3.9,<3.10.0a0']
- python_abi=3.9[build=*_cp39]
- pyyaml
- rdkit
- sire[version='>=2022.2.0,<2023.0a0']
- watchdog
The following packages will be downloaded:
package | build
---------------------------|-----------------
jupyter_core-4.10.0 | py39h6e9494a_0 81 KB conda-forge
libpq-14.3 | h2b7167c_0 3.0 MB conda-forge
scikit-learn-1.1.0 | py39hafe9688_0 7.6 MB conda-forge
------------------------------------------------------------
Total: 10.6 MB
The following NEW packages will be INSTALLED:
appnope conda-forge/noarch::appnope-0.1.3-pyhd8ed1ab_0
argh conda-forge/noarch::argh-0.26.2-pyh9f0ad1d_1002
argon2-cffi conda-forge/noarch::argon2-cffi-21.3.0-pyhd8ed1ab_0
argon2-cffi-bindi~ conda-forge/osx-64::argon2-cffi-bindings-21.2.0-py39h63b48b0_2
asttokens conda-forge/noarch::asttokens-2.0.5-pyhd8ed1ab_0
astunparse conda-forge/noarch::astunparse-1.6.3-pyhd8ed1ab_0
attrs conda-forge/noarch::attrs-21.4.0-pyhd8ed1ab_0
backcall conda-forge/noarch::backcall-0.2.0-pyh9f0ad1d_0
backports pkgs/main/noarch::backports-1.1-pyhd3eb1b0_0
backports.functoo~ conda-forge/noarch::backports.functools_lru_cache-1.6.4-pyhd8ed1ab_0
beautifulsoup4 conda-forge/noarch::beautifulsoup4-4.11.1-pyha770c72_0
biopython conda-forge/osx-64::biopython-1.79-py39h89e85a6_1
blas conda-forge/osx-64::blas-2.114-openblas
blas-devel conda-forge/osx-64::blas-devel-3.9.0-14_osx64_openblas
bleach conda-forge/noarch::bleach-5.0.0-pyhd8ed1ab_0
blosc conda-forge/osx-64::blosc-1.21.1-h97e831e_2
boost conda-forge/osx-64::boost-1.74.0-py39ha1f3e3e_5
boost-cpp conda-forge/osx-64::boost-cpp-1.74.0-hdbf7018_7
brotli conda-forge/osx-64::brotli-1.0.9-h5eb16cf_7
brotli-bin conda-forge/osx-64::brotli-bin-1.0.9-h5eb16cf_7
brotlipy conda-forge/osx-64::brotlipy-0.7.0-py39h63b48b0_1004
bzip2 conda-forge/osx-64::bzip2-1.0.8-h0d85af4_4
c-ares conda-forge/osx-64::c-ares-1.18.1-h0d85af4_0
ca-certificates pkgs/main/osx-64::ca-certificates-2022.4.26-hecd8cb5_0
cairo conda-forge/osx-64::cairo-1.16.0-h9e0e54b_1010
certifi conda-forge/osx-64::certifi-2021.10.8-py39h6e9494a_2
cffi conda-forge/osx-64::cffi-1.15.0-py39he338e87_0
cftime conda-forge/osx-64::cftime-1.6.0-py39h86b5767_1
charset-normalizer conda-forge/noarch::charset-normalizer-2.0.12-pyhd8ed1ab_0
colorama conda-forge/noarch::colorama-0.4.4-pyh9f0ad1d_0
configargparse conda-forge/noarch::configargparse-1.5.3-pyhd8ed1ab_0
cryptography pkgs/main/osx-64::cryptography-37.0.1-py39hf6deb26_0
curl conda-forge/osx-64::curl-7.83.1-h23f1065_0
cycler conda-forge/noarch::cycler-0.11.0-pyhd8ed1ab_0
dbus pkgs/main/osx-64::dbus-1.13.18-h18a8e69_0
debugpy conda-forge/osx-64::debugpy-1.6.0-py39hfd1d529_0
decorator conda-forge/noarch::decorator-5.1.1-pyhd8ed1ab_0
defusedxml conda-forge/noarch::defusedxml-0.7.1-pyhd8ed1ab_0
entrypoints conda-forge/noarch::entrypoints-0.4-pyhd8ed1ab_0
executing conda-forge/noarch::executing-0.8.3-pyhd8ed1ab_0
expat conda-forge/osx-64::expat-2.4.8-h96cf925_0
fftw conda-forge/osx-64::fftw-3.3.10-nompi_hf082fe4_102
fkcombu michellab/label/dev/osx-64::fkcombu-2020.04.14-h254f028_0
flit-core conda-forge/noarch::flit-core-3.7.1-pyhd8ed1ab_0
font-ttf-dejavu-s~ conda-forge/noarch::font-ttf-dejavu-sans-mono-2.37-hab24e00_0
font-ttf-inconsol~ conda-forge/noarch::font-ttf-inconsolata-3.000-h77eed37_0
font-ttf-source-c~ conda-forge/noarch::font-ttf-source-code-pro-2.038-h77eed37_0
font-ttf-ubuntu conda-forge/noarch::font-ttf-ubuntu-0.83-hab24e00_0
fontconfig conda-forge/osx-64::fontconfig-2.14.0-h676cef8_0
fonts-conda-ecosy~ conda-forge/noarch::fonts-conda-ecosystem-1-0
fonts-conda-forge conda-forge/noarch::fonts-conda-forge-1-0
fonttools conda-forge/osx-64::fonttools-4.33.3-py39h701faf5_0
freetype pkgs/main/osx-64::freetype-2.11.0-hd8bbffd_0
gettext conda-forge/osx-64::gettext-0.19.8.1-hd1a6beb_1008
giflib conda-forge/osx-64::giflib-5.2.1-hbcb3906_2
glib conda-forge/osx-64::glib-2.70.2-hcf210ce_4
glib-tools conda-forge/osx-64::glib-tools-2.70.2-hcf210ce_4
greenlet conda-forge/osx-64::greenlet-1.1.2-py39hfd1d529_2
griddataformats conda-forge/noarch::griddataformats-0.7.0-pyhd8ed1ab_0
gsd conda-forge/osx-64::gsd-2.5.2-py39h86b5767_0
gsl pkgs/main/osx-64::gsl-2.7.1-h0cefce6_0
hdf4 conda-forge/osx-64::hdf4-4.2.15-hefd3b78_3
hdf5 conda-forge/osx-64::hdf5-1.12.1-nompi_h0aa1fa2_104
icu conda-forge/osx-64::icu-69.1-he49afe7_0
idna conda-forge/noarch::idna-3.3-pyhd8ed1ab_0
importlib-metadata conda-forge/osx-64::importlib-metadata-4.11.3-py39h6e9494a_1
importlib_resourc~ conda-forge/noarch::importlib_resources-5.7.1-pyhd8ed1ab_0
iniconfig conda-forge/noarch::iniconfig-1.1.1-pyh9f0ad1d_0
ipykernel conda-forge/osx-64::ipykernel-6.13.0-py39h71a6800_0
ipython conda-forge/osx-64::ipython-8.3.0-py39h6e9494a_0
ipython_genutils conda-forge/noarch::ipython_genutils-0.2.0-py_1
ipywidgets conda-forge/noarch::ipywidgets-7.7.0-pyhd8ed1ab_0
jbig conda-forge/osx-64::jbig-2.1-h0d85af4_2003
jedi conda-forge/osx-64::jedi-0.18.1-py39h6e9494a_1
jinja2 conda-forge/noarch::jinja2-3.1.2-pyhd8ed1ab_0
joblib conda-forge/noarch::joblib-1.1.0-pyhd8ed1ab_0
jpeg conda-forge/osx-64::jpeg-9e-h5eb16cf_1
jsonschema conda-forge/noarch::jsonschema-4.5.1-pyhd8ed1ab_0
jupyter conda-forge/osx-64::jupyter-1.0.0-py39h6e9494a_7
jupyter_client conda-forge/noarch::jupyter_client-7.3.1-pyhd8ed1ab_0
jupyter_console conda-forge/noarch::jupyter_console-6.4.3-pyhd8ed1ab_0
jupyter_core conda-forge/osx-64::jupyter_core-4.10.0-py39h6e9494a_0
jupyterlab_pygmen~ conda-forge/noarch::jupyterlab_pygments-0.2.2-pyhd8ed1ab_0
jupyterlab_widgets conda-forge/noarch::jupyterlab_widgets-1.1.0-pyhd8ed1ab_0
khronos-opencl-ic~ conda-forge/osx-64::khronos-opencl-icd-loader-2022.01.04-h0d85af4_0
kiwisolver conda-forge/osx-64::kiwisolver-1.4.2-py39h7248d28_1
krb5 conda-forge/osx-64::krb5-1.19.3-hb98e516_0
lcms2 conda-forge/osx-64::lcms2-2.12-h577c468_0
lerc conda-forge/osx-64::lerc-3.0-he49afe7_0
libblas conda-forge/osx-64::libblas-3.9.0-14_osx64_openblas
libbrotlicommon conda-forge/osx-64::libbrotlicommon-1.0.9-h5eb16cf_7
libbrotlidec conda-forge/osx-64::libbrotlidec-1.0.9-h5eb16cf_7
libbrotlienc conda-forge/osx-64::libbrotlienc-1.0.9-h5eb16cf_7
libcblas conda-forge/osx-64::libcblas-3.9.0-14_osx64_openblas
libclang conda-forge/osx-64::libclang-13.0.1-default_he082bbe_0
libcpuid michellab/label/dev/osx-64::libcpuid-0.5.1-h01edc0c_0
libcurl conda-forge/osx-64::libcurl-7.83.1-h23f1065_0
libcxx conda-forge/osx-64::libcxx-14.0.3-hc203e6f_0
libdeflate conda-forge/osx-64::libdeflate-1.10-h0d85af4_0
libedit pkgs/main/osx-64::libedit-3.1.20210910-hca72f7f_0
libev conda-forge/osx-64::libev-4.33-haf1e3a3_1
libffi conda-forge/osx-64::libffi-3.4.2-h0d85af4_5
libgfortran conda-forge/osx-64::libgfortran-5.0.0-9_3_0_h6c81a4c_23
libgfortran5 conda-forge/osx-64::libgfortran5-9.3.0-h6c81a4c_23
libglib conda-forge/osx-64::libglib-2.70.2-hf1fb8c0_4
libiconv conda-forge/osx-64::libiconv-1.16-haf1e3a3_0
liblapack conda-forge/osx-64::liblapack-3.9.0-14_osx64_openblas
liblapacke conda-forge/osx-64::liblapacke-3.9.0-14_osx64_openblas
libllvm13 conda-forge/osx-64::libllvm13-13.0.1-h64f94b2_2
libnetcdf conda-forge/osx-64::libnetcdf-4.8.1-nompi_h6609ca0_102
libnghttp2 conda-forge/osx-64::libnghttp2-1.47.0-hca56917_0
libopenblas conda-forge/osx-64::libopenblas-0.3.20-openmp_hb3cd9ec_0
libpng conda-forge/osx-64::libpng-1.6.37-h7cec526_2
libpq conda-forge/osx-64::libpq-14.3-h2b7167c_0
libsodium conda-forge/osx-64::libsodium-1.0.18-hbcb3906_1
libssh2 conda-forge/osx-64::libssh2-1.10.0-hd3787cc_2
libtiff conda-forge/osx-64::libtiff-4.3.0-h17f2ce3_3
libwebp conda-forge/osx-64::libwebp-1.2.2-h28dabe5_0
libwebp-base conda-forge/osx-64::libwebp-base-1.2.2-h0d85af4_1
libxcb conda-forge/osx-64::libxcb-1.13-h0d85af4_1004
libzip conda-forge/osx-64::libzip-1.8.0-h7e5727d_1
libzlib conda-forge/osx-64::libzlib-1.2.11-h6c3fc93_1014
llvm-openmp conda-forge/osx-64::llvm-openmp-14.0.3-ha654fa7_0
lomap2 conda-forge/noarch::lomap2-2.0.0-pyhd8ed1ab_0
lz4-c conda-forge/osx-64::lz4-c-1.9.3-he49afe7_1
markupsafe conda-forge/osx-64::markupsafe-2.1.1-py39h63b48b0_1
matplotlib-base conda-forge/osx-64::matplotlib-base-3.5.2-py39h64a0072_0
matplotlib-inline conda-forge/noarch::matplotlib-inline-0.1.3-pyhd8ed1ab_0
mdanalysis conda-forge/osx-64::mdanalysis-2.1.0-py39hd408605_2
mdtraj conda-forge/osx-64::mdtraj-1.9.7-py39h996af62_1
mistune conda-forge/osx-64::mistune-0.8.4-py39h89e85a6_1005
mmtf-python conda-forge/noarch::mmtf-python-1.1.2-py_0
mrcfile conda-forge/noarch::mrcfile-1.3.0-pyh44b312d_0
msgpack-python conda-forge/osx-64::msgpack-python-1.0.3-py39h7248d28_1
munkres conda-forge/noarch::munkres-1.1.4-pyh9f0ad1d_0
mysql-common conda-forge/osx-64::mysql-common-8.0.29-h9525fdc_0
mysql-libs conda-forge/osx-64::mysql-libs-8.0.29-h8d0e597_0
nbclient conda-forge/noarch::nbclient-0.6.3-pyhd8ed1ab_0
nbconvert conda-forge/noarch::nbconvert-6.5.0-pyhd8ed1ab_0
nbconvert-core conda-forge/noarch::nbconvert-core-6.5.0-pyhd8ed1ab_0
nbconvert-pandoc conda-forge/noarch::nbconvert-pandoc-6.5.0-pyhd8ed1ab_0
nbformat conda-forge/noarch::nbformat-5.4.0-pyhd8ed1ab_0
ncurses conda-forge/osx-64::ncurses-6.3-h96cf925_1
nest-asyncio conda-forge/noarch::nest-asyncio-1.5.5-pyhd8ed1ab_0
netcdf4 conda-forge/osx-64::netcdf4-1.5.8-nompi_py39he7d1c46_101
networkx conda-forge/noarch::networkx-2.8-pyhd8ed1ab_0
nglview conda-forge/noarch::nglview-3.0.3-pyh8a188c0_0
notebook conda-forge/noarch::notebook-6.4.11-pyha770c72_0
nspr pkgs/main/osx-64::nspr-4.33-he9d5cce_0
nss conda-forge/osx-64::nss-3.77-hfce436b_0
numexpr pkgs/main/osx-64::numexpr-2.8.1-py39h9c3cb84_0
numpy conda-forge/osx-64::numpy-1.22.3-py39h214027c_2
ocl_icd_wrapper_a~ conda-forge/osx-64::ocl_icd_wrapper_apple-1.0.0-hbcb3906_0
openblas conda-forge/osx-64::openblas-0.3.20-openmp_h5ad848b_0
openff-forcefields conda-forge/noarch::openff-forcefields-2.0.0-pyh6c4a22f_0
openff-toolkit-ba~ conda-forge/noarch::openff-toolkit-base-0.10.6-pyhd8ed1ab_0
openjpeg conda-forge/osx-64::openjpeg-2.4.0-h6e7aa92_1
openmm conda-forge/osx-64::openmm-7.7.0-py39h8d72adf_0_khronos
openssl conda-forge/osx-64::openssl-3.0.3-hfe4f2af_0
packaging conda-forge/noarch::packaging-21.3-pyhd8ed1ab_0
pandas conda-forge/osx-64::pandas-1.4.2-py39hbd61c47_1
pandoc conda-forge/osx-64::pandoc-2.18-h694c41f_0
pandocfilters conda-forge/noarch::pandocfilters-1.5.0-pyhd8ed1ab_0
parmed conda-forge/osx-64::parmed-3.4.3-py39hfd1d529_2
parso conda-forge/noarch::parso-0.8.3-pyhd8ed1ab_0
patsy conda-forge/noarch::patsy-0.5.2-pyhd8ed1ab_0
pcre conda-forge/osx-64::pcre-8.45-he49afe7_0
pexpect conda-forge/noarch::pexpect-4.8.0-pyh9f0ad1d_2
pickleshare conda-forge/noarch::pickleshare-0.7.5-py_1003
pillow conda-forge/osx-64::pillow-9.1.0-py39hd2c7aa1_2
pip conda-forge/noarch::pip-22.1-pyhd8ed1ab_0
pixman conda-forge/osx-64::pixman-0.40.0-hbcb3906_0
pluggy conda-forge/osx-64::pluggy-1.0.0-py39h6e9494a_3
prometheus_client conda-forge/noarch::prometheus_client-0.14.1-pyhd8ed1ab_0
prompt-toolkit conda-forge/noarch::prompt-toolkit-3.0.29-pyha770c72_0
prompt_toolkit conda-forge/noarch::prompt_toolkit-3.0.29-hd8ed1ab_0
psutil conda-forge/osx-64::psutil-5.9.0-py39h63b48b0_1
pthread-stubs conda-forge/osx-64::pthread-stubs-0.4-hc929b4f_1001
ptyprocess conda-forge/noarch::ptyprocess-0.7.0-pyhd3deb0d_0
pure_eval conda-forge/noarch::pure_eval-0.2.2-pyhd8ed1ab_0
py conda-forge/noarch::py-1.11.0-pyh6c4a22f_0
py3dmol conda-forge/noarch::py3dmol-1.8.0-pyhd8ed1ab_0
pycairo conda-forge/osx-64::pycairo-1.21.0-py39ha25c624_1
pycparser conda-forge/noarch::pycparser-2.21-pyhd8ed1ab_0
pydot conda-forge/noarch::pydot-1.2.4-py_0
pygments conda-forge/noarch::pygments-2.12.0-pyhd8ed1ab_0
pygtail conda-forge/noarch::pygtail-0.12.0-pyhd8ed1ab_0
pymbar conda-forge/osx-64::pymbar-3.0.6-py39h86b5767_0
pyopenssl conda-forge/noarch::pyopenssl-22.0.0-pyhd8ed1ab_0
pyparsing conda-forge/noarch::pyparsing-3.0.9-pyhd8ed1ab_0
pypdb conda-forge/noarch::pypdb-2.1-pyhd8ed1ab_0
pyqt conda-forge/osx-64::pyqt-5.12.3-py39h9d385e7_4
pyrsistent conda-forge/osx-64::pyrsistent-0.18.1-py39h63b48b0_1
pysocks conda-forge/osx-64::pysocks-1.7.1-py39h6e9494a_5
pytables conda-forge/osx-64::pytables-3.7.0-py39hfd850c7_0
pytest conda-forge/osx-64::pytest-7.1.2-py39h6e9494a_0
python conda-forge/osx-64::python-3.9.12-h1cc4136_1_cpython
python-constraint conda-forge/noarch::python-constraint-1.4.0-py_0
python-dateutil conda-forge/noarch::python-dateutil-2.8.2-pyhd8ed1ab_0
python-fastjsonsc~ conda-forge/noarch::python-fastjsonschema-2.15.3-pyhd8ed1ab_0
python_abi conda-forge/osx-64::python_abi-3.9-2_cp39
pytz conda-forge/noarch::pytz-2022.1-pyhd8ed1ab_0
pyyaml conda-forge/osx-64::pyyaml-6.0-py39h63b48b0_4
pyzmq conda-forge/osx-64::pyzmq-22.3.0-py39hc2dc7ec_2
qt conda-forge/osx-64::qt-5.12.9-h2a607e2_5
qtconsole conda-forge/noarch::qtconsole-5.3.0-pyhd8ed1ab_0
qtconsole-base conda-forge/noarch::qtconsole-base-5.3.0-pyhd8ed1ab_0
qtpy conda-forge/noarch::qtpy-2.1.0-pyhd8ed1ab_0
rdkit conda-forge/osx-64::rdkit-2022.03.2-py39h8747f4c_0
readline pkgs/main/osx-64::readline-8.1.2-hca72f7f_1
reportlab conda-forge/osx-64::reportlab-3.5.68-py39hf37cc50_1
requests conda-forge/noarch::requests-2.27.1-pyhd8ed1ab_0
scikit-learn conda-forge/osx-64::scikit-learn-1.1.0-py39hafe9688_0
scipy conda-forge/osx-64::scipy-1.8.0-py39h056f1c0_1
seaborn conda-forge/noarch::seaborn-0.11.2-hd8ed1ab_0
seaborn-base conda-forge/noarch::seaborn-base-0.11.2-pyhd8ed1ab_0
send2trash conda-forge/noarch::send2trash-1.8.0-pyhd8ed1ab_0
setuptools conda-forge/osx-64::setuptools-62.2.0-py39h6e9494a_0
sire michellab/label/dev/osx-64::sire-2022.2.0-py39h01edc0c_50
six conda-forge/noarch::six-1.16.0-pyh6c4a22f_0
smirnoff99frosst conda-forge/noarch::smirnoff99frosst-1.1.0-pyh44b312d_0
snappy conda-forge/osx-64::snappy-1.1.9-h6e38e02_0
soupsieve conda-forge/noarch::soupsieve-2.3.1-pyhd8ed1ab_0
sqlalchemy conda-forge/osx-64::sqlalchemy-1.4.36-py39h701faf5_0
sqlite conda-forge/osx-64::sqlite-3.38.5-hd9f0692_0
stack_data conda-forge/noarch::stack_data-0.2.0-pyhd8ed1ab_0
statsmodels conda-forge/osx-64::statsmodels-0.13.2-py39hc89836e_0
tbb conda-forge/osx-64::tbb-2021.5.0-hbb4e6a2_1
terminado conda-forge/osx-64::terminado-0.13.3-py39h6e9494a_1
threadpoolctl conda-forge/noarch::threadpoolctl-3.1.0-pyh8a188c0_0
tinycss2 conda-forge/noarch::tinycss2-1.1.1-pyhd8ed1ab_0
tk conda-forge/osx-64::tk-8.6.12-h5dbffcc_0
tomli conda-forge/noarch::tomli-2.0.1-pyhd8ed1ab_0
tornado conda-forge/osx-64::tornado-6.1-py39h63b48b0_3
tqdm conda-forge/noarch::tqdm-4.64.0-pyhd8ed1ab_0
traitlets conda-forge/noarch::traitlets-5.2.0-pyhd8ed1ab_0
typing-extensions conda-forge/noarch::typing-extensions-4.2.0-hd8ed1ab_1
typing_extensions conda-forge/noarch::typing_extensions-4.2.0-pyha770c72_1
tzdata conda-forge/noarch::tzdata-2022a-h191b570_0
unicodedata2 conda-forge/osx-64::unicodedata2-14.0.0-py39h63b48b0_1
urllib3 conda-forge/noarch::urllib3-1.26.9-pyhd8ed1ab_0
watchdog conda-forge/osx-64::watchdog-2.1.7-py39h147bbb7_1
wcwidth conda-forge/noarch::wcwidth-0.2.5-pyh9f0ad1d_2
webencodings conda-forge/noarch::webencodings-0.5.1-py_1
wheel conda-forge/noarch::wheel-0.37.1-pyhd8ed1ab_0
widgetsnbextension conda-forge/osx-64::widgetsnbextension-3.6.0-py39h6e9494a_0
xmltodict conda-forge/noarch::xmltodict-0.13.0-pyhd8ed1ab_0
xorg-libxau conda-forge/osx-64::xorg-libxau-1.0.9-h35c211d_0
xorg-libxdmcp conda-forge/osx-64::xorg-libxdmcp-1.1.3-h35c211d_0
xz conda-forge/osx-64::xz-5.2.5-haf1e3a3_1
yaml conda-forge/osx-64::yaml-0.2.5-h0d85af4_2
zeromq conda-forge/osx-64::zeromq-4.3.4-he49afe7_1
zipp conda-forge/noarch::zipp-3.8.0-pyhd8ed1ab_0
zlib conda-forge/osx-64::zlib-1.2.11-h6c3fc93_1014
zstd conda-forge/osx-64::zstd-1.5.2-h582d3a0_0
Proceed ([y]/n)? y
Downloading and Extracting Packages
jupyter_core-4.10.0 | 81 KB | ##################################### | 100%
scikit-learn-1.1.0 | 7.6 MB | ##################################### | 100%
libpq-14.3 | 3.0 MB | ##################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: - Enabling notebook extension jupyter-js-widgets/extension...
- Validating: OK
Exception ignored in: <function Application.__del__ at 0x10b1413a0>
Traceback (most recent call last):
File "/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/python3.9/site-packages/traitlets/config/application.py", line 955, in __del__
File "/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/python3.9/site-packages/traitlets/config/application.py", line 945, in close_handlers
File "/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/python3.9/site-packages/traitlets/traitlets.py", line 683, in __get__
File "/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/python3.9/site-packages/traitlets/traitlets.py", line 657, in get
File "/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/python3.9/site-packages/traitlets/traitlets.py", line 1428, in _notify_observers
File "/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/python3.9/site-packages/traitlets/config/application.py", line 273, in _observe_logging_default
File "/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/python3.9/site-packages/traitlets/config/application.py", line 278, in _configure_logging
File "/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/python3.9/logging/config.py", line 809, in dictConfig
File "/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/python3.9/logging/config.py", line 546, in configure
ValueError: Unable to configure formatter 'console'
done
#
# To activate this environment, use
#
# $ conda activate biosimspace_test
#
# To deactivate an active environment, use
#
# $ conda deactivate
(base) zwu@EXS-001525 ~ % conda activate biosimspace_test
(biosimspace_test) zwu@EXS-001525 ~ % cd ~/Documents/GitHub/BioSimSpace
(biosimspace_test) zwu@EXS-001525 BioSimSpace % cd python
(biosimspace_test) zwu@EXS-001525 python % python setup.py develop
Traceback (most recent call last):
File "/Users/zwu/Documents/GitHub/BioSimSpace/python/setup.py", line 11, in <module>
import Sire.Base
File "/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/python3.9/site-packages/Sire/__init__.py", line 186, in <module>
from . import Config
File "/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/python3.9/site-packages/Sire/Config/__init__.py", line 7, in <module>
import Sire.Base
File "/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/python3.9/site-packages/Sire/Base/__init__.py", line 15, in <module>
from Sire.Base._Base import *
ImportError: dlopen(/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/python3.9/site-packages/Sire/Base/_Base.so, 0x0002): Library not loaded: @rpath/libgsl.25.dylib
Referenced from: /Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/libSireCAS.2022.2.0.dylib
Reason: tried: '/Users/zwu/build/gromacs/2022/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace_test/bin/.:@executable_path/../lib:@executable_path/../bundled/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace_test/bin/../lib:@executable_path/../bundled/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/python3.9/site-packages/Sire/Base/../../../../libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace_test/bin/../lib:@executable_path/../bundled/lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/python3.9/site-packages/Sire/Base/../../../../libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace_test/bin/../lib/libgsl.25.dylib' (no such file), '/Users/zwu/miniconda3_x86/envs/biosimspace_test/bin/../lib/libgsl.25.dylib' (no such file), '/usr/local/lib/libgsl.25.dylib' (no such file), '/usr/lib/libgsl.25.dylib' (no such file)
I have also try to identify the gsl version that is being installed
(base) zwu@EXS-001525 ~ % conda activate biosimspace_test
(biosimspace_test) zwu@EXS-001525 ~ % find /Users/zwu/miniconda3_x86/envs/biosimspace_test/lib -name 'libgsl*'
/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/libgsl.dylib
/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/libgsl.27.dylib
/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/libgslcblas.a
/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/libgsl.a
/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/libgslcblas.0.dylib
/Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/libgslcblas.dylib
(biosimspace_test) zwu@EXS-001525 ~ % conda list | grep gsl
gsl 2.7.1 h0cefce6_0
As you can see, it's downloading gsl from the defaults channel, not conda-forge. This is the problem:
gsl pkgs/main/osx-64::gsl-2.7.1-h0cefce6_0
It looks like the package is only 10 days old, so I assume the new version will soon make it's way into conda-forge too.
I'm not sure why your setup is prioritising the newer version from defaults, rather than using the channel priority. This doesn't happen for me. You could try removing defaults, e.g. by using something like MiniForge or using a strict channel priority.
That is weird. MacOS uses @rpath for finding the relative path to import shared libraries. That is why libSireCAS links to @rpath/libgsl.25.dylib.
MacOS looks for this name (@rpath/libgsl.25.dylib) to exported from the libraries it searches. It cannot link to /Users/zwu/build/gromacs/2022/lib/libgsl.25.dylib (and gives no such file) because this library comes with Gromacs, and has not been linked to support @rpath linking. It thus doesn't export the name @rpath/libgsl.25.dylib, despite it likely being a usable library.
The right library is /Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/libgsl.27.dylib, but this won't link because it will have the exported name @rpath/libgsl.27.dylib (as will /Users/zwu/miniconda3_x86/envs/biosimspace_test/lib/libgsl.dylib) as this is just a copy or symbolic link.
The error is caused by something upgrading GSL to version 2.7, when Sire was compiled against GSL 2.5. Unfortunately these aren't compatible on MacOS, so things break.
I think that the fix is in the Sire conda package to add a pin that it can't change GSL version? This way, it will keep the GSL 2.5 from conda-forge rather than take GSL 2.7 from defaults.
Yes - if this is possible for conda packages. I know that Lester has struggled a lot to stop conda from upgrading dependencies behind our back ;-)
Lester is right that the fix to enable you to work today is to uninstall the gsl-2.7, reprioritise conda-forge, and then install gel-2.5 from conda-forge.
@chryswoods This is my current approach where I do conda install gsl=2.7.0 to get it to work.
I'd rather not pin in the build as we are doing a pure conda-forge compliant build in the correct way, i.e. this is what they would want if we were to be accepted into their channel. This is just a fundamental issue with conda, i.e. once you start mixing channels things get messy very quickly. (For example, the soname might be different for the same package version.) We also can't make assumptions about a users conda environment.
I think the easiest solution would be to update our docs to suggest using MiniForge or MambaForge as the conda environment for Sire/BioSimSpace. This is now what we use to build Sire and BioSImSpace, so we know that it will work. There will still be issues if the user requires other packages in their environment, which require dependencies from defaults, but that's not really a problem that we can anticipate or resolve in an easy way.
Yes, that sounds right. I'm keen to use minimamba both for speed but also for license reasons. The current license opaqueness around anaconda makes me uncomfortable give potential commercial users of BioSimSpace. Having everything on top of minimamba/conda-forge makes the license terms clear and open source.
Yes, I agree. Also, pinning wouldn't work since it's possible that the exact same package version has a different soname, so we'd still have the same issue if a dependency was pulled from defaults for some reason.
I'll update the docs when I get a chance.