PARC
PARC copied to clipboard
Published
20 hours ago •
ShobiStassen
ShobiStassen
pip install issue: Mac OS X and AWS Linux
Hey Shobi!
I get the following when trying to pip install parc in a fresh Anaconda environment with Python 3.7 on both Mac OS X 10.14.6 (Mojave) and on AWS' flavor of Linux:
Collecting parc
Downloading https://files.pythonhosted.org/packages/2d/82/7dcddc30ba3cd6cec30a1f939d40bb21882bef791d87a1ddab174e825b5e/parc-0.18-py3-none-any.whl
Collecting python-igraph (from parc)
Using cached https://files.pythonhosted.org/packages/0f/a0/4e7134f803737aa6eebb4e5250565ace0e2599659e22be7f7eba520ff017/python-igraph-0.7.1.post6.tar.gz
Collecting leidenalg (from parc)
Collecting pandas (from parc)
Using cached https://files.pythonhosted.org/packages/39/73/99aa822ee88cef5829607217c11bf24ecc1171ae5d49d5f780085f5da518/pandas-0.25.1-cp37-cp37m-macosx_10_9_x86_64.macosx_10_10_x86_64.whl
Collecting hnswlib (from parc)
Using cached https://files.pythonhosted.org/packages/51/ee/850ac2cdc9483a5a26fd4173be486f48db0bdb9e2b200dfc3149a572a907/hnswlib-0.3.2.0.tar.gz
Collecting pybind11 (from parc)
Downloading https://files.pythonhosted.org/packages/4b/4d/ae1c4d8e8b139afa9682054dd42df3b0e3b5c1731287933021b9fd7e9cc4/pybind11-2.4.3-py2.py3-none-any.whl (150kB)
|████████████████████████████████| 153kB 3.6MB/s
Collecting scipy (from parc)
Using cached https://files.pythonhosted.org/packages/d5/06/1a696649f4b2e706c509cb9333fdc6331fbe71251cede945f9e1fa13ea34/scipy-1.3.1-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Collecting numpy (from parc)
Using cached https://files.pythonhosted.org/packages/b4/e8/5ececadd9cc220bb783b4ce6ffaa9266925d37ed41237bc23bc530ab4f3d/numpy-1.17.2-cp37-cp37m-macosx_10_6_intel.whl
Collecting pytz>=2017.2 (from pandas->parc)
Downloading https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl (509kB)
|████████████████████████████████| 512kB 16.1MB/s
Collecting python-dateutil>=2.6.1 (from pandas->parc)
Using cached https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl
Collecting six>=1.5 (from python-dateutil>=2.6.1->pandas->parc)
Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Building wheels for collected packages: python-igraph, hnswlib
Building wheel for python-igraph (setup.py) ... done
Created wheel for python-igraph: filename=python_igraph-0.7.1.post6-cp37-cp37m-macosx_10_9_x86_64.whl size=1903873 sha256=0c6639b7e49b66a679add893935b4cdabec863b711406adf644e1a59b6726a56
Stored in directory: /Users/wyatt.mcdonnell/Library/Caches/pip/wheels/41/d6/02/34eebae97e25f5b87d60f4c0687e00523e3f244fa41bc3f4a7
Building wheel for hnswlib (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: //anaconda3/envs/parc/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/bq/7jsphcrn5jj6rt7jrll1x5y80000gp/T/pip-install-yfhtmyc0/hnswlib/setup.py'"'"'; __file__='"'"'/private/var/folders/bq/7jsphcrn5jj6rt7jrll1x5y80000gp/T/pip-install-yfhtmyc0/hnswlib/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/bq/7jsphcrn5jj6rt7jrll1x5y80000gp/T/pip-wheel-wlmtzn3j --python-tag cp37
cwd: /private/var/folders/bq/7jsphcrn5jj6rt7jrll1x5y80000gp/T/pip-install-yfhtmyc0/hnswlib/
Complete output (41 lines):
running bdist_wheel
running build
running build_ext
creating var
creating var/folders
creating var/folders/bq
creating var/folders/bq/7jsphcrn5jj6rt7jrll1x5y80000gp
creating var/folders/bq/7jsphcrn5jj6rt7jrll1x5y80000gp/T
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I//anaconda3/envs/parc/include -arch x86_64 -I//anaconda3/envs/parc/include -arch x86_64 -I//anaconda3/envs/parc/include/python3.7m -c /var/folders/bq/7jsphcrn5jj6rt7jrll1x5y80000gp/T/tmpq3tagf4m.cpp -o var/folders/bq/7jsphcrn5jj6rt7jrll1x5y80000gp/T/tmpq3tagf4m.o -std=c++14
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I//anaconda3/envs/parc/include -arch x86_64 -I//anaconda3/envs/parc/include -arch x86_64 -I//anaconda3/envs/parc/include/python3.7m -c /var/folders/bq/7jsphcrn5jj6rt7jrll1x5y80000gp/T/tmpw16kygtu.cpp -o var/folders/bq/7jsphcrn5jj6rt7jrll1x5y80000gp/T/tmpw16kygtu.o -fvisibility=hidden
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/bq/7jsphcrn5jj6rt7jrll1x5y80000gp/T/pip-install-yfhtmyc0/hnswlib/setup.py", line 116, in <module>
zip_safe=False,
File "//anaconda3/envs/parc/lib/python3.7/site-packages/setuptools/__init__.py", line 145, in setup
return distutils.core.setup(**attrs)
File "//anaconda3/envs/parc/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "//anaconda3/envs/parc/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "//anaconda3/envs/parc/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "//anaconda3/envs/parc/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 192, in run
self.run_command('build')
File "//anaconda3/envs/parc/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "//anaconda3/envs/parc/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "//anaconda3/envs/parc/lib/python3.7/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "//anaconda3/envs/parc/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "//anaconda3/envs/parc/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "//anaconda3/envs/parc/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 84, in run
_build_ext.run(self)
File "//anaconda3/envs/parc/lib/python3.7/distutils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/private/var/folders/bq/7jsphcrn5jj6rt7jrll1x5y80000gp/T/pip-install-yfhtmyc0/hnswlib/setup.py", line 88, in build_extensions
import pybind11
ModuleNotFoundError: No module named 'pybind11'
----------------------------------------
ERROR: Failed building wheel for hnswlib
Running setup.py clean for hnswlib
Successfully built python-igraph
Failed to build hnswlib
Installing collected packages: python-igraph, leidenalg, numpy, pytz, six, python-dateutil, pandas, pybind11, hnswlib, scipy, parc
Running setup.py install for hnswlib ... done
Successfully installed hnswlib-0.3.2.0 leidenalg-0.7.0 numpy-1.17.2 pandas-0.25.1 parc-0.18 pybind11-2.4.3 python-dateutil-2.8.0 python-igraph-0.7.1.post6 pytz-2019.3 scipy-1.3.1 six-1.12.0
For what it's worth, if I manually install everything first with the following bash script and then clone PARC.git and python3 setup.py install everything works!
!/usr/bin/env bash
conda create --name parc_20191014 python=3 --yes |& tee -a parc.log
conda activate parc_20191014 |& tee -a parc.log
conda install -c anaconda numpy pandas scipy setuptools scikit-learn seaborn --yes |& tee -a parc.log
conda install -c conda-forge pybind11 python-igraph umap-learn --yes |& tee -a parc.log
pip install leidenalg |& tee -a parc.log
pip install ipykernel |& tee -a parc.log
pip3 install jupyter |& tee -a parc.log
python -m ipykernel install --user --name parc --display-name "Python (parc)" |& tee -a parc.log
git clone https://github.com/nmslib/hnswlib |& tee -a parc.log
cd hnswlib |& tee -a parc.log
cd python_bindings |& tee -a parc.log
python3 setup.py install |& tee -a parc.log
cd ../../ |& tee -a parc.log
git clone https://github.com/ShobiStassen/PARC.git |& tee -a parc.log
cd PARC |& tee -a parc.log
# 2019-10-14, most recently tested with: git checkout 46a36996539bf57bb0d544898d900a63fdaa5b90 |& tee -a parc.log
python3 setup.py install
Cheers, Wyatt
hi Wyatt! I think this has to do with setup.py not installing pybind11 before it tries to install hnswlib. Does the pip install work if you first install pybind11 followed by pip install parc pip install pybind11 then pip install parc
Hi Shobi!
If I pip install pybind11 and then pip install parc, I get effectively the same error, but with numpy, as follows:
File "/mnt/home/wyatt.mcdonnell/anaconda3/envs/parc_pip/lib/python3.7/distut
ils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/mnt/home/wyatt.mcdonnell/anaconda3/envs/parc_pip/lib/python3.7/distut
ils/dist.py", line 985, in run_command
cmd_obj.run()
File "/mnt/home/wyatt.mcdonnell/anaconda3/envs/parc_pip/lib/python3.7/site-p
ackages/setuptools/command/build_ext.py", line 84, in run
_build_ext.run(self)
File "/mnt/home/wyatt.mcdonnell/anaconda3/envs/parc_pip/lib/python3.7/distut
ils/command/build_ext.py", line 340, in run
self.build_extensions()
File "/tmp/pip-install-_xfidwoz/hnswlib/setup.py", line 89, in build_extensi
ons
import numpy as np
ModuleNotFoundError: No module named 'numpy'
----------------------------------------
ERROR: Failed building wheel for hnswlib
Running setup.py clean for hnswlib
Congrats to Shobi and team for the official publication in Bioinformatics yesterday!
@wyattmcdonnell I can't comment on how this would work in an AWS Linux instance, but in OS X Catalina, the steps below worked to install PARC inside Anaconda:
- Download and install Anaconda with graphical GUI (Anaconda Navigator) and Python 3.7
- In Anaconda Navigator, go to "Environments" tab
- Create an environment called "ParcEnv" with Python 3.7
- In Anaconda navigator, go to "Home" tab
- Change channel to "ParcEnv"
- Launch Qt console
- Install as many packages as possible with conda:
conda install pybind11 numpy scipy pandas matplotlib sklearn - Install remaining packages with pip:
pip install hnswlib python-igraph leidenalg - You should get this output:
Successfully built python-igraph leidenalg hnswlib
Installing collected packages: python-igraph, leidenalg, hnswlib, parc
Successfully installed hnswlib-0.3.4 leidenalg-0.7.0 parc-0.19 python-igraph-0.7.1.post6
Note: you may need to restart the kernel to use updated packages.
- install sklearn to run the demo dataset:
pip install sklearn - You should get this output:
Successfully installed joblib-0.14.1 scikit-learn-0.22.1 sklearn-0.0
- From the PARC GitHub home page, run the code in "Example Usage 1. (small test sets) - IRIS and Digits dataset from sklearn". Just be sure to convert all of the "//" into "#" if running as a script (in the Spyder script editor, for example).
- Output looks like this:
input data has shape 150 (samples) x 4 (features)
Starting local pruning based on Minkowski metric at 2 s.dev above mean
Starting global pruning
Starting community detection
0.01191401481628418
list of cluster labels and populations 6 [(0, 36), (1, 34), (2, 29), (3, 21), (4, 19), (5, 11)]
time elapsed 0.1 seconds
target is 0
cluster 2 has majority 0 with population 29
cluster 3 has majority 0 with population 21
target 0 has f1-score of 100.00
...
target is 9
cluster 1 has majority 9 with population 147
target 9 has f1-score of 88.69
f1-score (unweighted) mean 95.28 %
f1-score weighted (by population) 95.30 %
...and some plots, too.
I realize your comment is a few months old, but I hope that helps someone get started on OS X!
- ES
