tools
tools copied to clipboard
nf-core
`nf-core` failing from conda on Mac Pro M1
Description of the bug
When trying to make a local module for my non-nf-core workflow, the module creation subcommand throws an error and so just calling nf-core itself.
The command works fine in Gitpod, but not on my MacPro M1, so I'm guessing it's something to do with the environment.
I made a fresh install of the running environment with:
mamba env create -f nextflow_conda-env.yml -n nextflow-env
where nextflow_conda-env.yml is:
channels:
- conda-forge
- bioconda
- defaults
dependencies:
- nextflow=22.04.0
- nf-core=2.4.1
- mamba=0.23.1
Package summary:
Package Version Build Channel Size
───────────────────────────────────────────────────────────────────────────────────────────────────────
Install:
───────────────────────────────────────────────────────────────────────────────────────────────────────
+ appdirs 1.4.4 pyh9f0ad1d_0 conda-forge/noarch Cached
+ attrs 21.4.0 pyhd8ed1ab_0 conda-forge/noarch Cached
+ backports 1.0 py_2 conda-forge/noarch Cached
+ backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge/noarch Cached
+ bleach 5.0.0 pyhd8ed1ab_0 conda-forge/noarch Cached
+ boltons 21.0.0 pyhd8ed1ab_0 conda-forge/noarch Cached
+ brotlipy 0.7.0 py310hf8d0d8f_1004 conda-forge/osx-arm64 Cached
+ bzip2 1.0.8 h3422bc3_4 conda-forge/osx-arm64 Cached
+ c-ares 1.18.1 h3422bc3_0 conda-forge/osx-arm64 Cached
+ ca-certificates 2022.5.18.1 h4653dfc_0 conda-forge/osx-arm64 Cached
+ cattrs 1.10.0 pyhd8ed1ab_0 conda-forge/noarch Cached
+ certifi 2022.5.18.1 py310hbe9552e_0 conda-forge/osx-arm64 Cached
+ cffi 1.15.0 py310hf10583b_0 conda-forge/osx-arm64 Cached
+ charset-normalizer 2.0.12 pyhd8ed1ab_0 conda-forge/noarch Cached
+ click 8.1.3 py310hbe9552e_0 conda-forge/osx-arm64 Cached
+ colorama 0.4.4 pyh9f0ad1d_0 conda-forge/noarch Cached
+ commonmark 0.9.1 py_0 conda-forge/noarch Cached
+ conda 4.12.0 py310hbe9552e_0 conda-forge/osx-arm64 Cached
+ conda-package-handling 1.8.1 py310hf8d0d8f_1 conda-forge/osx-arm64 Cached
+ coreutils 9.1 h1c322ee_0 conda-forge/osx-arm64 Cached
+ cryptography 37.0.2 py310h94bb23d_0 conda-forge/osx-arm64 1 MB
+ curl 7.83.1 h2fcd78c_0 conda-forge/osx-arm64 145 KB
+ dataclasses 0.8 pyhc8e2a94_3 conda-forge/noarch Cached
+ docutils 0.18.1 py310hbe9552e_1 conda-forge/osx-arm64 Cached
+ expat 2.4.8 h6b3803e_0 conda-forge/osx-arm64 Cached
+ future 0.18.2 py310hbe9552e_5 conda-forge/osx-arm64 Cached
+ galaxy-containers 20.5.0 py_0 bioconda/noarch Cached
+ galaxy-tool-util 20.9.1 py_0 bioconda/noarch Cached
+ galaxy-util 20.9.1 py_0 bioconda/noarch Cached
+ gettext 0.19.8.1 h049c9fb_1008 conda-forge/osx-arm64 Cached
+ git 2.35.3 pl5321h8b11ce0_0 conda-forge/osx-arm64 13 MB
+ gitdb 4.0.9 pyhd8ed1ab_0 conda-forge/noarch Cached
+ gitpython 3.1.27 pyhd8ed1ab_0 conda-forge/noarch Cached
+ icu 70.1 h6b3803e_0 conda-forge/osx-arm64 13 MB
+ idna 3.3 pyhd8ed1ab_0 conda-forge/noarch Cached
+ importlib-metadata 4.11.3 py310hbe9552e_1 conda-forge/osx-arm64 Cached
+ importlib_resources 5.7.1 pyhd8ed1ab_1 conda-forge/noarch Cached
+ iniconfig 1.1.1 pyh9f0ad1d_0 conda-forge/noarch Cached
+ itsdangerous 2.1.2 pyhd8ed1ab_0 conda-forge/noarch Cached
+ jinja2 3.1.2 pyhd8ed1ab_0 conda-forge/noarch Cached
+ jsonschema 4.5.1 pyhd8ed1ab_0 conda-forge/noarch Cached
+ krb5 1.19.3 hf9b2bbe_0 conda-forge/osx-arm64 Cached
+ libarchive 3.5.2 h69ec738_2 conda-forge/osx-arm64 2 MB
+ libcurl 7.83.1 h2fcd78c_0 conda-forge/osx-arm64 300 KB
+ libcxx 14.0.3 h6a5c8ee_0 conda-forge/osx-arm64 Cached
+ libedit 3.1.20191231 hc8eb9b7_2 conda-forge/osx-arm64 Cached
+ libev 4.33 h642e427_1 conda-forge/osx-arm64 Cached
+ libffi 3.4.2 h3422bc3_5 conda-forge/osx-arm64 Cached
+ libiconv 1.16 h642e427_0 conda-forge/osx-arm64 Cached
+ libmamba 0.23.1 h1c735bf_1 conda-forge/osx-arm64 1 MB
+ libmambapy 0.23.1 py310h19f903a_1 conda-forge/osx-arm64 206 KB
+ libnghttp2 1.47.0 he723fca_0 conda-forge/osx-arm64 Cached
+ libsolv 0.7.22 h1280f1d_0 conda-forge/osx-arm64 386 KB
+ libssh2 1.10.0 hb80f160_2 conda-forge/osx-arm64 Cached
+ libxml2 2.9.14 h035c1df_0 conda-forge/osx-arm64 655 KB
+ libzlib 1.2.11 h90dfc92_1014 conda-forge/osx-arm64 Cached
+ lz4-c 1.9.3 hbdafb3b_1 conda-forge/osx-arm64 Cached
+ lzo 2.10 h642e427_1000 conda-forge/osx-arm64 Cached
+ mamba 0.23.1 py310h25b57eb_1 conda-forge/osx-arm64 46 KB
+ markdown 3.3.7 pyhd8ed1ab_0 conda-forge/noarch Cached
+ markupsafe 2.1.1 py310hf8d0d8f_1 conda-forge/osx-arm64 Cached
+ ncurses 6.3 h07bb92c_1 conda-forge/osx-arm64 Cached
+ nextflow 22.04.0 h4a94de4_0 bioconda/noarch Cached
+ nf-core 2.4.1 pyh5e36f6f_0 bioconda/noarch Cached
+ openjdk 11.0.9.1 he8acd94_3 conda-forge/osx-arm64 Cached
+ openssl 1.1.1o ha287fd2_0 conda-forge/osx-arm64 2 MB
+ packaging 21.3 pyhd8ed1ab_0 conda-forge/noarch Cached
+ pcre2 10.37 hc5de063_0 conda-forge/osx-arm64 Cached
+ perl 5.32.1 2_h9b22ae9_perl5 conda-forge/osx-arm64 Cached
+ pip 22.0.4 pyhd8ed1ab_0 conda-forge/noarch Cached
+ pluggy 1.0.0 py310hbe9552e_3 conda-forge/osx-arm64 Cached
+ prompt-toolkit 3.0.29 pyha770c72_0 conda-forge/noarch Cached
+ prompt_toolkit 3.0.29 hd8ed1ab_0 conda-forge/noarch 5 KB
+ py 1.11.0 pyh6c4a22f_0 conda-forge/noarch Cached
+ pybind11-abi 4 hd8ed1ab_3 conda-forge/noarch 10 KB
+ pycosat 0.6.3 py310hf8d0d8f_1010 conda-forge/osx-arm64 Cached
+ pycparser 2.21 pyhd8ed1ab_0 conda-forge/noarch Cached
+ pygments 2.12.0 pyhd8ed1ab_0 conda-forge/noarch Cached
+ pyopenssl 22.0.0 pyhd8ed1ab_0 conda-forge/noarch Cached
+ pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge/noarch Cached
+ pyrsistent 0.18.1 py310hf8d0d8f_1 conda-forge/osx-arm64 Cached
+ pysocks 1.7.1 py310hbe9552e_5 conda-forge/osx-arm64 Cached
+ pytest 7.1.2 py310hbe9552e_0 conda-forge/osx-arm64 Cached
+ pytest-workflow 1.6.0 pyhd8ed1ab_0 conda-forge/noarch Cached
+ python 3.10.4 hfc7342c_0_cpython conda-forge/osx-arm64 12 MB
+ python_abi 3.10 2_cp310 conda-forge/osx-arm64 4 KB
+ pyyaml 6.0 py310hf8d0d8f_4 conda-forge/osx-arm64 Cached
+ questionary 1.10.0 pyhd8ed1ab_1 conda-forge/noarch Cached
+ readline 8.1 hedafd6a_0 conda-forge/osx-arm64 Cached
+ repoze.lru 0.7 py_0 conda-forge/noarch Cached
+ reproc 14.2.3 h3422bc3_0 conda-forge/osx-arm64 Cached
+ reproc-cpp 14.2.3 hbdafb3b_0 conda-forge/osx-arm64 Cached
+ requests 2.27.1 pyhd8ed1ab_0 conda-forge/noarch Cached
+ requests-cache 0.9.4 pyhd8ed1ab_0 conda-forge/noarch Cached
+ rich 12.4.1 pyhd8ed1ab_1 conda-forge/noarch Cached
+ rich-click 1.4 pyhd8ed1ab_0 conda-forge/noarch Cached
+ routes 2.5.1 pyhd8ed1ab_0 conda-forge/noarch Cached
+ ruamel_yaml 0.15.80 py310he2143c4_1006 conda-forge/osx-arm64 Cached
+ setuptools 62.3.2 py310hbe9552e_0 conda-forge/osx-arm64 Cached
+ six 1.16.0 pyh6c4a22f_0 conda-forge/noarch Cached
+ smmap 3.0.5 pyh44b312d_0 conda-forge/noarch Cached
+ sqlite 3.38.5 h40dfcc0_0 conda-forge/osx-arm64 Cached
+ tabulate 0.8.9 pyhd8ed1ab_0 conda-forge/noarch Cached
+ tk 8.6.12 he1e0b03_0 conda-forge/osx-arm64 Cached
+ tomli 2.0.1 pyhd8ed1ab_0 conda-forge/noarch Cached
+ tqdm 4.64.0 pyhd8ed1ab_0 conda-forge/noarch Cached
+ typing_extensions 4.2.0 pyha770c72_1 conda-forge/noarch Cached
+ tzdata 2022a h191b570_0 conda-forge/noarch Cached
+ url-normalize 1.4.3 pyhd8ed1ab_0 conda-forge/noarch Cached
+ urllib3 1.26.9 pyhd8ed1ab_0 conda-forge/noarch Cached
+ wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge/noarch Cached
+ webencodings 0.5.1 py_1 conda-forge/noarch Cached
+ wheel 0.37.1 pyhd8ed1ab_0 conda-forge/noarch Cached
+ whoosh 2.7.4 pyhd8ed1ab_2 conda-forge/noarch Cached
+ xz 5.2.5 h642e427_1 conda-forge/osx-arm64 Cached
+ yaml 0.2.5 h3422bc3_2 conda-forge/osx-arm64 Cached
+ yaml-cpp 0.6.3 hc88da5d_4 conda-forge/osx-arm64 122 KB
+ zipp 3.8.0 pyhd8ed1ab_0 conda-forge/noarch Cached
+ zlib 1.2.11 h90dfc92_1014 conda-forge/osx-arm64 Cached
+ zstd 1.5.2 hd705a24_1 conda-forge/osx-arm64 374 KB
Command used and terminal output
% nf-core modules create mash/screen
Traceback (most recent call last):
File "/Users/mahpa906/miniconda3/envs/nextflow-env/bin/nf-core", line 7, in <module>
from nf_core.__main__ import run_nf_core
File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/nf_core/__main__.py", line 19, in <module>
import nf_core.lint
File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/nf_core/lint/__init__.py", line 22, in <module>
import nf_core.modules.lint
File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/nf_core/modules/__init__.py", line 12, in <module>
from .mulled import MulledImageNameGenerator
File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/nf_core/modules/mulled.py", line 10, in <module>
from galaxy.tool_util.deps.mulled.util import build_target, v2_image_name
File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/galaxy/tool_util/deps/__init__.py", line 16, in <module>
from galaxy.util.oset import OrderedSet
File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/galaxy/util/oset.py", line 7, in <module>
class OrderedSet(collections.MutableSet):
AttributeError: module 'collections' has no attribute 'MutableSet'
% nf-core
Traceback (most recent call last):
File "/Users/mahpa906/miniconda3/envs/nextflow-env/bin/nf-core", line 7, in <module>
from nf_core.__main__ import run_nf_core
File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/nf_core/__main__.py", line 19, in <module>
import nf_core.lint
File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/nf_core/lint/__init__.py", line 22, in <module>
import nf_core.modules.lint
File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/nf_core/modules/__init__.py", line 12, in <module>
from .mulled import MulledImageNameGenerator
File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/nf_core/modules/mulled.py", line 10, in <module>
from galaxy.tool_util.deps.mulled.util import build_target, v2_image_name
File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/galaxy/tool_util/deps/__init__.py", line 16, in <module>
from galaxy.util.oset import OrderedSet
File "/Users/mahpa906/miniconda3/envs/nextflow-env/lib/python3.10/site-packages/galaxy/util/oset.py", line 7, in <module>
class OrderedSet(collections.MutableSet):
AttributeError: module 'collections' has no attribute 'MutableSet'
System information
% nextflow info
Version: 22.04.0 build 5697
Created: 23-04-2022 18:00 UTC (20:00 CEST)
System: Mac OS X 12.2.1
Runtime: Groovy 3.0.10 on OpenJDK 64-Bit Server VM 11.0.9.1+1-LTS
Encoding: UTF-8 (UTF-8)
Can't reproduce this error on my M1 Pro setup... (weirdly my nextflow info shows the wrong system version:
╰─ nextflow info
Version: 22.04.0 build 5697
Created: 23-04-2022 18:00 UTC (20:00 CEST)
System: Mac OS X 10.16
Runtime: Groovy 3.0.10 on OpenJDK 64-Bit Server VM 11.0.9.1+1-LTS
Encoding: UTF-8 (UTF-8)
should be Mac OS X 12.3.1 🤔 )
# packages in environment at /opt/homebrew/Caskroom/miniconda/base/envs/nextflow-env:
#
# Name Version Build Channel
appdirs 1.4.4 pyh9f0ad1d_0 conda-forge
attrs 21.4.0 pyhd8ed1ab_0 conda-forge
backports 1.0 py_2 conda-forge
backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge
bleach 5.0.0 pyhd8ed1ab_0 conda-forge
boltons 21.0.0 pyhd8ed1ab_0 conda-forge
brotlipy 0.7.0 py310h1961e1f_1004 conda-forge
bzip2 1.0.8 h0d85af4_4 conda-forge
c-ares 1.18.1 h0d85af4_0 conda-forge
ca-certificates 2022.5.18.1 h033912b_0 conda-forge
cattrs 1.10.0 pyhd8ed1ab_0 conda-forge
certifi 2022.5.18.1 py310h2ec42d9_0 conda-forge
cffi 1.15.0 py310hcc37b68_0 conda-forge
charset-normalizer 2.0.12 pyhd8ed1ab_0 conda-forge
cheetah3 3.2.6.post2 py310he24745e_1 conda-forge
click 8.1.3 py310h2ec42d9_0 conda-forge
colorama 0.4.4 pyh9f0ad1d_0 conda-forge
commonmark 0.9.1 py_0 conda-forge
conda 4.12.0 py310h2ec42d9_0 conda-forge
conda-package-handling 1.8.1 py310h1961e1f_1 conda-forge
coreutils 9.1 h5eb16cf_0 conda-forge
cryptography 37.0.2 py310h52c3658_0 conda-forge
curl 7.83.1 h372c54d_0 conda-forge
dataclasses 0.8 pyhc8e2a94_3 conda-forge
docutils 0.18.1 py310h2ec42d9_1 conda-forge
expat 2.4.8 h96cf925_0 conda-forge
future 0.18.2 py310h2ec42d9_5 conda-forge
galaxy-containers 21.9.0 pyhd8ed1ab_0 conda-forge
galaxy-tool-util 21.9.2 pyhd8ed1ab_0 conda-forge
galaxy-util 21.9.0 pyhd8ed1ab_0 conda-forge
gettext 0.19.8.1 hd1a6beb_1008 conda-forge
git 2.35.3 pl5321h33a4a8a_0 conda-forge
gitdb 4.0.9 pyhd8ed1ab_0 conda-forge
gitpython 3.1.27 pyhd8ed1ab_0 conda-forge
gmp 6.2.1 h2e338ed_0 conda-forge
icu 70.1 h96cf925_0 conda-forge
idna 3.3 pyhd8ed1ab_0 conda-forge
importlib-metadata 4.11.3 py310h2ec42d9_1 conda-forge
importlib_resources 5.7.1 pyhd8ed1ab_1 conda-forge
iniconfig 1.1.1 pyh9f0ad1d_0 conda-forge
itsdangerous 2.1.2 pyhd8ed1ab_0 conda-forge
jinja2 3.1.2 pyhd8ed1ab_0 conda-forge
jsonschema 4.5.1 pyhd8ed1ab_0 conda-forge
krb5 1.19.3 hb49756b_0 conda-forge
libarchive 3.5.2 hde4784d_2 conda-forge
libcurl 7.83.1 h372c54d_0 conda-forge
libcxx 14.0.3 hc203e6f_0 conda-forge
libedit 3.1.20191231 h0678c8f_2 conda-forge
libev 4.33 haf1e3a3_1 conda-forge
libffi 3.4.2 h0d85af4_5 conda-forge
libiconv 1.16 haf1e3a3_0 conda-forge
libmamba 0.23.1 h2d3d89a_1 conda-forge
libmambapy 0.23.1 py310hdd39e55_1 conda-forge
libnghttp2 1.47.0 h942079c_0 conda-forge
libsolv 0.7.22 hd9580d2_0 conda-forge
libssh2 1.10.0 h52ee1ee_2 conda-forge
libxml2 2.9.14 h08a9926_0 conda-forge
libzlib 1.2.11 h6c3fc93_1014 conda-forge
lz4-c 1.9.3 he49afe7_1 conda-forge
lzo 2.10 haf1e3a3_1000 conda-forge
mamba 0.23.1 py310h795411f_1 conda-forge
markdown 3.3.7 pyhd8ed1ab_0 conda-forge
markupsafe 2.1.1 py310h1961e1f_1 conda-forge
ncurses 6.3 h96cf925_1 conda-forge
nextflow 22.04.0 h4a94de4_0 bioconda
nf-core 2.4.1 pyh5e36f6f_0 bioconda
openjdk 11.0.9.1 h2292cb8_3 conda-forge
openssl 1.1.1o hfe4f2af_0 conda-forge
packaging 21.3 pyhd8ed1ab_0 conda-forge
pcre2 10.37 ha16e1b2_0 conda-forge
perl 5.32.1 2_h0d85af4_perl5 conda-forge
pip 22.0.4 pyhd8ed1ab_0 conda-forge
pluggy 1.0.0 py310h2ec42d9_3 conda-forge
prompt-toolkit 3.0.29 pyha770c72_0 conda-forge
prompt_toolkit 3.0.29 hd8ed1ab_0 conda-forge
py 1.11.0 pyh6c4a22f_0 conda-forge
pybind11-abi 4 hd8ed1ab_3 conda-forge
pycosat 0.6.3 py310h1961e1f_1010 conda-forge
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pycryptodome 3.14.1 py310h1ecf4fd_1 conda-forge
pygments 2.12.0 pyhd8ed1ab_0 conda-forge
pyopenssl 22.0.0 pyhd8ed1ab_0 conda-forge
pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge
pyrsistent 0.18.1 py310h1961e1f_1 conda-forge
pysocks 1.7.1 py310h2ec42d9_5 conda-forge
pytest 7.1.2 py310h2ec42d9_0 conda-forge
pytest-workflow 1.6.0 pyhd8ed1ab_0 conda-forge
python 3.10.4 h8b4d769_0_cpython conda-forge
python_abi 3.10 2_cp310 conda-forge
pyyaml 6.0 py310h1961e1f_4 conda-forge
questionary 1.10.0 pyhd8ed1ab_1 conda-forge
readline 8.1 h05e3726_0 conda-forge
repoze.lru 0.7 py_0 conda-forge
reproc 14.2.3 h0d85af4_0 conda-forge
reproc-cpp 14.2.3 he49afe7_0 conda-forge
requests 2.27.1 pyhd8ed1ab_0 conda-forge
requests-cache 0.9.4 pyhd8ed1ab_0 conda-forge
rich 12.4.1 pyhd8ed1ab_1 conda-forge
rich-click 1.4 pyhd8ed1ab_0 conda-forge
routes 2.5.1 pyhd8ed1ab_0 conda-forge
ruamel_yaml 0.15.80 py310he24745e_1006 conda-forge
setuptools 62.3.2 py310h2ec42d9_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
smmap 3.0.5 pyh44b312d_0 conda-forge
sqlite 3.38.5 hd9f0692_0 conda-forge
tabulate 0.8.9 pyhd8ed1ab_0 conda-forge
tk 8.6.12 h5dbffcc_0 conda-forge
tomli 2.0.1 pyhd8ed1ab_0 conda-forge
tqdm 4.64.0 pyhd8ed1ab_0 conda-forge
typing_extensions 4.2.0 pyha770c72_1 conda-forge
tzdata 2022a h191b570_0 conda-forge
url-normalize 1.4.3 pyhd8ed1ab_0 conda-forge
urllib3 1.26.9 pyhd8ed1ab_0 conda-forge
wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge
webencodings 0.5.1 py_1 conda-forge
wheel 0.37.1 pyhd8ed1ab_0 conda-forge
whoosh 2.7.4 py310h2ec42d9_5 conda-forge
xz 5.2.5 haf1e3a3_1 conda-forge
yaml 0.2.5 h0d85af4_2 conda-forge
yaml-cpp 0.6.3 hb1e8313_4 conda-forge
zipp 3.8.0 pyhd8ed1ab_0 conda-forge
zipstream-new 1.1.8 pyhd8ed1ab_0 conda-forge
zlib 1.2.11 h6c3fc93_1014 conda-forge
zstd 1.5.2 ha9df2e0_1 conda-forge
I seem to have a different version of the galaxy tools, if it is the faulty package as phil mentioned on Slack.
why did mine take from bioconda, while yours took from forge, if they're built from the same file?
Probably something to do with the channel priority in you conda settings. conda-forge should have greater priority than bioconda
Probably something to do with the channel priority in you conda settings. conda-forge should have greater priority than bioconda
That is the channel priority:
channels:
- conda-forge
- bioconda
- defaults
Also I can run nf-core locally through the docker image so I can still use the tool. It's just that it should be fixed for other users.
I think the cause for this is the architecture. In your env, everything's arm64, while @mashehu's env is the 'regular' x86-x64 build. Depends on which conda/mamba you initially installed I presume.
Yes:
channels:
- conda-forge
- bioconda
- defaults
dependencies:
- nextflow=22.04.5
- nf-core=2.5
- mamba=0.23.1
% nf-core create my_module/my_tool
Traceback (most recent call last):
File "/Users/mahpa906/miniconda3/envs/nxf_test/bin/nf-core", line 7, in <module>
from nf_core.__main__ import run_nf_core
File "/Users/mahpa906/miniconda3/envs/nxf_test/lib/python3.10/site-packages/nf_core/__main__.py", line 19, in <module>
import nf_core.lint
File "/Users/mahpa906/miniconda3/envs/nxf_test/lib/python3.10/site-packages/nf_core/lint/__init__.py", line 23, in <module>
import nf_core.modules.lint
File "/Users/mahpa906/miniconda3/envs/nxf_test/lib/python3.10/site-packages/nf_core/modules/__init__.py", line 11, in <module>
from .mulled import MulledImageNameGenerator
File "/Users/mahpa906/miniconda3/envs/nxf_test/lib/python3.10/site-packages/nf_core/modules/mulled.py", line 9, in <module>
from galaxy.tool_util.deps.mulled.util import build_target, v2_image_name
File "/Users/mahpa906/miniconda3/envs/nxf_test/lib/python3.10/site-packages/galaxy/tool_util/deps/__init__.py", line 16, in <module>
from galaxy.util.oset import OrderedSet
File "/Users/mahpa906/miniconda3/envs/nxf_test/lib/python3.10/site-packages/galaxy/util/oset.py", line 7, in <module>
class OrderedSet(collections.MutableSet):
AttributeError: module 'collections' has no attribute 'MutableSet'
Can you please try the following to see if it is a M1 issue or not:
conda create --name tst-glxy python==3.7
conda activate tst-glxy
conda install --channel base --channel bioconda --channel conda-forge galaxy-tool-util
And then report back if the OrderedSet can be imported?
python -c "from galaxy.util.oset import OrderedSet"
If this works, nf-core installation should work too, and it might be a local setting that prevents the installation. If that throws the same error as before, we at least have a less complicated way to reproduce it.
After the test clean-up with:
conda deactivate
conda env remove --name tst-glxy
This line fails already:
% conda create --name tst-glxy python==3.7
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:
- python==3.7
Current channels:
- https://conda.anaconda.org/conda-forge/osx-arm64
- https://conda.anaconda.org/conda-forge/noarch
- https://conda.anaconda.org/bioconda/osx-arm64
- https://conda.anaconda.org/bioconda/noarch
- https://repo.anaconda.com/pkgs/main/osx-arm64
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/r/osx-arm64
- https://repo.anaconda.com/pkgs/r/noarch
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
Can please try with conda create --name tst-glxy python==3.8. After all Python 3.7 is deprecated.
Oh, maybe my bad, I think it might be because it should be a single equal sign instead of two 🙈
% conda create --name tst-glxy python=3.8
<success>
% conda activate tst-glxy
<success>
% conda install --channel base --channel bioconda --channel conda-forge galaxy-tool-util
<success>
% python -c "from galaxy.util.oset import OrderedSet"
So last step seems to have worked.
If you now install nf-core into this environment, can you then create a new workflow?
% conda install nf-core=2.5
<success>
% nf-core modules create samtools/test
,--./,-.
___ __ __ __ ___ /,-._.--~\
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'
nf-core/tools version 2.5 - https://nf-co.re
INFO Repository type: modules create.py:93
INFO Press enter to use default values (shown in brackets) or type your own responses. ctrl+click underlined text to open links. create.py:97
INFO Using Bioconda package: 'bioconda::samtools=1.15.1' create.py:165
INFO Using Docker container: 'quay.io/biocontainers/samtools:1.15.1--h1170115_0' create.py:191
INFO Using Singularity container: 'https://depot.galaxyproject.org/singularity/samtools:1.15.1--h1170115_0' create.py:192
GitHub Username: (@author): @mahesh-panchal
INFO Provide an appropriate resource label for the process, taken from the nf-core pipeline template. create.py:218
For example: process_low, process_medium, process_high, process_long
? Process resource label: process_low
INFO Where applicable all sample-specific information e.g. 'id', 'single_end', 'read_group' MUST be provided as an input via a Groovy Map create.py:232
called 'meta'. This information may not be required in some instances, for example indexing reference genome files.
Will the module require a meta map of sample information? [y/n] (y):
INFO Created / edited following files: create.py:270
./modules/samtools/test/main.nf
./modules/samtools/test/meta.yml
./tests/modules/samtools/test/main.nf
./tests/modules/samtools/test/test.yml
./tests/modules/samtools/test/nextflow.config
./tests/config/pytest_modules.yml
new workflow works too:
% nf-core create
,--./,-.
___ __ __ __ ___ /,-._.--~\
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'
nf-core/tools version 2.5 - https://nf-co.re
? Workflow name test
? Description tester
? Author me
? Do you want to customize which parts of the template are used? Yes
? Pipeline prefix test
? Skip template areas? done
INFO Creating new nf-core pipeline: 'test/test' create.py:236
WARNING Prettier not found. Please install it and run it on the pipeline to fix linting issues. create.py:383
WARNING Prettier is not installed. Please install it and run it on the pipeline to fix linting issues. create.py:461
INFO Initialising pipeline git repository create.py:538
INFO Done. Remember to add a remote and push to GitHub: create.py:545
cd /Users/mahpa906/Documents/Sandbox/Nextflow/test-test
git remote add origin [email protected]:USERNAME/REPO_NAME.git
git push --all origin
INFO This will also push your newly created dev branch and the TEMPLATE branch for syncing. create.py:551
OK, so we know it is not per-se impossible to have nf-core tools work on an M1 🥳
Can you maybe run
find . -name "oset.py" -exec grep --with-filename --line-number 'collections.MutableSet' {} \;
Maybe there is an erroneous installation in the conda base conda environment? Or are the galaxy packages installed in your conda base environment?
Based on your feedback @mahesh-panchal, I think there should be a new issue about nf-core not supporting Python3.7 on M1 processors and propose to deprecate support for Python 3.7.
Ping @ewels
If you reproduce the steps above that led to a working nf-core install but skip the galaxy-tool-util installation, does it still work?
Does it work with and without the channel specification on the CLI when installing nf-core?
Pinning python to 3.8 seems to fix the issue:
conda environment file:
channels:
- conda-forge
- bioconda
- defaults
dependencies:
- python=3.8
- nextflow=22.04.5
- nf-core=2.5
- mamba=0.25.0
fresh install:
mamba env create -f nextflow-env.yml -n test_nf-core
New workflow creation works:
% nf-core create
,--./,-.
___ __ __ __ ___ /,-._.--~\
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'
nf-core/tools version 2.5 - https://nf-co.re
? Workflow name test
? Description tester
? Author me
? Do you want to customize which parts of the template are used? No
INFO Creating new nf-core pipeline: 'nf-core/test' create.py:236
INFO Initialising pipeline git repository create.py:538
INFO Done. Remember to add a remote and push to GitHub: create.py:545
cd /Users/mahpa906/Documents/Sandbox/Bash/test_conda_nfcore/nf-core-test
git remote add origin [email protected]:USERNAME/REPO_NAME.git
git push --all origin
INFO This will also push your newly created dev branch and the TEMPLATE branch for syncing. create.py:551
INFO !!!!!! IMPORTANT !!!!!! create.py:227
If you are interested in adding your pipeline to the nf-core community,
PLEASE COME AND TALK TO US IN THE NF-CORE SLACK BEFORE WRITING ANY CODE!
Please read: https://nf-co.re/developers/adding_pipelines#join-the-community
and so does module creation:
% nf-core modules create samtools/tester
,--./,-.
___ __ __ __ ___ /,-._.--~\
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'
nf-core/tools version 2.5 - https://nf-co.re
INFO Repository type: modules create.py:93
INFO Press enter to use default values (shown in brackets) or type your own responses. ctrl+click underlined text to open links. create.py:97
INFO Using Bioconda package: 'bioconda::samtools=1.15.1' create.py:165
INFO Using Docker container: 'quay.io/biocontainers/samtools:1.15.1--h1170115_0' create.py:191
INFO Using Singularity container: 'https://depot.galaxyproject.org/singularity/samtools:1.15.1--h1170115_0' create.py:192
GitHub Username: (@author): @me
INFO Provide an appropriate resource label for the process, taken from the nf-core pipeline template. create.py:218
For example: process_low, process_medium, process_high, process_long
? Process resource label: process_low
INFO Where applicable all sample-specific information e.g. 'id', 'single_end', 'read_group' MUST be provided as an input via a Groovy Map create.py:232
called 'meta'. This information may not be required in some instances, for example indexing reference genome files.
Will the module require a meta map of sample information? [y/n] (y):
INFO Created / edited following files: create.py:270
./modules/samtools/tester/main.nf
./modules/samtools/tester/meta.yml
./tests/modules/samtools/tester/main.nf
./tests/modules/samtools/tester/test.yml
./tests/modules/samtools/tester/nextflow.config
./tests/config/pytest_modules.yml