ampliseq icon indicating copy to clipboard operation
ampliseq copied to clipboard

Published 20 hours ago • verified publisher icon nf-core

Update software & nf-core modules

Open d4straub opened this issue 2 years ago • 3 comments

This updates QIIME2 to the most recent container, Picrust2, and nf-core modules.

Hm, tests worked locally, but in CI they fail with:

Error executing process > 'NFCORE_AMPLISEQ:AMPLISEQ:QIIME2_PREPTAX:QIIME2_EXTRACT (GTGYCAGCMGCCGCGGTAA-GGACTACNVGGGTWTCTAAT)'

Caused by:
  Process `NFCORE_AMPLISEQ:AMPLISEQ:QIIME2_PREPTAX:QIIME2_EXTRACT (GTGYCAGCMGCCGCGGTAA-GGACTACNVGGGTWTCTAAT)` terminated with an error exit status (1)

Command executed:

  export XDG_CONFIG_HOME="${PWD}/HOME"
  export MPLCONFIGDIR="${PWD}/HOME"
  
  ### Import
  qiime tools import --type 'FeatureData[Sequence]'         --input-path greengenes85.fna         --output-path ref-seq.qza
  qiime tools import --type 'FeatureData[Taxonomy]'         --input-format HeaderlessTSVTaxonomyFormat         --input-path greengenes85.tax         --output-path ref-taxonomy.qza
  #Extract sequences based on primers
  qiime feature-classifier extract-reads         --i-sequences ref-seq.qza         --p-f-primer GTGYCAGCMGCCGCGGTAA         --p-r-primer GGACTACNVGGGTWTCTAAT         --o-reads GTGYCAGCMGCCGCGGTAA-GGACTACNVGGGTWTCTAAT-ref-seq.qza         --quiet
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_AMPLISEQ:AMPLISEQ:QIIME2_PREPTAX:QIIME2_EXTRACT":
      qiime2: $( qiime --version | sed -e "s/q2cli version //g" | tr -d '`' | sed -e "s/Run qiime info for more version details.//g" )
  END_VERSIONS

Command exit status:
  1

Command output:
  (empty)

Command error:
  f3455a9d1521: Pull complete
  e48cafe6838b: Pull complete
  d567a5191575: Pull complete
  5516372d4765: Pull complete
  1caf0a6f49c2: Pull complete
  cf7ed69f5061: Pull complete
  Digest: sha256:5b0fb4b78b6df236c1d77d97e1f7c800187675c609584cde9c6bb7acefac76ea
  Status: Downloaded newer image for quay.io/qiime2/core:2022.2
  Traceback (most recent call last):
    File "/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/q2cli/builtin/tools.py", line 157, in import_data
      artifact = qiime2.sdk.Artifact.import_data(type, input_path,
    File "/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/sdk/result.py", line 249, in import_data
      pm = qiime2.sdk.PluginManager()
    File "/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/sdk/plugin_manager.py", line 67, in __new__
      self._init(add_plugins=add_plugins)
    File "/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/qiime2/sdk/plugin_manager.py", line 105, in _init
      plugin = entry_point.load()
    File "/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2465, in load
      return self.resolve()
    File "/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2471, in resolve
      module = __import__(self.module_name, fromlist=['__name__'], level=0)
    File "/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/q2_diversity/__init__.py", line 11, in <module>
      from ._beta import (beta, beta_phylogenetic, bioenv,
    File "/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/q2_diversity/_beta/__init__.py", line 13, in <module>
      from ._beta_rarefaction import beta_rarefaction
    File "/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/q2_diversity/_beta/_beta_rarefaction.py", line 23, in <module>
      from .._ordination import pcoa
    File "/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/q2_diversity/_ordination.py", line 13, in <module>
      import umap as up
    File "/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/umap/__init__.py", line 2, in <module>
      from .umap_ import UMAP
    File "/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/umap/umap_.py", line 41, in <module>
      from umap.layouts import (
    File "/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/umap/layouts.py", line 40, in <module>
      def rdist(x, y):
    File "/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/numba/core/decorators.py", line 214, in wrapper
      disp.enable_caching()
    File "/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/numba/core/dispatcher.py", line 812, in enable_caching
      self._cache = FunctionCache(self.py_func)
    File "/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/numba/core/caching.py", line 610, in __init__
      self._impl = self._impl_class(py_func)
    File "/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/numba/core/caching.py", line 347, in __init__
      raise RuntimeError("cannot cache function %r: no locator available "
  RuntimeError: cannot cache function 'rdist': no locator available for file '/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/umap/layouts.py'
  
  An unexpected error has occurred:
  
    cannot cache function 'rdist': no locator available for file '/opt/conda/envs/qiime2-2022.2/lib/python3.8/site-packages/umap/layouts.py'
  
  See above for debug info.

Work dir:
  /home/runner/work/ampliseq/ampliseq/work/74/fd0dafb5befdb8043c18a09d1ebdef

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`


Execution cancelled -- Finishing pending tasks before exit
Cannot invoke method getAt() on null object

 -- Check script './workflows/ampliseq.nf' at line: 272 or see '.nextflow.log' file for more details
WARN: Got an interrupted exception while taking agent result | java.lang.InterruptedException
-[nf-core/ampliseq] Pipeline completed with errors-
WARN: Killing running tasks (1)
Error: Process completed with exit code 1.

Only Info I found was https://forum.qiime2.org/t/docker-throwing-error-in-latest-version-of-qiime/23653/5 which identifies docker's -u make the process fail while when omitting -u it succeeds.

PR checklist

  • [x] This comment contains a description of changes (with reason).
  • [ ] If you've fixed a bug or added code that should be tested, add tests!
    • [ ] If you've added a new tool - have you followed the pipeline conventions in the contribution docs
    • [ ] If necessary, also make a PR on the nf-core/ampliseq branch on the nf-core/test-datasets repository.
  • [x] Make sure your code lints (nf-core lint).
  • [x] Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • [ ] Usage Documentation in docs/usage.md is updated.
  • [ ] Output Documentation in docs/output.md is updated.
  • [x] CHANGELOG.md is updated.
  • [ ] README.md is updated (including new tool citations and authors/contributors).

d4straub avatar Aug 19 '22 09:08 d4straub

nf-core lint overall result: Passed :white_check_mark: :warning:

Posted for pipeline commit 721bea9

+| ✅ 147 tests passed       |+
#| ❔   1 tests were ignored |#
!| ❗   1 tests had warnings |!

:heavy_exclamation_mark: Test warnings:

  • readme - README did not have a Nextflow minimum version badge.

:grey_question: Tests ignored:

:white_check_mark: Tests passed:

Run details

  • nf-core/tools version 2.4.1
  • Run at 2022-08-19 09:46:09

github-actions[bot] avatar Aug 19 '22 09:08 github-actions[bot]

Yes, the QIIME2 container seems to be broken for docker (which is used for GitHub tests, but it is fine with singularity that I use to test the pipeline locally). I'll contact the QIIME2 devs but I think I may not update QIIME2 at this point.

d4straub avatar Aug 24 '22 07:08 d4straub

Yes, the QIIME2 container seems to be broken for docker (which is used for GitHub tests, but it is fine with singularity that I use to test the pipeline locally). I'll contact the QIIME2 devs but I think I may not update QIIME2 at this point.

Strange, I thought singularity images were built from docker... In any case, not updating sounds like a good choice.

erikrikarddaniel avatar Aug 25 '22 07:08 erikrikarddaniel

I close that for now.

d4straub avatar Sep 01 '22 14:09 d4straub