`NFCORE_MAG:MAG:PROKKA (MEGAHIT-group-14.13)` terminated with an error exit status (2)

[anugos]

Hi, I am using NFCORE MAG for the first time and unable to understand the error I ran through with prokka. I have attached the error log below. I appreciate your help. Thanks Anu

The full error message was:

Error executing process > 'NFCORE_MAG:MAG:PROKKA (MEGAHIT-group-14.13)'

Caused by: Process NFCORE_MAG:MAG:PROKKA (MEGAHIT-group-14.13) terminated with an error exit status (2)

Command executed:

prokka
--metagenome
--cpus 2
--prefix MEGAHIT-group-14.13


MEGAHIT-group-14.13.fa

cat <<-END_VERSIONS > versions.yml PROKKA: prokka: $(echo $(prokka --version 2>&1) | sed 's/^.*prokka //') END_VERSIONS

Command exit status: 2

Command output: (empty)

Command error: [11:46:18] Determined minced version is 004002 from 'minced 0.4.2' [11:46:18] Looking for 'parallel' - found /usr/local/bin/parallel [11:46:18] Determined parallel version is 20200122 from 'GNU parallel 20200122' [11:46:18] Looking for 'prodigal' - found /usr/local/bin/prodigal [11:46:18] Determined prodigal version is 002006 from 'Prodigal V2.6.3: February, 2016' [11:46:18] Looking for 'prokka-genbank_to_fasta_db' - found /usr/local/bin/prokka-genbank_to_fasta_db [11:46:18] Looking for 'sed' - found /bin/sed [11:46:18] Looking for 'tbl2asn' - found /usr/local/bin/tbl2asn [11:46:18] Determined tbl2asn version is 025007 from 'tbl2asn 25.7 arguments:' [11:46:18] Using genetic code table 11. [11:46:18] Loading and checking input file: MEGAHIT-group-14.13.fa [11:46:18] Wrote 90 contigs totalling 264395 bp. [11:46:18] Predicting tRNAs and tmRNAs [11:46:18] Running: aragorn -l -gc11 -w MEGAHIT-group-14.13/MEGAHIT-group-14.13.fna [11:46:18] Found 0 tRNAs [11:46:18] Predicting Ribosomal RNAs [11:46:18] Running Barrnap with 2 threads [11:46:18] Found 0 rRNAs [11:46:18] Skipping ncRNA search, enable with --rfam if desired. [11:46:18] Total of 0 tRNA + rRNA features [11:46:18] Searching for CRISPR repeats [11:46:18] Found 0 CRISPRs [11:46:18] Predicting coding sequences [11:46:18] Contigs total 264395 bp, so using meta mode [11:46:18] Running: prodigal -i MEGAHIT-group-14.13/MEGAHIT-group-14.13.fna -c -m -g 11 -p meta -f sco -q [11:46:19] Found 245 CDS [11:46:19] Connecting features back to sequences [11:46:19] Not using genus-specific database. Try --usegenus to enable it. [11:46:19] Annotating CDS, please be patient. [11:46:19] Will use 2 CPUs for similarity searching.

--------------------- WARNING --------------------- MSG: Seq [k127_283175]: Terminator codon inside CDS!

--------------------- WARNING --------------------- MSG: Seq [k127_554079]: Terminator codon inside CDS!

--------------------- WARNING --------------------- MSG: Seq [k127_219440]: Terminator codon inside CDS!

--------------------- WARNING --------------------- MSG: Seq [k127_1177168]: Terminator codon inside CDS!

[11:46:19] There are still 245 unannotated CDS left (started with 245) [11:46:19] Will use blast to search against /usr/local/db/kingdom/Bacteria/IS with 2 CPUs [11:46:19] Running: cat MEGAHIT-group-14.13/MEGAHIT-group-14.13.IS.tmp.155872.faa | parallel --gnu --plain -j 2 --block 14971 --recstart '>' --pipe blastp -query - -db /usr/local/db/kingdom/Bacteria/IS -evalue 1e-30 -qcov_hsp_perc 90 -num_threads 1 -num_descriptions 1 -num_alignments 1 -seg no > MEGAHIT-group-14.13/MEGAHIT-group-14.13.IS.tmp.155872.blast 2> /dev/null [11:46:19] Could not run command: cat MEGAHIT-group-14.13/MEGAHIT-group-14.13.IS.tmp.155872.faa | parallel --gnu --plain -j 2 --block 14971 --recstart '>' --pipe blastp -query - -db /usr/local/db/kingdom/Bacteria/IS -evalue 1e-30 -qcov_hsp_perc 90 -num_threads 1 -num_descriptions 1 -num_alignments 1 -seg no > MEGAHIT-group-14.13/MEGAHIT-group-14.13.IS.tmp.155872.blast 2> /dev/null

Work dir: /data/user/anugos24/Black-Queen-analysis/Shotgun-Metagenome/work/7a/b063017aba87ec3d207c28fafc24ab

Tip: you can replicate the issue by changing to the process work dir and entering the command bash .[command.run](http://command.run/) The workflow was completed at 2022-06-04T11:46:21.455665-05:00 (duration: 19.2s)

The command used to launch the workflow was as follows:

nextflow run nf-core/mag -r 2.1.1 -name bq-mag3 -profile cheaha -params-file nf-params.json -c custom.config -resume bq-mag Pipeline Configuration: revision 2.1.1 runName bq-mag3 containerEngine singularity launchDir /data/user/anugos24/Black-Queen-analysis/Shotgun-Metagenome workDir /data/user/anugos24/Black-Queen-analysis/Shotgun-Metagenome/work projectDir /home/anugos24/.nextflow/assets/nf-core/mag userName anugos24 profile cheaha configFiles /home/anugos24/.nextflow/assets/nf-core/mag/nextflow.config, /data/user/anugos24/Black-Queen-analysis/Shotgun-Metagenome/custom.config input /data/user/anugos24/Black-Queen-analysis/Shotgun-Metagenome/sample_input.csv email [email protected] multiqc_title MultiQC config_profile_name cheaha config_profile_description University of Alabama at Birmingham Cheaha HPC config_profile_contact Lara Ianov ([email protected]) or Austyn Trull ([email protected]) config_profile_url https://www.uab.edu/cores/ircp/bds max_cpus 128 max_memory 750 GB max_time 6d 6h phix_reference /home/anugos24/.nextflow/assets/nf-core/mag/assets/data/GCA_002596845.1_ASM259684v1_genomic.fna.gz lambda_reference /home/anugos24/.nextflow/assets/nf-core/mag/assets/data/GCA_000840245.1_ViralProj14204_genomic.fna.gz centrifuge_db /data/user/anugos24/Black-Queen-analysis/Shotgun-Metagenome/database/p_compressed+h+v.tar.gz kraken2_db /data/user/anugos24/Black-Queen-analysis/Shotgun-Metagenome/database/minikraken_8GB_202003.tgz cat_db /data/user/anugos24/Black-Queen-analysis/Shotgun-Metagenome/database/CAT_prepare_20210107.tar.gz gtdb https://data.ace.uq.edu.au/public/gtdb/data/releases/release202/202.0/auxillary_files/gtdbtk_r202_data.tar.gz coassemble_group true megahit_options --presets meta-large skip_spades true skip_spadeshybrid true busco_reference /data/user/anugos24/Black-Queen-analysis/Shotgun-Metagenome/database/bacteria_odb10.2020-03-06.tar.gz Date Started 2022-06-04T11:46:02.209580-05:00 Date Completed 2022-06-04T11:46:21.455665-05:00 Pipeline script file path /home/anugos24/.nextflow/assets/nf-core/mag/main.nf Pipeline script hash ID d8ab3b479c5710afc59c1c28658b47a5 Pipeline repository Git URL https://github.com/nf-core/mag Pipeline repository Git Commit e065754c46eedc85d46cd8a71b32ad73b3b741cb Pipeline Git branch/tag 2.1.1 Nextflow Version 21.08.0.edge Nextflow Build 5609 Nextflow Compile Timestamp 30-08-2021 16:39 UTC nf-core/mag

https://github.com/nf-core/mag

[d4straub]

Hi there,

first thing that seems error prone is

Nextflow Version 21.08.0.edge

Using an edge version can get quickly out of hand, those have often bugs and are intended for developer, not for end users. Better use the latest sable release. I do not think that this is the problem though.

Your error message

[11:46:19] Running: cat MEGAHIT-group-14.13/MEGAHIT-group-14.13.IS.tmp.155872.faa | parallel --gnu --plain -j 2 --block 14971 --recstart '>' --pipe blastp -query - -db /usr/local/db/kingdom/Bacteria/IS -evalue 1e-30 -qcov_hsp_perc 90 -num_threads 1 -num_descriptions 1 -num_alignments 1 -seg no > MEGAHIT-group-14.13/MEGAHIT-group-14.13.IS.tmp.155872.blast 2> /dev/null [11:46:19] Could not run command: cat MEGAHIT-group-14.13/MEGAHIT-group-14.13.IS.tmp.155872.faa | parallel --gnu --plain -j 2 --block 14971 --recstart '>' --pipe blastp -query - -db /usr/local/db/kingdom/Bacteria/IS -evalue 1e-30 -qcov_hsp_perc 90 -num_threads 1 -num_descriptions 1 -num_alignments 1 -seg no > MEGAHIT-group-14.13/MEGAHIT-group-14.13.IS.tmp.155872.blast 2> /dev/null

sounds a lot like https://github.com/tseemann/prokka/issues/402 It might be possible to solve your issue with another Prokka version, by using a different container, e.g. following https://nf-co.re/mag/2.1.1/usage#updating-containers using https://bioconda.github.io/recipes/prokka/README.html or container tags https://quay.io/repository/biocontainers/prokka?tab=tags. Currently used container is https://github.com/nf-core/mag/blob/e065754c46eedc85d46cd8a71b32ad73b3b741cb/modules/nf-core/modules/prokka/main.nf#L13-L18

[skrakau]

Hi @anugos, did you try the solution @d4straub suggested, could the problem be solved?

[anugos]

No it didn't work. As of now I am reassembling reads with metaspades and will retry the prokka. I will send you updates if it work this time or not. Thank you for your time. Have a nice day.

Best- Anu

On Mon, Jul 18, 2022, 08:17 Sabrina Krakau @.***> wrote:

Hi @anugos https://github.com/anugos, did you try the solution @d4straub https://github.com/d4straub suggested, could the problem be solved?

— Reply to this email directly, view it on GitHub https://github.com/nf-core/mag/issues/320#issuecomment-1187402808, or unsubscribe https://github.com/notifications/unsubscribe-auth/API7OGUQCIU3TA2FKDKCVIDVUVKQJANCNFSM5X4CQT5Q . You are receiving this because you were mentioned.Message ID: @.***>

[maxibor]

I'm running into the same problem on a cluster when using remote execution. Running the process on a local node, even using singularity works fine for me.

[d4straub]

Not solving you problem here, but I think we should change to https://github.com/oschwengers/bakta, because prokka seems to be not updated any more and bakta (at least from what I read) seems superior.

[jfy133]

nf-core module exists already if someone fancies jumping on this ;)

[CJREID]

@anugos @maxibor I had the same problem running via slurm on HPC and was able to resolve it by switching the prokka container to this one https://depot.galaxyproject.org/singularity/prokka%3A1.14.6--pl5321hdfd78af_4

[plaquette]

Not solving you problem here, but I think we should change to https://github.com/oschwengers/bakta, because prokka seems to be not updated any more and bakta (at least from what I read) seems superior.

i have the same issue - but using the mentioned container on slurm did not solve it for me :(

[plaquette]

... as an addon - i tried basically every container i could find on dockerhub - always with the same file

it runs on a hpc cluster via singularity.

[it works fine to install prokka via conda and use it that way -> it only fails in the container]

[jfy133]

@plaquette could you please supply more information e.g. commands, error messages, .nextflkow.log files etc

[plaquette]

sure:

i tried it as part of the nf-core/bacass pipeline.

where i used slurm as an executer on our centos 7.9 based cluster with the singularity profile and nf version 22.10.6

i tried the following containers as a replacement for the default ones:

depot.galaxyproject.org-singularity-prokka-1.14.6--pl526_0.img
depot.galaxyproject.org-singularity-prokka-1.14.6--pl5321hdfd78af_4.img
nanozoo-prokka-1.14.6--c99ff65.img
plaquette-prokka_helper-conda.img
staphb-prokka-1.13.img
staphb-prokka-1.14.0.img
staphb-prokka-1.14.6.img
staphb-prokka-latest.img

and as mentioned above - it always fails with something like this:

[12:14:00] Running: cat \/data\/ERR016613\.AMR\.tmp\.1\.faa | parallel --gnu --plain -j 4 --block 61191 --recstart '>' --pipe blastp -query - -db /root/prokka/db/kingdom/Bacteria/AMR -evalue 9.99999999999999e-301 -qcov_hsp_perc 90 -num_threads 1 -num_descriptions 1 -num_alignments 1 -seg no > \/data\/ERR016613\.AMR\.tmp\.1\.blast 2> /dev/null

[jfy133]

@plaquette could you please share the .command.log file oin the reported working directroy of the errored profcess?

[plaquette]

sure:

WARNING: Skipping mount /var/singularity/mnt/session/etc/resolv.conf [files]: /etc/resolv.conf doesn't exist in container
[11:02:49] This is prokka 1.14.6
[11:02:49] Written by Torsten Seemann <[email protected]>
[11:02:49] Homepage is https://github.com/tseemann/prokka
[11:02:49] Local time is Thu Apr 27 11:02:49 2023
[11:02:49] You are not telling me who you are!
[11:02:49] Operating system is linux
[11:02:49] You have BioPerl 1.007002
[11:02:49] System has 128 cores.
[11:02:49] Will use maximum of 1 cores.
[11:02:49] Annotating as >>> Bacteria <<<
[11:02:49] Generating locus_tag from 'ERR016616.scaffolds.fa' contents.
[11:02:49] Setting --locustag JPFOJJCG from MD5 39f833c0760e34d88161046764734911
[11:02:49] Creating new output folder: ERR016616
[11:02:49] Running: mkdir -p ERR016616
[11:02:49] Using filename prefix: ERR016616.XXX
[11:02:49] Setting HMMER_NCPU=1
[11:02:49] Writing log to: ERR016616/ERR016616.log
[11:02:49] Command: /usr/local/bin/prokka --cpus 1 --prefix ERR016616 ERR016616.scaffolds.fa
[11:02:49] Appending to PATH: /usr/local/bin
[11:02:49] Looking for 'aragorn' - found /usr/local/bin/aragorn
[11:02:49] Determined aragorn version is 001002 from 'ARAGORN v1.2.38 Dean Laslett'
[11:02:49] Looking for 'barrnap' - found /usr/local/bin/barrnap
[11:02:49] Determined barrnap version is 000009 from 'barrnap 0.9'
[11:02:49] Looking for 'blastp' - found /usr/local/bin/blastp
[11:02:50] Determined blastp version is 002009 from 'blastp: 2.9.0+'
[11:02:50] Looking for 'cmpress' - found /usr/local/bin/cmpress
[11:02:50] Determined cmpress version is 001001 from '# INFERNAL 1.1.2 (July 2016)'
[11:02:50] Looking for 'cmscan' - found /usr/local/bin/cmscan
[11:02:50] Determined cmscan version is 001001 from '# INFERNAL 1.1.2 (July 2016)'
[11:02:50] Looking for 'egrep' - found /bin/egrep
[11:02:50] Looking for 'find' - found /usr/bin/find
[11:02:50] Looking for 'grep' - found /bin/grep
[11:02:50] Looking for 'hmmpress' - found /usr/local/bin/hmmpress
[11:02:50] Determined hmmpress version is 003003 from '# HMMER 3.3 (Nov 2019); http://hmmer.org/'
[11:02:50] Looking for 'hmmscan' - found /usr/local/bin/hmmscan
[11:02:50] Determined hmmscan version is 003003 from '# HMMER 3.3 (Nov 2019); http://hmmer.org/'
[11:02:50] Looking for 'java' - found /usr/local/bin/java
[11:02:50] Looking for 'makeblastdb' - found /usr/local/bin/makeblastdb
[11:02:50] Determined makeblastdb version is 002009 from 'makeblastdb: 2.9.0+'
[11:02:50] Looking for 'minced' - found /usr/local/bin/minced
[11:02:50] Determined minced version is 004002 from 'minced 0.4.2'
[11:02:50] Looking for 'parallel' - found /usr/local/bin/parallel
[11:02:50] Determined parallel version is 20200122 from 'GNU parallel 20200122'
[11:02:50] Looking for 'prodigal' - found /usr/local/bin/prodigal
[11:02:50] Determined prodigal version is 002006 from 'Prodigal V2.6.3: February, 2016'
[11:02:50] Looking for 'prokka-genbank_to_fasta_db' - found /usr/local/bin/prokka-genbank_to_fasta_db
[11:02:50] Looking for 'sed' - found /bin/sed
[11:02:50] Looking for 'tbl2asn' - found /usr/local/bin/tbl2asn
[11:02:50] Determined tbl2asn version is 025007 from 'tbl2asn 25.7   arguments:'
[11:02:50] Using genetic code table 11.
[11:02:50] Loading and checking input file: ERR016616.scaffolds.fa
[11:02:50] Wrote 111 contigs totalling 1730507 bp.
[11:02:50] Predicting tRNAs and tmRNAs
[11:02:50] Running: aragorn -l -gc11  -w ERR016616\/ERR016616\.fna
[11:02:51] 1 tRNA-Met c[1033,1109] 35 (cat)
[11:02:51] 2 tRNA-Ala [137857,137932] 34 (tgc)
[11:02:51] 3 tRNA-Ile [137940,138016] 35 (gat)
[11:02:51] 1 tRNA-Gln [48018,48092] 33 (ttg)
[11:02:51] 2 tRNA-Met [48124,48199] 34 (cat)
[11:02:51] 3 tRNA-Phe c[48414,48489] 34 (gaa)
[11:02:51] 1 tRNA-Ser c[34949,35036] 35 (tga)
[11:02:51] 2 tRNA-Cys c[35068,35141] 33 (gca)
[11:02:51] 3 tRNA-Leu c[35163,35251] 35 (taa)
[11:02:51] 4 tRNA-Gly c[35262,35336] 33 (gcc)
[11:02:51] 1 tmRNA [34441,34799] 89,133 ANNVKFAPAYAKAA*
[11:02:51] 2 tRNA-Ser c[45070,45160] 35 (gct)
[11:02:51] 1 tRNA-Leu c[20914,20998] 35 (tag)
[11:02:51] 2 tRNA-Arg c[21004,21080] 35 (tct)
[11:02:51] 3 tRNA-Arg c[21095,21171] 35 (tcg)
[11:02:51] 4 tRNA-His c[21176,21252] 35 (gtg)
[11:02:51] 5 tRNA-Pro c[21277,21354] 36 (tgg)
[11:02:51] 1 tRNA-Trp c[11911,11986] 34 (cca)
[11:02:51] 2 tRNA-Thr c[13469,13543] 33 (ggt)
[11:02:51] 3 tRNA-Gly c[13638,13714] 35 (tcc)
[11:02:51] 4 tRNA-Tyr c[13721,13806] 35 (gta)
[11:02:51] 5 tRNA-Thr c[13868,13943] 34 (tgt)
[11:02:51] 1 tRNA-Asn c[25363,25437] 33 (gtt)
[11:02:51] 1 tRNA-Glu c[193,267] 34 (ttc)
[11:02:51] 1 tRNA-Leu c[9319,9404] 35 (gag)
[11:02:51] 1 tRNA-Arg [3151,3227] 35 (cct)
[11:02:51] 1 tRNA-SeC c[16493,16586] 35 (tca)
[11:02:51] 1 tRNA-Arg [13511,13587] 35 (gcg)
[11:02:51] 1 tRNA-Val c[317,392] 34 (gac)
[11:02:51] 2 tRNA-Ala c[394,469] 34 (ggc)
[11:02:51] 3 tRNA-Ser c[1679,1766] 35 (gga)
[11:02:51] 4 tRNA-Met [1871,1947] 35 (cat)
[11:02:51] Found 32 tRNAs
[11:02:51] Predicting Ribosomal RNAs
[11:02:51] Running Barrnap with 1 threads
[11:02:52] 1 1 136246 16S ribosomal RNA
[11:02:52] 2 71 621 23S ribosomal RNA
[11:02:52] Found 2 rRNAs
[11:02:52] Skipping ncRNA search, enable with --rfam if desired.
[11:02:52] Total of 33 tRNA + rRNA features
[11:02:52] Searching for CRISPR repeats
[11:02:53] CRISPR1 1 18 with 3 spacers
[11:02:53] Found 1 CRISPRs
[11:02:53] Predicting coding sequences
[11:02:53] Contigs total 1730507 bp, so using single mode
[11:02:53] Running: prodigal -i ERR016616\/ERR016616\.fna -c -m -g 11 -p single -f sco -q
[11:02:55] Excluding CDS which overlaps existing RNA (rRNA) at 71:3024..3392 on - strand
[11:02:55] Found 1723 CDS
[11:02:55] Connecting features back to sequences
[11:02:55] Not using genus-specific database. Try --usegenus to enable it.
[11:02:55] Annotating CDS, please be patient.
[11:02:55] Will use 1 CPUs for similarity searching.
[11:02:55] There are still 1723 unannotated CDS left (started with 1723)
[11:02:55] Will use blast to search against /usr/local/db/kingdom/Bacteria/IS with 1 CPUs
[11:02:55] Running: cat ERR016616\/ERR016616\.IS\.tmp\.49065\.faa | parallel --gnu --plain -j 1 --block 267740 --recstart '>' --pipe blastp -query - -db /usr/local/db/kingdom/Bacteria/IS -evalue 1e-30 -qcov_hsp_perc 90 -num_threads 1 -num_descriptions 1 -num_alignments 1 -seg no > ERR016616\/ERR016616\.IS\.tmp\.49065\.blast 2> /dev/null
[11:02:55] Could not run command: cat ERR016616\/ERR016616\.IS\.tmp\.49065\.faa | parallel --gnu --plain -j 1 --block 267740 --recstart '>' --pipe blastp -query - -db /usr/local/db/kingdom/Bacteria/IS -evalue 1e-30 -qcov_hsp_perc 90 -num_threads 1 -num_descriptions 1 -num_alignments 1 -seg no > ERR016616\/ERR016616\.IS\.tmp\.49065\.blast 2> /dev/null

sorry for the delayed response!

[jfy133]

Hm the log is unfortunately not very helpful there, I'm not sure what else I can suggest @plaquette sorry :(.

[plaquette]

nonetheless, thank you very much @jfy133 ! ... i assume its a strange centOS issue.

prokka works from conda env. but not in the container

i build (for testing purposes) a centOS specific prokka container (-> https://github.com/plaquette/prokka_container) but even this died with the same error.

(in case its relevant i tested many of the prokka containers on docker hub as well - but all with the same error as above)

[jfy133]

Yes sounds like it :(, sorry @plaquette !

I guess we can shut this now as someone else reported the issue with the updated container (1.14.6) which we already have in the pipeline now.

We can reopen if it crops up again.