no contig files on metaspades run, but no error in log file

[ewissel]

Description of bug

When I run metaspades.py via a conda environment, the process will run but no contig files are created. I seem to get all the intermediate files. I was able to assemble successfully in the past with the same conda environment so I am not sure what I've done to produce these weird results. Replicated with a fresh conda env.

Here is my code to install spades:

conda create -y -n spades-test -c conda-forge -c bioconda spades python=3.7.7

conda activate spades-test

There are no errors in the spades log, but it looks as though the log just times out. I use the following command to launch metaspades:

metaspades.py --12 /home/ewissel/minham/complex_meta/og_metas/largepan_01/largepan_01.fq -o ../og_metas/metaspades_out/largepan_01.fq/ -t 15

I have also tried to add the --continue flag, but get the error that the params.txt file cannot be found, even though it is in the output directory. I've tried changing the number of threads allocated as well without any change in the result. I have also tried with samples where the paired ends are in different files with the same result.

Should I be using spades.py --meta instead of metaspades.py? Maybe the memory limit is insufficient? But I would expect these things to throw errors.

At this stage, I think I've been staring at it for a bit too long and am looking forward to your feedback!

spades.log

Command line: /home/ewissel/miniconda3/envs/mags/bin/metaspades.py --12 /home/ewissel/minham/complex_meta/og_metas/largepan_01/largepan_01.fq -o /home/ewissel/minham/complex_meta/og_metas/metaspades_out/largepan_01.fq -t 15

System information: SPAdes version: 3.15.4 Python version: 3.8.13 OS: Linux-4.19.0-14-amd64-x86_64-with-glibc2.10

Output dir: /home/ewissel/minham/complex_meta/og_metas/metaspades_out/largepan_01.fq Mode: read error correction and assembling Debug mode is turned OFF

Dataset parameters: Metagenomic mode Reads: Library number: 1, library type: paired-end orientation: fr left reads: not specified right reads: not specified interlaced reads: ['/home/ewissel/minham/complex_meta/og_metas/largepan_01/largepan_01.fq'] single reads: not specified merged reads: not specified Read error correction parameters: Iterations: 1 PHRED offset will be auto-detected Corrected reads will be compressed Assembly parameters: k: [21, 33, 55] Repeat resolution is enabled Mismatch careful mode is turned OFF MismatchCorrector will be SKIPPED Coverage cutoff is turned OFF Other parameters: Dir for temp files: /home/ewissel/minham/complex_meta/og_metas/metaspades_out/largepan_01.fq/tmp Threads: 15 Memory limit (in Gb): 250

======= SPAdes pipeline started. Log can be found here: /home/ewissel/minham/complex_meta/og_metas/metaspades_out/largepan_01.fq/spades.log

/home/ewissel/minham/complex_meta/og_metas/largepan_01/largepan_01.fq: max reads length: 100

Reads length: 100

===== Before start started.

===== Preprocess reads started.

===== Preprocess interlaced reads started.

== Running: /home/ewissel/miniconda3/envs/mags/bin/python /home/ewissel/miniconda3/envs/mags/share/spades/spades_pipeline/scripts/preprocess_interlaced_reads.py --args_filename /home/ewissel/minham/complex_meta/og_metas/metaspades_out/largepan_01.fq/split_input/tmp/interlaced --dst /home/ewissel/minham/complex_meta/og_metas/metaspades_out/largepan_01.fq/split_input

== Splitting /home/ewissel/minham/complex_meta/og_metas/largepan_01/largepan_01.fq into left and right reads (in /home/ewissel/minham/complex_meta/og_metas/metaspades_out/largepan_01.fq/split_input directory)

===== Preprocess interlaced reads finished.

===== Preprocess reads finished.

===== Read error correction started.

===== Read error correction started.

== Running: /home/ewissel/miniconda3/envs/mags/bin/spades-hammer /home/ewissel/minham/complex_meta/og_metas/metaspades_out/largepan_01.fq/corrected/configs/config.info

0:00:00.000 1M / 23M INFO General (main.cpp : 75) Starting BayesHammer, built from N/A, git revision N/A 0:00:00.032 1M / 23M INFO General (main.cpp : 76) Loading config from /home/ewissel/minham/complex_meta/og_metas/metaspades_out/largepan_01.fq/corrected/configs/config.info 0:00:00.063 1M / 23M INFO General (main.cpp : 78) Maximum # of threads to use (adjusted due to OMP capabilities): 15 0:00:00.064 1M / 23M INFO General (memory_limit.cpp : 54) Memory limit set to 250 Gb 0:00:00.064 1M / 23M INFO General (main.cpp : 86) Trying to determine PHRED offset 0:00:00.068 1M / 23M INFO General (main.cpp : 92) Determined value is 33 0:00:00.068 1M / 23M INFO General (hammer_tools.cpp : 38) Hamming graph threshold tau=1, k=21, subkmer positions = [ 0 10 ] 0:00:00.068 1M / 23M INFO General (main.cpp : 113) Size of aux. kmer data 24 bytes === ITERATION 0 begins === 0:00:00.068 1M / 23M INFO K-mer Counting (kmer_data.cpp : 283) Estimating k-mer count 0:00:00.204 241M / 256M INFO K-mer Counting (kmer_data.cpp : 288) Processing /home/ewissel/minham/complex_meta/og_metas/metaspades_out/largepan_01.fq/split_input/largepan_01_1.fastq 0:00:11.407 241M / 274M INFO K-mer Counting (kmer_data.cpp : 297) Processed 5000000 reads 0:00:11.420 241M / 274M INFO K-mer Counting (kmer_data.cpp : 288) Processing /home/ewissel/minham/complex_meta/og_metas/metaspades_out/largepan_01.fq/split_input/largepan_01_2.fastq 0:00:24.259 241M / 274M INFO K-mer Counting (kmer_data.cpp : 297) Processed 10000000 reads 0:00:24.272 241M / 274M INFO K-mer Counting (kmer_data.cpp : 302) Total 10000000 reads processed 0:00:25.293 241M / 274M INFO K-mer Counting (kmer_data.cpp : 305) Estimated 259484721 distinct kmers 0:00:25.303 1M / 274M INFO K-mer Counting (kmer_data.cpp : 309) Filtering singleton k-mers 0:00:25.350 653M / 653M INFO K-mer Counting (kmer_data.cpp : 315) Processing /home/ewissel/minham/complex_meta/og_metas/metaspades_out/largepan_01.fq/split_input/largepan_01_1.fastq 0:03:51.031 654M / 851M INFO K-mer Counting (kmer_data.cpp : 324) Processed 5000000 reads 0:03:51.036 653M / 851M INFO K-mer Counting (kmer_data.cpp : 315) Processing /home/ewissel/minham/complex_meta/og_metas/metaspades_out/largepan_01.fq/split_input/largepan_01_2.fastq

params.txt

Command line: /home/ewissel/miniconda3/envs/mags/bin/metaspades.py --12 /home/ewissel/minham/complex_meta/og_metas/largepan_01/largepan_01.fq -o /home/ewissel/minham/complex_meta/og_metas/metaspades_out/largepan_01.fq -t 15

System information: SPAdes version: 3.15.4 Python version: 3.8.13 OS: Linux-4.19.0-14-amd64-x86_64-with-glibc2.10

Output dir: /home/ewissel/minham/complex_meta/og_metas/metaspades_out/largepan_01.fq Mode: read error correction and assembling Debug mode is turned OFF

Dataset parameters: Metagenomic mode Reads: Library number: 1, library type: paired-end orientation: fr left reads: not specified right reads: not specified interlaced reads: ['/home/ewissel/minham/complex_meta/og_metas/largepan_01/largepan_01.fq'] single reads: not specified merged reads: not specified Read error correction parameters: Iterations: 1 PHRED offset will be auto-detected Corrected reads will be compressed Assembly parameters: k: [21, 33, 55] Repeat resolution is enabled Mismatch careful mode is turned OFF MismatchCorrector will be SKIPPED Coverage cutoff is turned OFF Other parameters: Dir for temp files: /home/ewissel/minham/complex_meta/og_metas/metaspades_out/largepan_01.fq/tmp Threads: 15 Memory limit (in Gb): 250

SPAdes version

v3.15.4

Operating System

Linux Debian 4.19.0-14-amd64

Python Version

3.8.13

Method of SPAdes installation

conda

No errors reported in spades.log

• [X] Yes

[asl]

Hello

You SPAdes job is still running. Maybe yet another incarnation of https://github.com/ablab/spades/issues/152

[ewissel]

When I look at the jobs running on the server, this metaspades job is no longer running. I accidentally recreated this with 100 samples (my check at the end of jobs only checked for errors, not a "complete" statement) and those jobs are also no longer running.

So the way forward is to run with --only-assembler until a new version of spades?

[asl]

Well, likely someone is killing your metaSPAdes jobs then. I would suggest you to ask your system administrator for this. You can give --only-assembler a try.

[wu1001]

Hello, I would like to ask if you finally solved the problem? I also encountered a similar problem