spades
spades copied to clipboard
ablab
Stuck and segmentation fault
hi,@asl
my command is
'metaspades.py -1 SRR9033753.rmhost.clean.1.fq.gz -2 SRR9033753.rmhost.clean.2.fq.gz -t 20 -o output'
when I run it by SPAdes-3.14.1, the task stuck when Processing reverse paired-end reads.
0:00:39.478 2G / 2G INFO K-mer Counting (kmer_data.cpp : 312) Processing ./SRR9033753.rmhost.clean.1.fq
0:09:55.001 2G / 2G INFO K-mer Counting (kmer_data.cpp : 321) Processed 10455463 reads
0:09:55.001 2G / 2G INFO K-mer Counting (kmer_data.cpp : 312) Processing ./SRR9033753/SRR9033753.rmhost.clean.2.fq
when I run it by SPAdes-3.15.2, "The program was terminated by segmentation fault" occuring when Processing reverse paired-end reads.
0:00:59.245 1302M / 1302M INFO K-mer Counting (kmer_data.cpp : 315) Processing ./SRR9033753/SRR9033753.rmhost.clean.1.fq.gz
0:08:34.135 1302M / 1491M INFO K-mer Counting (kmer_data.cpp : 324) Processed 10455463 reads
0:08:34.135 1302M / 1491M INFO K-mer Counting (kmer_data.cpp : 315) Processing ./SRR9033753/SRR9033753.rmhost.clean.2.fq.gz
The program was terminated by segmentation fault
=== Stack Trace ===
[0x40b828]
[0x40b956]
thanks!
Will you please attach your params.txt and spades.log files as you were instructed in the error message?
Ok, does the assembly finish if you'd use --only-assembler option in addition to yours?
Yes, it works! Plus --only-assemble option can accomplish assembly.
So, does the 'read error correction' process introduce conflicts to the pipeline?
No, there are no "conflicts". Likely the problem will be solve if you'd compile SPAdes for the host machine instead of using pre-built binaries.
Hey @asl , not quite the same for me here. Also weirder because while SPAdes runs normally for one of my samples, for the other (separate run) it hangs at K-mer Counting (kmer_data.cpp). This is occurring for both a binary and a compiled version of SPAdes and the sample runs fully with --only-assemble. All this running version 3.15.2.
What information can I provide to get some help with this?
Hey @asl,
I am also running into the same issue as well. Is there any update on this problem? One sample sample out of a larger set is also failing.
I am running metaspades in a docker image I built that is using the pre-compiled binaries.
Here is the content of the log file:
Command line: /opt2/SPAdes-3.15.4-Linux/bin/metaspades.py -t 32 -m 64 -1 /gs1/RTS/NextGen/ActiveProjects/NGS_Munster_CustomSeqCap/Kwe_Hendra_PROJECTS/Final_HEV/Metavirs/outs_test/17249_noHg38/trim/17249_noHg38.R1.trim.host_removed.fastq.gz -/gs1/RTS/NextGen/ActiveProjects/NGS_Munster_CustomSeqCap/Kwe_Hendra_PROJECTS/Final_HEV/Metavirs/outs_test/17249_noHg38/trim/17249_noHg38.R2.trim.host_removed.fastq.gz -o /gs1/RTS/NextGen/ActiveProjects/NGS_Munster_CustomSeqCap/Kwe_Hendra_PROJECTS/Final_HEV/Metavirs/outs_test/17249_noHg38/metaspades
System information:
SPAdes version: 3.15.4
Python version: 3.8.10
OS: Linux-3.10.0-1160.66.1.el7.x86_64-x86_64-with-glibc2.29
Output dir: /gs1/RTS/NextGen/ActiveProjects/NGS_Munster_CustomSeqCap/Kwe_Hendra_PROJECTS/Final_HEV/Metavirs/outs_test/17249_noHg38/metaspades
Mode: read error correction and assembling
Debug mode is turned OFF
Dataset parameters:
Metagenomic mode
Reads:
Library number: 1, library type: paired-end
orientation: fr
left reads: ['/gs1/RTS/NextGen/ActiveProjects/NGS_Munster_CustomSeqCap/Kwe_Hendra_PROJECTS/Final_HEV/Metavirs/outs_test/17249_noHg38/trim/17249_noHg38.R1.trim.host_removed.fastq.gz']
right reads: ['/gs1/RTS/NextGen/ActiveProjects/NGS_Munster_CustomSeqCap/Kwe_Hendra_PROJECTS/Final_HEV/Metavirs/outs_test/17249_noHg38/trim/17249_noHg38.R2.trim.host_removed.fastq.gz']
interlaced reads: not specified
single reads: not specified
merged reads: not specified
Read error correction parameters:
Iterations: 1
PHRED offset will be auto-detected
Corrected reads will be compressed
Assembly parameters:
k: [21, 33, 55]
Repeat resolution is enabled
Mismatch careful mode is turned OFF
MismatchCorrector will be SKIPPED
Coverage cutoff is turned OFF
Other parameters:
Dir for temp files: /gs1/RTS/NextGen/ActiveProjects/NGS_Munster_CustomSeqCap/Kwe_Hendra_PROJECTS/Final_HEV/Metavirs/outs_test/17249_noHg38/metaspades/tmp
Threads: 32
Memory limit (in Gb): 64
======= SPAdes pipeline started. Log can be found here: /gs1/RTS/NextGen/ActiveProjects/NGS_Munster_CustomSeqCap/Kwe_Hendra_PROJECTS/Final_HEV/Metavirs/outs_test/17249_noHg38/metaspades/spades.log
/gs1/RTS/NextGen/ActiveProjects/NGS_Munster_CustomSeqCap/Kwe_Hendra_PROJECTS/Final_HEV/Metavirs/outs_test/17249_noHg38/trim/17249_noHg38.R1.trim.host_removed.fastq.gz: max reads length: 151
/gs1/RTS/NextGen/ActiveProjects/NGS_Munster_CustomSeqCap/Kwe_Hendra_PROJECTS/Final_HEV/Metavirs/outs_test/17249_noHg38/trim/17249_noHg38.R2.trim.host_removed.fastq.gz: max reads length: 151
Reads length: 151
===== Before start started.
===== Read error correction started.
===== Read error correction started.
== Running: /opt2/SPAdes-3.15.4-Linux/bin/spades-hammer /gs1/RTS/NextGen/ActiveProjects/NGS_Munster_CustomSeqCap/Kwe_Hendra_PROJECTS/Final_HEV/Metavirs/outs_test/17249_noHg38/metaspades/corrected/configs/config.info
0:00:00.000 1M / 12M INFO General (main.cpp : 75) Starting BayesHammer, built from refs/heads/spades_3.15.4, git revision 1a5332e5608a073a9d28a6d0e30836f38dcc5613
0:00:00.016 1M / 12M INFO General (main.cpp : 76) Loading config from /gs1/RTS/NextGen/ActiveProjects/NGS_Munster_CustomSeqCap/Kwe_Hendra_PROJECTS/Final_HEV/Metavirs/outs_test/17249_noHg38/metaspades/corrected/configs/config.info
0:00:00.029 1M / 12M INFO General (main.cpp : 78) Maximum # of threads to use (adjusted due to OMP capabilities): 32
0:00:00.046 1M / 12M INFO General (memory_limit.cpp : 54) Memory limit set to 64 Gb
0:00:00.058 1M / 12M INFO General (main.cpp : 86) Trying to determine PHRED offset
0:00:00.064 1M / 12M INFO General (main.cpp : 92) Determined value is 33
0:00:00.076 1M / 12M INFO General (hammer_tools.cpp : 38) Hamming graph threshold tau=1, k=21, subkmer positions = [ 0 10 ]
0:00:00.088 1M / 12M INFO General (main.cpp : 113) Size of aux. kmer data 24 bytes
=== ITERATION 0 begins ===
0:00:00.101 1M / 12M INFO K-mer Counting (kmer_data.cpp : 283) Estimating k-mer count
0:00:00.168 513M / 581M INFO K-mer Counting (kmer_data.cpp : 288) Processing /gs1/RTS/NextGen/ActiveProjects/NGS_Munster_CustomSeqCap/Kwe_Hendra_PROJECTS/Final_HEV/Metavirs/outs_test/17249_noHg38/trim/17249_noHg38.R1.trim.host_removed.fastq.gz
0:00:02.472 513M / 637M INFO K-mer Counting (kmer_data.cpp : 297) Processed 612152 reads
0:00:02.484 513M / 637M INFO K-mer Counting (kmer_data.cpp : 288) Processing /gs1/RTS/NextGen/ActiveProjects/NGS_Munster_CustomSeqCap/Kwe_Hendra_PROJECTS/Final_HEV/Metavirs/outs_test/17249_noHg38/trim/17249_noHg38.R2.trim.host_removed.fastq.gz
0:00:04.768 513M / 637M INFO K-mer Counting (kmer_data.cpp : 297) Processed 1224304 reads
0:00:04.781 513M / 637M INFO K-mer Counting (kmer_data.cpp : 302) Total 1224304 reads processed
0:00:05.930 513M / 637M INFO K-mer Counting (kmer_data.cpp : 305) Estimated 30674495 distinct kmers
0:00:05.943 1M / 637M INFO K-mer Counting (kmer_data.cpp : 309) Filtering singleton k-mers
0:00:05.949 85M / 637M INFO K-mer Counting (kmer_data.cpp : 315) Processing /gs1/RTS/NextGen/ActiveProjects/NGS_Munster_CustomSeqCap/Kwe_Hendra_PROJECTS/Final_HEV/Metavirs/outs_test/17249_noHg38/trim/17249_noHg38.R1.trim.host_removed.fastq.gz
0:00:20.264 85M / 637M INFO K-mer Counting (kmer_data.cpp : 324) Processed 612152 reads
0:00:20.265 85M / 637M INFO K-mer Counting (kmer_data.cpp : 315) Processing /gs1/RTS/NextGen/ActiveProjects/NGS_Munster_CustomSeqCap/Kwe_Hendra_PROJECTS/Final_HEV/Metavirs/outs_test/17249_noHg38/trim/17249_noHg38.R2.trim.host_removed.fastq.gz
The program was terminated by segmentation fault
=== Stack Trace ===
The program was terminated by segmentation fault
=== Stack Trace ===
The program was terminated by segmentation fault
=== Stack Trace ===
The program was terminated by segmentation fault
The program was terminated by segmentation fault
=== Stack Trace ===
=== Stack Trace ===
/opt2/SPAdes-3.15.4-Linux/bin/spades-hammer() [0x421e29]
/opt2/SPAdes-3.15.4-Linux/bin/spades-hammer() [0x423956]
/lib/x86_64-linux-gnu/libc.so.6(+0x430c0) [0x7fb65a91b0c0]
/opt2/SPAdes-3.15.4-Linux/bin/spades-hammer() [0x4a40db]
/opt2/SPAdes-3.15.4-Linux/bin/spades-hammer() [0x44f6ca]
/opt2/SPAdes-3.15.4-Linux/bin/spades-hammer() [0x5c93ee]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7fb65aad2609]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x43) [0x7fb65a9f7163]
/opt2/SPAdes-3.15.4-Linux/bin/spades-hammer() [0x421e29]
/opt2/SPAdes-3.15.4-Linux/bin/spades-hammer() [0x423956]
/lib/x86_64-linux-gnu/libc.so.6(+0x430c0) [0x7fb65a91b0c0]
/opt2/SPAdes-3.15.4-Linux/bin/spades-hammer() [0x4a40db]
/opt2/SPAdes-3.15.4-Linux/bin/spades-hammer() [0x44f6ca]
/opt2/SPAdes-3.15.4-Linux/bin/spades-hammer() [0x5c93ee]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x7fb65aad2609]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x43) [0x7fb65a9f7163]
The program was terminated by segmentation fault
=== Stack Trace ===
== Error == system call for: "['/opt2/SPAdes-3.15.4-Linux/bin/spades-hammer', '/gs1/RTS/NextGen/ActiveProjects/NGS_Munster_CustomSeqCap/Kwe_Hendra_PROJECTS/Final_HEV/Metavirs/outs_test/17249_noHg38/metaspades/corrected/configs/config.info']" finished abnormally, OS return value: 1
None
In case you have troubles running SPAdes, you can write to [email protected]
or report an issue on our GitHub repository github.com/ablab/spades
Please provide us with params.txt and spades.log files from the output directory.
SPAdes log can be found here: /gs1/RTS/NextGen/ActiveProjects/NGS_Munster_CustomSeqCap/Kwe_Hendra_PROJECTS/Final_HEV/Metavirs/outs_test/17249_noHg38/metaspades/spades.log
I also have this problem! Error correction runs perfectly with 11/12 samples but for some reason, it hangs on the one specific sample, namely R2 of the sample's paired gzipped FastQs. I'm running v3.14.1 installed from conda.
We have the same problem with metaSpades here, where some (but not all) our larger samples output the following error :
1:54:42.987 2599M / 2819M INFO K-mer Counting (kmer_data.cpp : 315) Processing /out/GQ39/GQ39_paired_sorted_2.fastq
The program was terminated by segmentation fault
=== Stack Trace ===
/miniconda3/envs/metawrap-env/bin/spades-hammer(+0x1ddac) [0x55bc7323adac]
/miniconda3/envs/metawrap-env/bin/spades-hammer(+0x1e2ac) [0x55bc7323b2ac]
/lib/x86_64-linux-gnu/libc.so.6(+0x43090) [0x2b3bc87af090]
/miniconda3/envs/metawrap-env/bin/spades-hammer(+0x9a719) [0x55bc732b7719]
/miniconda3/envs/metawrap-env/bin/spades-hammer(+0x5210b) [0x55bc7326f10b]
/miniconda3/envs/metawrap-env/bin/spades-hammer(+0x5224e) [0x55bc7326f24e]
/miniconda3/envs/metawrap-env/bin/../lib/libgomp.so.1(+0x177f0) [0x2b3bc870c7f0]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8609) [0x2b3bc8751609]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x43) [0x2b3bc888b133]
== Error == system call for: "['/miniconda3/envs/metawrap-env/bin/spades-hammer', '/out/assembly/metaspades/corrected/configs/config.info']" finished abnormally, OS return value: 1
None
Any idea what might be happening?
System information:
SPAdes version: 3.15.4
Python version: 2.7.15
OS: Linux-3.10.0-1160.62.1.el7.x86_64-x86_64-with-debian-bullseye-sid
Dataset parameters:
Metagenomic mode
Reads:
Library number: 1, library type: paired-end
orientation: fr
left reads: ['/out/GQ39/GQ39_paired_sorted_1.fastq']
right reads: ['/out/GQ39/GQ39_paired_sorted_2.fastq']
interlaced reads: not specified
single reads: not specified
merged reads: not specified
Read error correction parameters:
Iterations: 1
PHRED offset will be auto-detected
Corrected reads will be compressed
Assembly parameters:
k: [21, 33, 55]
Repeat resolution is enabled
Mismatch careful mode is turned OFF
MismatchCorrector will be SKIPPED
Coverage cutoff is turned OFF
Other parameters:
Dir for temp files: /out/assembly/metaspades.tmp
Threads: 48
Memory limit (in Gb): 251
