Segmentation fault when use minimap2 to map pacbio data

[anhong11]

Hi, I use minimap2(2.17-r974-dirty or 2.17-r941) to map pacbio data to fasta. It works well at the beginning, and then got this error as below: [M::worker_pipeline::8353.332*54.76] mapped 18539 sequences /var/spool/slurmd/job20248750/slurm_script: line 4: 53834 Segmentation fault minimap2 -x map-pb -t 56 -a -p 0.5 -N 10 --sam-hit-only -L -Q --secondary=yes 10-consensus/chunks.fasta a.subreads.fa -x map-pb -t 56 > a.sam

I tried several times, the number “53834” may change.

Thanks, Hong

[lh3]

What's your command line? How many sequences are there in the query file?

[anhong11]

My command line is: minimap2 -x map-pb -t 56 -a -p 0.5 -N 10 --sam-hit-only -L -Q --secondary=yes 10-consensus/chunks.fasta a.subreads.fa -x map-pb -t 56 > a.sam

and the query file has : Sequences: 6900494 Tatal: 179822394061 Max: 299247 Min: 10000 N25: 41111 L25: 888865 N50: 30844 L50: 2157127 N75: 21053 L75: 3896880

[lh3]

Why do you use two "-x" and "-t"? What if you remove the duplicate?

[anhong11]

I have not realized that I duplicate these two parameters. I retry it now, and will let you know what happens.

[anhong11]

Unfortunately, I got the same error: [M::worker_pipeline::10317.089*54.93] mapped 19683 sequences /var/spool/slurmd/job20256118/slurm_script: line 4: 12914 Segmentation fault minimap2 -x map-pb -t 56 -a -p 0.5 -N 10 --sam-hit-only -L -Q --secondary=yes /storage/hpc/group/pireslab/hong_hpc/genome_sequence_hpc/pacbio/flye_r3045_cov50/10-consensus/chunks.fasta /home/hongan/htc/hongan_htc/genome_sequencing_storage/pacbio/subreads_fasta/r_3045.subreads_l10k.fa > a.sam

[lh3]

I think it is caused by something else such as memory or slurm scheduling. The hint is that you are getting segfault at different places:

[M::worker_pipeline::8353.332*54.76] mapped 18539 sequences

vs

[M::worker_pipeline::10317.089*54.93] mapped 19683 sequences

Could you run minimap2 on a machine without slurm?

[fengyuanli304]

Hi， I encountered a similar problem and got this error as below: minimap2.sh: line 1: 19034 Segmentation fault (core dumped) /lustre2/weizhang_pkuhpc/lfy/software/minimap2/minimap2 -t 16 -ax map-pb --split-prefix STR /lustre2/weizhang_pkuhpc/lfy/racon/racon1.fasta /lustre2/weizhang_pkuhpc/lfy/merge.fastq.gz > /lustre2/weizhang_pkuhpc/lfy/purge_haplotigs/round_2.sam pkurun: error: c07b02n08: task 0: Exited with exit code 139 However, I have to run minimap2 on a machine with slurm. How can I solve this ? Thank you very much.

[jdmontenegro]

Hi guys,

I also found random segfaults on the same dataset. Sometimes it finishes correctly and other times it will randomly segfault. I am using a cluster with slurm, but I am not really sure how it would affect minimap2 performance. I am asking for a complete 28 cpu node and 96 GB of RAM and not sharing it. Minimap2 is using roughly 40GB of RAM and I am only using 26 cores for the program, so there are still some resources for minimap2 to run correctly. What could be causing this behaviour?

Some details: version: 2.17-r974-dirty cmd line:

minimap2 -ax map-pb -o aligned.sam -t 28 --MD reference.fa subreads.fastq.gz

The first time it died almost immediatly with this error:

[M::mm_idx_gen::53.806*1.76] collected minimizers
[M::mm_idx_gen::61.591*2.13] sorted minimizers
[M::main::61.602*2.13] loaded/built the index for 41254 target sequence(s)
[M::mm_mapopt_update::63.253*2.10] mid_occ = 1258
[M::mm_idx_stat] kmer size: 19; skip: 10; is_hpc: 1; #seq: 41254
[M::mm_idx_stat::64.299*2.08] distinct minimizers: 82952932 (53.15% are singletons); average occurrences: 4.496; average spacing: 7.849; total length: 2927826954
/var/spool/slurm/job4646391/slurm_script: line 43: 25646 Segmentation fault      (core dumped) minimap2 -ax map-pb -o inia601.sam -t 28 --MD ${ref} ${read}

The second time it finished correctly:

[M::worker_pipeline::13962.646*27.05] mapped 39278 sequences
[M::worker_pipeline::14040.802*27.06] mapped 38225 sequences
[M::worker_pipeline::14117.826*27.06] mapped 38750 sequences
[M::worker_pipeline::14194.840*27.06] mapped 38910 sequences
[M::worker_pipeline::14270.335*27.06] mapped 38101 sequences
[M::worker_pipeline::14287.163*27.06] mapped 13533 sequences
[M::main] Version: 2.17-r974-dirty
[M::main] CMD: minimap2 -ax map-pb -o inia601..sam -t 28 --MD reference.fa subreads.fastq.gz
[M::main] Real time: 14287.645 sec; CPU: 386544.371 sec; Peak RSS: 39.938 GB

Any suggestions/comments are more than welcome.

[AG-Run]

Hi

I run next command and I got segfault error after three days of running.

The script run is minimap2 -a -Y -x map-pb --MD -R '@RG\tID:foo\tPL:Pacbio\tSM:bar' -I8g -t 32 reference.mmi CLR_reads.fasta.gz > output.sam

I used -I8g because this is a big genome (larger than 4.5Gb)

Thanks for the help

[zhiyongli1995]

Hi

I also get a 'Segmentation fault' when I run next command.

minimap2 -ax asm5 --eqx $refgenome $qrygenome -o B73-Mo17.sam

The size of genomes approximately 2Gb, my linux is centOS7 with 48 threads and 400+G available memory.

Can any one help me, please!

Thanks for the help.

[Coaxecva]

Hi, got the "segment fault (core dumped)" when aligning PacBio reads to hg38:

$ minimap2 -ax map-pb GRCh38_full_analysis_set_plus_decoy_hla.fa HG03065-clr-r54329U_20191115_214119-B01.fq > HG03065-clr-r54329U_20191115_214119-B01.sam

[M::mm_idx_gen::52.3351.35] collected minimizers [M::mm_idx_gen::62.0061.59] sorted minimizers [M::main::62.0061.59] loaded/built the index for 3366 target sequence(s) [M::mm_mapopt_update::63.9661.57] mid_occ = 488 [M::mm_idx_stat] kmer size: 19; skip: 10; is_HPC: 1; #seq: 3366 [M::mm_idx_stat::64.9381.56] distinct minimizers: 86782898 (44.04% are singletons); average occurrences: 4.479; average spacing: 8.277 [M::worker_pipeline::138.9562.32] mapped 6867 sequences [M::worker_pipeline::212.3632.56] mapped 6944 sequences [M::worker_pipeline::283.7572.68] mapped 6871 sequences [M::worker_pipeline::354.5512.74] mapped 6749 sequences [M::worker_pipeline::425.5582.79] mapped 6897 sequences [M::worker_pipeline::498.426*2.82] mapped 6774 sequences Segmentation fault (core dumped)

Coax