HiFiasm out of memory

[charlottellcc]

Hello,

I have a memory problem when trying to assemble my heterozygous plant genome. I managed to get the haplotypes with 220G of memory and 32 CPUs but I can't produce the primary/alternative assembly. I have 115 Gb of corrected data (< Q20) as input. The quality of the data seems to be correct and the distribution of the Kmers also :

I tried the -f39 option to save memory but it didn't work. I did not find a solution in the issues that were listed in the FAQ.

Thanks for your help,

Ch

[chhylp123]

Could you please show the log of hifiasm?

[chhylp123]

Sorry I cannot find it...

[charlottellcc]

runHiFiasm_j31288317.log

[chhylp123]

Thanks a lot. I guess it is just caused by too much data, and hifiasm should work with a little bit more RAM.

[charlottellcc]

Indeed, an assembly with less data works (I have already tried), but we would like to use all the data, if possible. I will ask about requesting more memory on our cluster.

Thanks for your help,

Charlotte

[fengli-eGen]

Hi Charlotte,

I may encounter a similar issue. But I wonder when you say "115 Gb of corrected data (< Q20)", did you refer to Pacbio reads.bam from ccs? https://ccs.how/faq/reads-bam

[charlottellcc]

Hi Fengli,

Indeed, I use ccs reads from PacBio or I make a correction on raw reads (sequencing output without correction) and I correct with the ccs script from SMRTLink.

Charlotte

[fengli-eGen]

Cool! Thank you for replying, Charlotte. I wonder when you ran ccs, did you run it with min_passes>=3 and min_quality (Q) 20 (HiFi standards)? I'm trying to run CLR ccs (not raw reads) using hifiasm, since my data don't have enough coverage for HiFi standards. Thanks!

[charlottellcc]

Hi,

Yes I used the PacBio recommendations by correcting my data in 3 passes (=reads hifi > Q20). I have never used CLR reads with HiFiasm but I am not sure if it works: See Issue "Can hifiasm take CLR and ccs.bam?"

Good luck,

Charlotte

[fengli-eGen]

Thanks Charlotte!

[Jokendo-collab]

I also have 160GB data and I have 232GB RAM but it is still failing. Is there a trick around the solve this problem?

[lh3]

@Jokendo-collab No. You need a machine with 512GB RAM, or with less data but that will make assembly worse.