predicting peak memory of a large run

Open PeterEmmrich opened this issue 5 years ago • 1 comments

Hi Ivan! I am trying to polish a 6.2 Gbp assembly using racon 1.3.1. I would like to feed it 301 Gb of Fastq long reads (PromethION) and a 15 Gb PAF file (without CIGAR).

I tried this with 32 cores and 200G memory, but it crashed due to memory capping out after about two days.

I can request more memory, but I would like to get it right. Could you advise on how much memory usage and wall-clock time you'd expect this to take and how many cores I should use?

Many thanks and I hope you are staying safe. Peter

Mar 21 '20 04:03 PeterEmmrich

Hi Peter, the memory is bound by the cumulative size of all three files needed as input plus some epsilon. For your case, I would say that 500Gb of RAM should be more than enough. Regarding the execution time, not sure how long it will take :/

Best regards, Robert

P.S. I would advise you to upgrade to the latest Racon version (1.4.12) at https://github.com/lbcb-sci/racon.

Mar 21 '20 11:03 rvaser