Too high memory consumption during disjointig deduplication

[diego-rt]

Hello,

I'm trying to assemble a giant 29 Gb genome using flye and approximately 18x coverage of Nanopore UL reads. While disjointigs were assembled successfully, the subsequent step failed because of insufficient memory.

Unfortunately this is our highest memory node with ~1920 GB and I was wondering whether you have any recommendations for parameters that could decrease memory consumption? Is reducing the number of threads my best bet?

flye.log

Thanks a lot!

[mikolmogorov]

Hello,

Unfortunately it ran out of memory on the very last deduplication step of the assembly. Usually it does not cause problems with memory, but seems like this genome is different.

One way to go around may be to reduce the number of threads just for the disjointig step. Alternatively, if you are able to compile from source, you can comment out this line and recompile: https://github.com/fenderglass/Flye/blob/4fc23c4f53c384468735ea2443ce8df465e02f63/src/assemble/main_assemble.cpp#L319

Let me know if either solution works. In the next release, I will add an option to reduce the number of threads / disable the deduplication stage from a config file.

[diego-rt]

Hey there,

Thanks a lot for the potential solutions! I commented it out and will see if it makes any difference but will likely take a few days to get to that point again.