bs3
bs3 copied to clipboard
Large FASTQ's in-core, can exhaust large system RAM
I have a server with 256GB of RAM, and two ~127GB FASTQ files. It seems that once you start creating Trimed files, they're entirely in-core, and eventually exhaust system memory.
There's also sort of an unbound limit to the number of snap processes you'll spawn, and snap (even with this -map flag) will devour 64GB+ per hg38 index load.
I can of course get access to larger servers, but you might want to issue a warning of some kind, or implement out of core resource access.
Thanks for your work here!