mosdepth
mosdepth copied to clipboard
brentp
Out of memory Issue
Try to calculate the depth on a BAM file around 1Gb, got a prompt: out of memory.
The command I performed is: mosdepth -Q 10 prefix <input_bam>
Here is the error log:
As shown above, no other error message. Just a simple statement: "Out of memory"
Hi, mosdepth will use an amount of memory that is about: 4 bytes * length-of-longest-chromosome. So, on human, where the largest chromosome is about 250 megabases, it will use about 1 GB of memory.
If you are using a worfklow manager, I'd recommend to give it double that amount.
Hi, mosdepth will use an amount of memory that is about:
4 bytes * length-of-longest-chromosome. So, on human, where the largest chromosome is about 250 megabases, it will use about 1 GB of memory. If you are using a worfklow manager, I'd recommend to give it double that amount.
Hi, thank you for the prompt reply! Appreciated!
The case is, I'm using GNU parallel to run 60 jobs in parallel, each job contains a step of mosdepth processing. The parallel command and the 60 jobs are running under a PBS interactive environment, which has been scheduled with 80 threads and 750 GB RAM. Given that there is no way 60 jobs running mosdepth will take up to near 700 GB, and parallel itself does not set a limit on mem eaten. I'm not sure which command/tool limit the memory usage.
P.S. Just in case, PBS is a job scheduler for HPC. We are using the PBS Pro commercial version as far as I know. Here should be the official website: https://www.altair.com/pbs-works/
Hi, this is likely not a mosdepth issue. The memory used by mosdepth is fixed. So it's something with your scheduler (PBS) or parallelizer (gnu parallel) and the way you're using them together.
Hi, this is likely not a mosdepth issue. The memory used by mosdepth is fixed. So it's something with your scheduler (PBS) or parallelizer (gnu parallel) and the way you're using them together.
I thought the same at the beginning. Then I googled about this error message and some posts says it may come from htslib. Is mosdepth using htslib?