methylKit icon indicating copy to clipboard operation
methylKit copied to clipboard

methread consumes a lot memory

Open alexg9010 opened this issue 6 years ago • 0 comments

Right now I saw that it used 2x the size of the input file in memory. This is fine for 40 million CpG sites, but can be quite large for 800 million CHH or 200 million CHG sites.

We are loading the whole file into memory anyways, but maybe we could limit copying during some of our processing functions.

For now this should just go into documentation that for non-cpg sites a lot more ressources are required.

alexg9010 avatar Sep 25 '19 10:09 alexg9010