CuratedAtlasQueryR icon indicating copy to clipboard operation
CuratedAtlasQueryR copied to clipboard

memory profile of workflow raw data processing, splitting jobs

Open stemangiola opened this issue 3 years ago • 0 comments

The job submission of the workflow is very inefficient because we are asking for much more memory than needed. It would be nice to profile memory consumption of the jobs launched so to learn how memory to ask for each job based on for example input file size.

stemangiola avatar Sep 18 '22 01:09 stemangiola