xqtl-protocol icon indicating copy to clipboard operation
xqtl-protocol copied to clipboard

Improvement for running per gene/traits analysis

Open hsun3163 opened this issue 2 years ago • 1 comments

It is rather common that, for tasks with a lot of subtask(10000+) , sos process will be interrupted due to all sorts of reasons on the cluster, namely kicked out of nodes, not enough mems .etc.

When resuming the jobs with -s build it will always take quite a long time for sos to loop through the existing file to ignore the corresponding subtask. I wonder if is there a way to optimize this behavior, such that we start from the second next directly. If we specify step_2 whose input is the output of step_1, then the sos will consider the input to be empty.

hsun3163 avatar Oct 30 '22 19:10 hsun3163