xqtl-protocol
xqtl-protocol copied to clipboard
Improvement for running per gene/traits analysis
It is rather common that, for tasks with a lot of subtask(10000+) , sos process will be interrupted due to all sorts of reasons on the cluster, namely kicked out of nodes, not enough mems .etc.
When resuming the jobs with -s build
it will always take quite a long time for sos to loop through the existing file to ignore the corresponding subtask. I wonder if is there a way to optimize this behavior, such that we start from the second next directly. If we specify step_2 whose input is the output of step_1, then the sos will consider the input to be empty.