pigx icon indicating copy to clipboard operation
pigx copied to clipboard

Speed up the dag building process

Open alexg9010 opened this issue 6 years ago • 0 comments

For the ChIPseq pipeline I had the feeling that the dag building took way too long for the dry-run (in total around 250 jobs), so I started searching a bit around and found this to be not only the case for me as it seems to be a general issue for larger number of jobs (https://bitbucket.org/snakemake/snakemake/issues/534/dryrun-takes-a-long-time-for-complex). One thing that I found that we (as developers) could improve is to use less wildcards and rather reuse the outputs of dependent rules (https://bitbucket.org/snakemake/snakemake/issues/745/building-dag-of-jobs-slower).

This is something that definitely is not urgent, but we might want to change this at some point.

alexg9010 avatar Mar 27 '18 15:03 alexg9010