pigx
pigx copied to clipboard
Speed up the dag building process
For the ChIPseq pipeline I had the feeling that the dag building took way too long for the dry-run (in total around 250 jobs), so I started searching a bit around and found this to be not only the case for me as it seems to be a general issue for larger number of jobs (https://bitbucket.org/snakemake/snakemake/issues/534/dryrun-takes-a-long-time-for-complex). One thing that I found that we (as developers) could improve is to use less wildcards and rather reuse the outputs of dependent rules (https://bitbucket.org/snakemake/snakemake/issues/745/building-dag-of-jobs-slower).
This is something that definitely is not urgent, but we might want to change this at some point.