rnaseq-pipeline
rnaseq-pipeline copied to clipboard
Minimize usage of network between DownloadSample and AlignSample
In the current setup, it's possible that the AlignSample task get scheduled on a different node than the matching DownloadSample task which disrupt the NFS caching.
We could first start by pinning align jobs on whichever node was used to download the data. If that works well, we would then switch to use the local scratch filesystem to completely avoid the NFS.