toil icon indicating copy to clipboard operation
toil copied to clipboard

Prevent simultaneous pulls of Singularity images by toil-cwl-runner: add locking

Open unito-bot opened this issue 8 months ago • 0 comments

We should make sure that, even if there isn’t a shared filesystem for a cluster, we don’t have several CWL jobs on the same node trying to simultaneously pull the same Singularity image, because this can corrupt Singularity’s internal files.

This is in addition to any prefetching we might want to do when we do have a shared filesystem.

┆Issue is synchronized with this Jira Story ┆Issue Number: TOIL-1603

unito-bot avatar Jun 25 '24 17:06 unito-bot