miniwdl
miniwdl copied to clipboard
mitigate cache invalidation caused by write_* used in workflows
The write_*
functions always generate a unique filename in the current workflow run directory. So if we have two sequential tasks in a workflow, where the second task consumes the output of the first and an intervening write_*
declaration, the second task can never be cached even if the first one is.
Perhaps we can use a content digest just for write_*
files, which shouldn't be very large since they necessarily represent data structures that fit in the runner memory.