pydra icon indicating copy to clipboard operation
pydra copied to clipboard

workflow output naming

Open djarecka opened this issue 4 years ago • 2 comments

Based on the discussion with @jw-96, @effigies

Right now all of the workflow output files are copied to the workflow directory without keeping any nested directory structure. But if a task creates a file with the same name regardless of the input, we could copy multiple files with the same name to the workflow directory.

We could have some renaming option (like DataSink) or add some structure to the workflow directory - e.g. <wfdir>/<outname>[.<statehash>]/out.nii instead of <wfdir>/out.nii. Comments are welcome

djarecka avatar Apr 22 '21 15:04 djarecka

i don't know if state comes into play at all, but you could use <wfdir>/taskname/files* this would allow non-replication of files being sent to multiple outputs.

note that a a task could produce similar named files in it's directory:

<taskdir>/path1/file1
<taskdir>/path2/file1

if these were also both outputs of workflow you could consider:

<wfdir>/<taskname>/path1/file1
<wfdir>/<taskname>/path2/file1

maintaining the relative path within the task directory.

satra avatar Apr 22 '21 15:04 satra

the state is important, since you can't use simply the workflow output name.

I thought we wanted to avoid possibly very nested directory structure, but we can always do it

djarecka avatar Apr 22 '21 23:04 djarecka