pydra icon indicating copy to clipboard operation
pydra copied to clipboard

defaults splitter in a workflow

Open djarecka opened this issue 4 years ago • 5 comments

what should be a defaults splitter for wf.mult:

wf = Workflow(name="wf_ndst_9", input_spec=["x"])
wf.add(add2(name="add2x", x=wf.lzin.x).split("x"))
wf.add(add2(name="add2y", x=wf.add2x.lzout.out))
wf.add(
    multiply(name="mult", x=wf.add2x.lzout.out, y=wf.add2y.lzout.out)
 )
wf.inputs.x = [1, 2]

If the splitter is set to ["_add2x", "_add2y"] should we really produce four elements?

djarecka avatar Sep 29 '21 18:09 djarecka

isn't that what setting that splitter means? however, it's been so long since we discussed all the options!

satra avatar Sep 29 '21 21:09 satra

and this should be the default? note that _add2y has the same state as add2x

djarecka avatar Sep 29 '21 22:09 djarecka

perhaps consider how it works in other use cases:

  • NA, NB with splits feeding to NC (this would have different state)
  • NA with split and multiple outputs feeding to NB (this would have same state - i think this is more analogous to your usage, unless someone explicitly wants to create a nested splitter)

satra avatar Sep 29 '21 23:09 satra

@satra - could you see if the new test make sense to you: https://github.com/nipype/pydra/pull/499/commits/5d685d2d24a09c641f2c7cd350234dab5ecafce5#diff-df3bf50d3866111b6eb1fefa205157ccb4017589f52860a3fb4c92a11aef39e1R1505

Previously I was not allowing for "repeating" a state

djarecka avatar Oct 05 '21 03:10 djarecka

I've decided to stop working on the changes for now, but unfinished work is in my branch: https://github.com/djarecka/pydra/tree/rf/state_repeated_notes

I think I actually don't like the idea of repeating states in a splitter. Even if we decide to allow it, it would require more changes to be consistent with everything (including combiner that would have to be forbidden in case of multiple appearance of the same state in the splitter?)

djarecka avatar Mar 20 '22 20:03 djarecka