Dynamically set the input arguments of a FunctionTask

[omar-rifai]

Hello,

I was wondering if there was a way to set input arguments of a Function task at runtime. I have tried the following but I get a KeyError at the last line when trying to set a input argument.

# initialize workflow
wf = Workflow("test_wf", input_spec=['none'])

@pydra.mark.task
@pydra.mark.annotate({"return": {"sum": int}})
def sum_list(**kwargs):
    sum = 0
    for i, v in kwargs.items():
        sum = sum + v
    return sum

wf.add(sum_list(name="sum_task"))
wf.sum_task.inputs.x = 2 

I also tried to "dynamically" construct the input_spec but to no avail. It is not clear from the specs if this is possible.

Thanks !

[effigies]

I think we considered permitting this, but couldn't come up with a use case to justify the effort.

Is this a toy example or is there a problem you're trying to solve this way?

[omar-rifai]

The use case I'm trying to solve is to write a generic output task that takes an arbitrary set of input parameters and writes them to a directory. A sort of dynamic datasink.

I'm not sure if there is a way around this.

[ghisvail]

Is this a toy example or is there a problem you're trying to solve this way?

For instance, how would we implement something like BIDSDataGrabber as a pydra native interface?

As a reminder for others, the challenge is that the output_spec is not known a priori and needs to be computed on construction based on the instance parameters. The same would happen for a generic BIDSDataWriter on the input_spec.

Right now, FunctionTask expects the wrapped function, input_spec and output_spec to be fully specified at construct time. AFAIU, dynamic input or output specifications could only be implemented in a generator function acting as a factory of FunctionTask.

Do you see an alternative to this approach?

[djarecka]

After talking to @ghisvail , I understand that this doesn't have to be set in runtime, but the input can be identify before the workflow is run. I think we could add an option to the wrapper for identifying kwargs argument that will be used by the task.

[omar-rifai]

Hello @djarecka, You are right, when mentioning runtime I was thinking about the most generic use case but having an option of identifying kwargs arguments would already allow us to handle most cases and would be great !