Cmdline from lzout

[chasejohnson3]

I would like to know the cmdline calls being used for a pipeline. However, the pipeline relies on lzout values. Is there a way to get the cmdline calls for tasks that use lzout values as input?

For example, the following does not print the cmdline for shelly because args for shelly is set to wf.append_filename.lzout.out.

cmd = "echo"
filename = "newfile_1.txt"

@pydra.mark.task
def append_filename(filename):
    return filename + "_appended"

wf = pydra.Workflow(name="wf", input_spec=["filename"], filename=filename)

my_output_spec = pydra.specs.SpecInfo(
    name="Output",
    fields=[
        (
            "out",
            attr.ib(
                type=pydra.specs.File,
                metadata={
                    "output_file_template": "{args}",
                    "help_string": "output file",
                },
            ),
        )
    ],
    bases=(pydra.specs.ShellOutSpec,),
)

wf.add(append_filename(name="append_filename", filename=wf.lzin.filename))
shelly = pydra.ShellCommandTask(name="shelly", executable=cmd, args=wf.append_filename.lzout.out, output_spec=my_output_spec)
wf.add(shelly)
wf.set_output([("out", wf.shelly.lzout.out)])

print("cmndline = ", shelly.cmdline)

with pydra.Submitter(plugin="cf") as sub:
    sub(wf)
wf.result()

The following error is returned:

Exception                                 Traceback (most recent call last)
<ipython-input-41-7c7d59e5238a> in <module>
     30 wf.set_output([("out", wf.shelly.lzout.out)])
     31 
---> 32 print("cmndline = ", shelly.cmdline)
     33 
     34 with pydra.Submitter(plugin="cf") as sub:

/mnt/c/2020_Grad_School/Research/pydra/pydra/engine/task.py in cmdline(self)
    492         """
    493         if is_lazy(self.inputs):
--> 494             raise Exception("can't return cmdline, self.inputs has LazyFields")
    495         # checking the inputs fields before returning the command line
    496         self.inputs.check_fields_input_spec()

Exception: can't return cmdline, self.inputs has LazyFields

[effigies]

It might be interesting to create another property like:

>>> wf.shelly.cmdline_fstring
'echo {wf.append_filename.lzout.out}'

But the actual values can't be known until they're calculated by running the workflow.

[djarecka]

yes, we could return fstring format for the situation when lazy output is not calculated yet. perhaps we can even do it for cmdline property, just return the fstring if there are any lazy fields

[chasejohnson3]

@effigies @djarecka As I understand it, the cmdline property cannot be printed for lazy output because it has not yet been calculated yet.

However, once the lazy output has been calculated after the workflow has run through the Submitter, is it possible to log all the cmdline calls made for the workflow? This would be useful to debug/document a large pipeline.