pydra
pydra copied to clipboard
"IsADirectoryError" only occurs when using pydra for the same task
Hi all,
I am currently trying to build some workflows with pydra. They should include functions of a library, which we have developed. one step of the workflow would be reading in the config file. Consequently, I tried to set up a really simple workflow for doing that. However, when executing the workflow I get error the following error message:
"name": "IsADirectoryError",
"message": "[Errno 21] Is a directory: '/data2/egapontseva/MEG_QC_stuff/data/from_openneuro/ds003483'",
The workflow looks as follows:
import nest_asyncio
nest_asyncio.apply()
import pydra
import meg_qc
from meg_qc.meg_qc_pipeline import make_derivative_meg_qc
from meg_qc.source.initial_meg_qc import get_all_config_params, Epoch_meg
# Define the task
@pydra.mark.task
def get_config_params(configfile):
config_dict = get_all_config_params(configfile)
return config_dict
# Define the Pydra workflow
workflow_name = "initial_processing"
input_spec = ["configfile"]
initial_processing_wf = pydra.Workflow(name=workflow_name, input_spec=input_spec)
initial_processing_wf.add(get_config_params(name='read_config', configfile=path_to_config))
# Specify the output of the workflow
initial_processing_wf.set_output([
("config_dict", initial_processing_wf.read_config.lzout.out)
])
# Submit the workflow for execution using the 'cf' plugin
with pydra.Submitter(plugin='cf') as sub:
sub(initial_processing_wf)
# View the output of the workflow
print(initial_processing_wf.result())
The problem is that when just running the function isolated like this: config_dict = get_all_config_params(path_to_config) config_dict it works perfectly and does not produce any error.
I would appreciate your help, since error does not really make sense to me. The directory_path in the dictionary is actually intended to be a path to a directory and that is what produces the error, in the case of running it with pydra, to my understanding.
I am using pydra 0.22.