bash command as a Pipeline step or specify execution path of a task within a Pipeline

[Waerden001]

Is there a way to specify the execution path of a task within a Pipeline, or use bash command as a Pipeline step?

I construct a pipeline with existing tasks, when running the pipeline in local mode with pipe.start_locally(run_pipeline_steps_locally=True), it seems all the steps/tasks are cloned and executed in the default root directory(i.e. /home/user/ on Linux), since I have some relative path settings(hydra.main())in the step scripts, the default behavior of ClearML makes it invalid anymore. I've tried to

• set_script for each step/task

• use Task.create to instantiate a task with local repo url and entry point

But neither method works. Basically a step/task of mine should be executed as a bash command, something like python /path/to/script.py, but in ClearML within a Pipeline, this step/task--seems to me--is executed as python /home/user/script.py , so is there a way to either

• set the execution path of a task/step within a Pipeline or

• use a shell command as a Pipeline task/step similar to some workflow management tools like Airflow and Prefect

[jkhenning]

Hi @Waerden001,

Sorry for the late response 🙏

Are you referring to tasks or functions? If this is tasks, it is on the Task; If this is a function, shouldn't it be the CWD of the git repository?