prefect icon indicating copy to clipboard operation
prefect copied to clipboard

`prefect.deployments.steps.set_working_directory` doesn't work with relative paths

Open NicholasPini opened this issue 6 months ago • 0 comments

Bug summary

I have a deployment that runs on Kubernetes. It depends on a custom Docker container which sets the WORKDIR to /app.

In my pull steps, I then clone a repository inside /app, which contains a folder prefect_flow(note that I do not need to change directory here: the flow starts in the WORKDIR as expected) . I need to change directory to this one in order for the flow to work. So, in the pull steps of my deployment, I've set the following step:

{
  "prefect.deployments.steps.set_working_directory": {
    "directory": "prefect_flow"
  }
}

But this fails, saying that there is no such file or directory named prefect_flow. If I use and absolute path instead (/app/prefect_flow) this works as expected.

Here is the weirder part. Looking at set_working_directory's source code, it uses os.chdir under the hood. I have written a similar function that looks like this:

def chdir(directory: str | Path, log_result: bool = False):
    os.chdir(directory)
    if log_result:
        print(f"Changed directory to {os.getcwd()}")

This function is defined in a local package named prefect_steps.steps which is pulled when cloning the repo.

If use this function in my pull steps, like this:

{
  "prefect_steps.steps.chdir": {
    "requires": "prefect-steps>=0.1.0",
    "directory": "prefect_flow"
  }
}

everything works as expected: I can pass a relative path to this step and it works. I am not sure why this is the case, it seems to me that these two functions do the same thing.

Version info

Version:             3.4.3
API version:         0.8.4
Python version:      3.10.15
Git commit:          1c2ba7a4
Built:               Thu, May 22, 2025 10:00 PM
OS/Arch:             darwin/arm64
Profile:             default
Server type:         cloud
Pydantic version:    2.11.5
Integrations:
  prefect-aws:       0.5.10

Additional context

No response

NicholasPini avatar May 28 '25 08:05 NicholasPini