a downloaded file was overwritten when using remote execution

[LanceKnight]

Hello,

When I remotely executing my codes, I got this error:

Yes, the downloaded file is .gz and it should look for .csv file. The codes should manually gunzip it. However, even if I manually gunzip it, the clearml-agent will disregard the existence of .gz and .csv file and re-download the folder and thus these .gz and .csv files got overwritten by the new downloaded folder, which makes it unable to find the needed .csv file.

It works perfectly fine if I manually execute the codes from the local machine in the .clearml venv environment.

Can someone help me with this? Thanks!

[bmartinn]

Hi @LanceKnight Seems like it is trying to reference a relative path (see last line in the log ../../data/raw ...) Since the code is running from a newly created virtual environment, the relative path changed. Maybe use absolute path ?

[LanceKnight]

HI, @bmartinn , it turned out to be a problem with another package. See this. Thanks for the absolute path suggestion so I can identify the problem :)