Create unique temporary directories

[braza2]

issue: Running parallel sessions leads to race conditions where each session tries to access or write to f"/tmp/DSSAT{VERSION}".

Replace line 50 in run.py https://github.com/daquinterop/Py_DSSATTools/blob/c9d9e1433cd822616394f7d4f0df426381d49cc2/DSSATTools/run.py#L50

which creates the folder /tmp/ with

with tempfile.TemporaryDirectory() as TMP:
    print('created temporary directory', TMP)

each session now creates a unique temporary folder, thus solving the issue.

[braza2]

Note: parallel session defined as running array jobs on a cluster with slurm.

[braza2]

Hi @daquinterop any way of integrating this fix into the next version?

[daquinterop]

Hi braza2. I tried replacing that line but it didn't work. It raises a FileNotFoundError. The f"/tmp/DSSAT{VERSION}" folder is created when the module is imported. I understand that when you create the folder using tempfile.TemporaryDirectory() the folder is removed after the context is finished, then the entire run module have to be included within that context. Please let me know if I am missing something.

[braza2]

Hi @daquinterop , yes you are right that my solution would have required to run everything in a context window. There is a better solution to just give the temporary directories a unique name, i.e. via import uuid temp_dir = os.path.join(home_dir, f"temp_{uuid.uuid4().hex}") os.makedirs(temp_dir)

Unfortunately I have repeatedly run into race conditions on clusters which make the simulations fail.

[braza2]

@daquinterop here is an example of what is going wrong. it does not only affect MODEL.ERR but also other files that already exist.

  File "***/lib/python3.12/site-packages/DSSATTools/__init__.py", line 78, in <module>
    from DSSATTools.run import DSSAT
  File "***/lib/python3.12/site-packages/DSSATTools/run.py", line 64, in <module>
    os.symlink(os.path.join(STATIC_PATH, file), file_link)
FileExistsError: [Errno 17] File exists: ***/python3.12/site-packages/DSSATTools/static/MODEL.ERR' -> '/tmp/DSSAT048/MODEL.ERR'