Custom .salem_cache location (if any)

[okhoma]

Hello,

The shared .salem_cache folder is causing an issue with parallel processing on the same machine under the same user (e.g. Azure Batch), when running two tasks using salem would fail simply because two processes would attempt to create the same folder and one of them may fail. I would, of course, prefer not to download extra (especially sample!) files as requested in #196. However, if that's more complicated, the location of the folder can be controlled by an environment variable for such cases.

Thank you, Oleh Khoma

[fmaussion]

OK yes. Salem only writes once in this folder (unpacking after download), so a quick workaround would be to trigger download before multiprocessing is used as suggested in #196.

That being said, I can see that preventing the automated download can be useful and I'll work on that in the next week(s)

[fmaussion]

currently cache_dir is hard coded here: https://github.com/fmaussion/salem/blob/d3f2e5e340c2af36c84c82a9de6099c90fba12e8/salem/init.py#L42

[okhoma]

Thanks for a quick response. Unfortunately, when batch starts multiple processes automatically at the same time, if I trigger download in all processes (which are equal), it will again be triggered at the same time and the issue will persist. However, I just thought that I should be able to try/except my way out of this conundrum when importing salem. That should be a good enough workaround.

[okhoma]

FYI this is a traceback I am getting.

Traceback (most recent call last):
...
    import salem  # noqa: F401
  File "D:\batch\tasks\applications\prism12021-03-29-16-27\venv\lib\site-packages\salem\__init__.py", line 68, in <module>
    from salem.graphics import get_cmap, DataLevels, Map
  File "D:\batch\tasks\applications\prism12021-03-29-16-27\venv\lib\site-packages\salem\graphics.py", line 47, in <module>
    _ = get_demo_file('world_borders.shp')
  File "D:\batch\tasks\applications\prism12021-03-29-16-27\venv\lib\site-packages\salem\utils.py", line 230, in get_demo_file
    d = download_demo_files()
  File "D:\batch\tasks\applications\prism12021-03-29-16-27\venv\lib\site-packages\salem\utils.py", line 209, in download_demo_files
    zf.extractall(odir)
  File "D:\batch\tasks\applications\prism12021-03-29-16-27\venv\lib\zipfile.py", line 1616, in extractall
    self._extract_member(zipinfo, path, pwd)
  File "D:\batch\tasks\applications\prism12021-03-29-16-27\venv\lib\zipfile.py", line 1666, in _extract_member
    os.mkdir(targetpath)
FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'D:\\Users\\PoolNonAdmin34074842\\.salem_cache\\salem-sample-data-758f7ddd0fa6b5b1bd4c63b6dcfe8d5eec0f4c59\\baseline_images\\2.0.x\\freetype_28'