TopoPyScale
TopoPyScale copied to clipboard
ArcticSnow
Running TopoPyScale examples on Mac (M3)
Hi all
When running the TopoPyScale examples on Mac (ARM M3) I get some IO errors with h5py, probably due to the fact that the tmp files are opened at the same time and then deleted by different processes. When running for 'CPU_cores: 1 ' I still get a similar error. The code endlessly restarts by itself but never really crashes. The same code runs fine on a Linux machine.
Any ideas ? Have there been any other tests for Mac ? I'm not 100% sure this is Mac related.
Below is the output when running pipeline_test.py from ex1_norway_finse (I've attached the config file(config_test.txt), the pipeline file (pipeline_test.txt and the list of the conda packages I'm using (conda_env.txt))
Best,
Maurice van Tiggelen
-------------------------------
Toposub Spatial Downscaling Example
-------------------------------
---> Output directory cleaned
---> no FSM directory to clean
---> no ensemble directory to clean
---> DEM file found
{'latN': 60.68443065002782, 'latS': 60.50790355399738, 'lonW': 7.354447266990793, 'lonE': 7.731798660953054}
Project lat/lon extent:
------------------------------
| North:60.7 |
|West:7.4 East:7.7|
| South:60.5 |
------------------------------
---> Loading ERA5 surf climate forcing
Start = 2018-Oct
End = 2018-Dec
ECWMF SURF data found:
0 SURF_201810.nc
1 SURF_201811.nc
2 SURF_201812.nc
Name: target_file, dtype: object
---> Loading ERA5 plev climate forcing
Start = 2018-Oct
End = 2018-Dec
ECWMF PLEV data found:
0 PLEV_201810.nc
1 PLEV_201811.nc
2 PLEV_201812.nc
Name: target_file, dtype: object
---> No ds_param.nc Dataset found. DEM /Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse//inputs/dem/ASTER_Finse.tif available.
---> Extracting DEM parameters (slope, aspect, svf)
Computing slope and aspect ...
Computing svf ...
---> File /Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/outputs/ds_param.nc saved
---> Scaling data prior to clustering
---> Clustering with Mini-Batch K-means in 5 clusters
---> Mini-Batch Kmean finished in 0.0s
---> File /Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/outputs/ds_param.nc saved
Convert coordinates from EPSG:32632 to EPSG:4326
---> Centroids file df_centroids.pck saved
---> Computing solar geometry
---> File /Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/outputs/ds_solar.nc saved
---> Computing horizons with 10 degree increments
---> File /Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/outputs/da_horizon.nc saved
---> Centroids file df_centroids.pck updated with horizons
-------------------------------------------------
TopoScale - Downscaling
---> Downscaling climate to list of points using TopoScale
/Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/outputs/tmp cleaned
No ERA5T PRESSURE data present with additional dimension <expver>
Preparing plev for point 0
Preparing plev for point 1
Preparing plev for point 2
Preparing plev for point 3
Preparing plev for point 4
No ERA5T PRESSURE data present with additional dimension <expver>
Preparing surf for point 0
Preparing surf for point 1
Preparing surf for point 2
Preparing surf for point 3
Preparing surf for point 4
-------------------------------
-------------------------------
Toposub Spatial Downscaling Example
Toposub Spatial Downscaling Example
-------------------------------
-------------------------------
Traceback (most recent call last):
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/site-packages/TopoPyScale/topoclass.py", line 68, in __init__
shutil.rmtree(self.config.outputs.path)
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/shutil.py", line 734, in rmtree
---> Output directory cleaned
---> no FSM directory to clean
---> no ensemble directory to clean
_rmtree_safe_fd(fd, path, onerror)
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/shutil.py", line 690, in _rmtree_safe_fd
onerror(os.unlink, fullname, sys.exc_info())
---> DEM file found File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/shutil.py", line 688, in _rmtree_safe_fd
os.unlink(entry.name, dir_fd=topfd)
FileNotFoundError: [Errno 2] No such file or directory: 'ds_param.nc'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/multiprocessing/spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/multiprocessing/spawn.py", line 125, in _main
prepare(preparation_data)
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/multiprocessing/spawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/runpy.py", line 288, in run_path
return _run_module_code(code, init_globals, run_name,
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/pipeline_test.py", line 9, in <module>
mp = tc.Topoclass(config_file)
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/site-packages/TopoPyScale/topoclass.py", line 71, in __init__
os.makedirs(self.config.outputs.path)
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/os.py", line 225, in makedirs
mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/outputs'
{'latN': 60.68443065002782, 'latS': 60.50790355399738, 'lonW': 7.354447266990793, 'lonE': 7.731798660953054}
Project lat/lon extent:
------------------------------
| North:60.7 |
|West:7.4 East:7.7|
| South:60.5 |
------------------------------
---> Loading ERA5 surf climate forcing
Start = 2018-Oct
End = 2018-Dec
ECWMF SURF data found:
0 SURF_201810.nc
1 SURF_201811.nc
2 SURF_201812.nc
Name: target_file, dtype: object
---> Loading ERA5 plev climate forcing
Start = 2018-Oct
End = 2018-Dec
ECWMF PLEV data found:
0 PLEV_201810.nc
1 PLEV_201811.nc
2 PLEV_201812.nc
Name: target_file, dtype: object
---> No ds_param.nc Dataset found. DEM /Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse//inputs/dem/ASTER_Finse.tif available.
---> Extracting DEM parameters (slope, aspect, svf)
Computing slope and aspect ...
Computing svf ...
---> File /Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/outputs/ds_param.nc saved
---> Scaling data prior to clustering
---> Clustering with Mini-Batch K-means in 5 clusters
---> Mini-Batch Kmean finished in 0.0s
---> File /Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/outputs/ds_param.nc saved
Convert coordinates from EPSG:32632 to EPSG:4326
---> Centroids file df_centroids.pck saved
---> Computing solar geometry
---> File /Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/outputs/ds_solar.nc saved
---> Computing horizons with 10 degree increments
---> File /Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/outputs/da_horizon.nc saved
---> Centroids file df_centroids.pck updated with horizons
-------------------------------------------------
TopoScale - Downscaling
---> Downscaling climate to list of points using TopoScale
/Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/outputs/tmp cleaned
No ERA5T PRESSURE data present with additional dimension <expver>
Preparing plev for point 0
Preparing plev for point 1
-------------------------------
Toposub Spatial Downscaling Example
-------------------------------
---> Output directory cleaned
---> no FSM directory to clean
---> no ensemble directory to clean
---> DEM file found
{'latN': 60.68443065002782, 'latS': 60.50790355399738, 'lonW': 7.354447266990793, 'lonE': 7.731798660953054}
Project lat/lon extent:
------------------------------
| North:60.7 |
|West:7.4 East:7.7|
| South:60.5 |
------------------------------
---> Loading ERA5 surf climate forcing
Start = 2018-Oct
End = 2018-Dec
ECWMF SURF data found:
0 SURF_201810.nc
1 SURF_201811.nc
2 SURF_201812.nc
Name: target_file, dtype: object
---> Loading ERA5 plev climate forcing
Start = 2018-Oct
End = 2018-Dec
ECWMF PLEV data found:
0 PLEV_201810.nc
1 PLEV_201811.nc
2 PLEV_201812.nc
Name: target_file, dtype: object
---> No ds_param.nc Dataset found. DEM /Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse//inputs/dem/ASTER_Finse.tif available.
---> Extracting DEM parameters (slope, aspect, svf)
Computing slope and aspect ...
Computing svf ...
Preparing plev for point 2
Preparing plev for point 3
---> File /Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/outputs/ds_param.nc saved
---> Scaling data prior to clustering
---> Clustering with Mini-Batch K-means in 5 clusters
---> Mini-Batch Kmean finished in 0.0s
---> File /Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/outputs/ds_param.nc saved
Convert coordinates from EPSG:32632 to EPSG:4326
---> Centroids file df_centroids.pck saved
---> Computing solar geometry
---> File /Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/outputs/ds_solar.nc saved
---> Computing horizons with 10 degree increments
Preparing plev for point 4
---> File /Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/outputs/da_horizon.nc saved
---> Centroids file df_centroids.pck updated with horizons
-------------------------------------------------
TopoScale - Downscaling
---> Downscaling climate to list of points using TopoScale
/Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/outputs/tmp cleaned
No ERA5T PRESSURE data present with additional dimension <expver>
Preparing plev for point 0
Preparing plev for point 1
No ERA5T PRESSURE data present with additional dimension <expver>
Preparing surf for point 0
Preparing surf for point 1
Preparing surf for point 2
Preparing surf for point 3
Preparing surf for point 4
Preparing plev for point 2
Preparing plev for point 3
Traceback (most recent call last):
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/site-packages/xarray/backends/file_manager.py", line 210, in _acquire_with_cache_info
file = self._cache[self._key]
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/site-packages/xarray/backends/lru_cache.py", line 56, in __getitem__
value = self._cache[key]
KeyError: [<class 'h5netcdf.core.File'>, ('/Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/outputs/tmp/ds_plev_pt_2.nc',), 'r', (('decode_vlen_strings', True), ('invalid_netcdf', None)), '3c6122fe-2263-432f-bd1c-5129e91f4c7c']
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/multiprocessing/spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/multiprocessing/spawn.py", line 125, in _main
prepare(preparation_data)
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/multiprocessing/spawn.py", line 236, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/runpy.py", line 288, in run_path
return _run_module_code(code, init_globals, run_name,
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/Users/Tigge006/surfdrive/04_Scripts/Python/TopoPyScale_examples/ex1_norway_finse/pipeline_test.py", line 15, in <module>
mp.downscale_climate()
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/site-packages/TopoPyScale/topoclass.py", line 627, in downscale_climate
ta.downscale_climate(self.config.project.directory,
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/site-packages/TopoPyScale/topo_scale.py", line 477, in downscale_climate
plev_pt_list.append(xr.open_dataset(output_directory / f'tmp/ds_plev_pt_{row.point_name}.nc', engine='h5netcdf'))
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/site-packages/xarray/backends/api.py", line 525, in open_dataset
backend_ds = backend.open_dataset(
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/site-packages/xarray/backends/h5netcdf_.py", line 413, in open_dataset
store = H5NetCDFStore.open(
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/site-packages/xarray/backends/h5netcdf_.py", line 176, in open
return cls(manager, group=group, mode=mode, lock=lock, autoclose=autoclose)
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/site-packages/xarray/backends/h5netcdf_.py", line 127, in __init__
self._filename = find_root_and_group(self.ds)[0].filename
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/site-packages/xarray/backends/h5netcdf_.py", line 187, in ds
return self._acquire()
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/site-packages/xarray/backends/h5netcdf_.py", line 179, in _acquire
with self._manager.acquire_context(needs_lock) as root:
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/contextlib.py", line 119, in __enter__
return next(self.gen)
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/site-packages/xarray/backends/file_manager.py", line 198, in acquire_context
file, cached = self._acquire_with_cache_info(needs_lock)
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/site-packages/xarray/backends/file_manager.py", line 216, in _acquire_with_cache_info
file = self._opener(*self._args, **kwargs)
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/site-packages/h5netcdf/core.py", line 1051, in __init__
self._h5file = self._h5py.File(
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/site-packages/h5py/_hl/files.py", line 567, in __init__
fid = make_fid(name, mode, userblock_size, fapl, fcpl, swmr=swmr)
File "/Users/Tigge006/opt/anaconda3/envs/downscaling/lib/python3.9/site-packages/h5py/_hl/files.py", line 231, in make_fid
fid = h5f.open(name, flags, fapl=fapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5f.pyx", line 106, in h5py.h5f.open
BlockingIOError: [Errno 35] Unable to synchronously open file (unable to lock file, errno = 35, error message = 'Resource temporarily unavailable')
Preparing plev for point 4
No ERA5T PRESSURE data present with additional dimension <expver>
Preparing surf for point 0
Preparing surf for point 1
Preparing surf for point 2
Preparing surf for point 3
Preparing surf for point 4
-------------------------------
Toposub Spatial Downscaling Example
-------------------------------
---> Output directory cleaned
---> no FSM directory to clean
---> no ensemble directory to clean
---> DEM file found
{'latN': 60.68443065002782, 'latS': 60.50790355399738, 'lonW': 7.354447266990793, 'lonE': 7.731798660953054}
Project lat/lon extent:
------------------------------
| North:60.7 |
|West:7.4 East:7.7|
| South:60.5 |
------------------------------
---> Loading ERA5 surf climate forcing
.
.
.
.
.
.
Hello @mvantiggelen ,
It was installed succesfully on a Mac, but maybe not an ARM architecture one. I have never tried myself. It is acutally Varya (@varyabazilova ), close to you that installed it on a Mac. So you may try to have a chat with her.
It may have to do how the Python multiprocessing toolbox is running on ARM processor. If you find a fix let us know
Any success @mvantiggelen ?
No solution yet! I've temporally switched to a Linux cluster for testing
Hello, I've also been running into the same problem on Mac, I'll try switching over to Linux as well for the time being.... I don't know if it's related, but Andrew's comment about topocalc not working with the Mac environment makes me wonder.
I have a question/request. Could I request/schedule a tutorial at some point? I had to take a break from trying to get TopoPyScale to run on my computer, and would like some additional help if possible, time willing...