monocongo
xarray NameError causes crash when running SPI on Linux
A user contacted me about an issue and as usual, it was related to NCO (or pynco?) when running on Windows. My initial idea was to run the processing on Linux since this almost always solves the issue and is the best I can advise until I get the time to excise the data wrangling code that uses NCO/pynco from this package. But to my surprise, when I attempted to run the user's command on their dataset I encountered an error related to xarray. This was in a fresh Python 3.7 conda environment with only the climate-indices package installed (from PyPI). The command and error are shown below:
$ process_climate_indices --index spi --periodicity daily --netcdf_precip imd_Rain_Saurashtra.nc --var_name_precip rain --scales 30 90 --calibration_start_year 1951 --calibration_end_year 2020 --multiprocessing all --output_file_base arjun
Traceback (most recent call last):
File "/home/james/miniconda3/envs/arjun/bin/process_climate_indices", line 5, in <module>
from climate_indices.__main__ import main
File "/home/james/miniconda3/envs/arjun/lib/python3.7/site-packages/climate_indices/__main__.py", line 12, in <module>
import xarray as xr
File "/home/james/miniconda3/envs/arjun/lib/python3.7/site-packages/xarray/__init__.py", line 3, in <module>
from . import testing, tutorial, ufuncs
File "/home/james/miniconda3/envs/arjun/lib/python3.7/site-packages/xarray/testing.py", line 8, in <module>
from xarray.core import duck_array_ops, formatting, utils
File "/home/james/miniconda3/envs/arjun/lib/python3.7/site-packages/xarray/core/duck_array_ops.py", line 16, in <module>
from . import dask_array_compat, dask_array_ops, dtypes, npcompat, nputils
File "/home/james/miniconda3/envs/arjun/lib/python3.7/site-packages/xarray/core/npcompat.py", line 81, in <module>
from numpy.typing import ArrayLike, DTypeLike
File "/home/james/miniconda3/envs/arjun/lib/python3.7/site-packages/numpy/typing/__init__.py", line 316, in <module>
from ._dtype_like import (
File "/home/james/miniconda3/envs/arjun/lib/python3.7/site-packages/numpy/typing/_dtype_like.py", line 95, in <module>
class _SupportsDType(Generic[_DType_co]):
NameError: name '_DType_co' is not defined
This appears to be an error with xarray itself, since I can get the same error when I do a simple import of xarray as the first command in a fresh Python REPL:
$ python
Python 3.7.10 (default, Jun 4 2021, 14:48:32)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import xarray
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/james/miniconda3/envs/arjun/lib/python3.7/site-packages/xarray/__init__.py", line 3, in <module>
from . import testing, tutorial, ufuncs
File "/home/james/miniconda3/envs/arjun/lib/python3.7/site-packages/xarray/testing.py", line 8, in <module>
from xarray.core import duck_array_ops, formatting, utils
File "/home/james/miniconda3/envs/arjun/lib/python3.7/site-packages/xarray/core/duck_array_ops.py", line 16, in <module>
from . import dask_array_compat, dask_array_ops, dtypes, npcompat, nputils
File "/home/james/miniconda3/envs/arjun/lib/python3.7/site-packages/xarray/core/npcompat.py", line 81, in <module>
from numpy.typing import ArrayLike, DTypeLike
File "/home/james/miniconda3/envs/arjun/lib/python3.7/site-packages/numpy/typing/__init__.py", line 316, in <module>
from ._dtype_like import (
File "/home/james/miniconda3/envs/arjun/lib/python3.7/site-packages/numpy/typing/_dtype_like.py", line 95, in <module>
class _SupportsDType(Generic[_DType_co]):
NameError: name '_DType_co' is not defined
A related issue was created in the xarray repository: https://github.com/pydata/xarray/issues/5631
This is actually caused by numpy, and is described here
I was able to work around this by installing typing-extensions
It seems that there will not be a fix in numpy soon for this for Python 3.7 and this is another impetus for this package to support later versions.