xarray-sentinel
xarray-sentinel copied to clipboard
Chunking works only with dask installed
I am not sure if this is a bug and, if so, of what.
>>> path = "tests/data/S1B_IW_SLC__1SDV_20210401T052622_20210401T052650_026269_032297_EFA4.SAFE/manifest.safe"
>>> group = 'IW1/R168-N471-E0118'
>>> xs.sentinel1.open_dataset(path, group=group)
<xarray.Dataset>
Dimensions: (azimuth_time: 1501, slant_range_time: 21632)
Coordinates:
line (azimuth_time) float64 0.5 1.5 2.5 ... 1.5e+03 1.5e+03
pixel (slant_range_time) float64 0.5 1.5 ... 2.163e+04 2.163e+04
* azimuth_time (azimuth_time) datetime64[ns] 2021-04-01T05:26:24.20999...
* slant_range_time (slant_range_time) float64 0.005343 0.005343 ... 0.005679
Data variables:
VH (azimuth_time, slant_range_time) complex128 ...
VV (azimuth_time, slant_range_time) complex128 ...
Attributes: (12/14)
constellation: sentinel-1
platform: sentinel-1b
instrument: ['c-sar']
sat:orbit_state: descending
sat:absolute_orbit: 26269
sat:relative_orbit: 168
... ...
sar:instrument_mode: IW
sar:polarizations: ['VV', 'VH']
sar:product_type: SLC
xs:instrument_mode_swaths: ['IW1', 'IW2', 'IW3']
Conventions: CF-1.7
history: created by xarray_sentinel-999
>>> xs.sentinel1.open_dataset(path, group=group, chunks=1000)
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-5-79175c680e53> in <module>
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-7-90f5963f6ea0> in <module>
----> 1 xs.sentinel1.open_dataset(path, group=group, chunks=1000)
~\xarray-sentinel\xarray_sentinel\sentinel1.py in open_dataset(product_urlpath, drop_variables, group, chunks, fs)
384 else:
385 annotation_path = list(groups[group]["annotation_path"].values())[0]
--> 386 ds = open_burst_dataset(
387 manifest_path,
388 measurement_paths=groups[group]["measurement_path"],
~\xarray-sentinel\xarray_sentinel\sentinel1.py in open_burst_dataset(manifest_path, burst_position, measurement_paths, annotation_path, chunks)
297 data_vars = {}
298 for pol, data_path in measurement_paths.items():
--> 299 arr = rioxarray.open_rasterio(data_path, chunks=chunks)
300
301 arr = arr.squeeze("band").drop_vars(["band", "spatial_ref"])
~\miniconda3\envs\xr-sentinel\lib\site-packages\rioxarray\_io.py in open_rasterio(filename, parse_coordinates, chunks, cache, lock, masked, mask_and_scale, variable, group, default_name, **open_kwargs)
863
864 if chunks is not None:
--> 865 result = _prepare_dask(result, riods, filename, chunks)
866
867 # Make the file closeable
~\miniconda3\envs\xr-sentinel\lib\site-packages\rioxarray\_io.py in _prepare_dask(result, riods, filename, chunks)
573 """
574 # pylint: disable=import-outside-toplevel
--> 575 from dask.base import tokenize
576
577 # augment the token with the file modification time
ModuleNotFoundError: No module named 'dask'
We could:
- Add can add a check to skip tests with dask if dask is not installed.
- Improve the message error message. That should be done in rioxarray
This is indeed the default behaviour of xarray. But I agree that the error message is awful.
@aurghs I propose to do in the reverse order:
- open a PR to rioxarray where it raises an exception with a better error message
- (if 1. fails) add the same check in xarray-sentinel
This is indeed the default behaviour of xarray. But I agree that the error message is awful.
1 and 2 are not an aut-aut. I would:
- Add a check to skip tests (pyest) with dask if dask is not installed.
- Add dask in setup.cfg as optional
- open a PR to rioxarray where it raises an exception with a better error message
- we do install dask for testing already, no need to test without dask, we don't have any dask related logic to test. (I propose no action)
- dask is not a needed dependency so it must not be listed in the install_requires (no action)
- we agree, @aurghs would you please take care of it?
- we could add an xfails on the tests with dask
- I agree
- I think @corrado9999 was preparing the PR.