rioxarray icon indicating copy to clipboard operation
rioxarray copied to clipboard

rasterio DatasetReader complex_int16 incompatible with auto chunking of dask (rioxarray.open_rasterio)

Open cdubos-fr opened this issue 2 years ago • 1 comments

Code Sample, a copy-pastable example if possible

import numpy as np
import rasterio
import rioxarray

filename = "complex_int16.tiff"

with rasterio.open(filename, mode='w', width=16, height=16, count=1, dtype="complex_int16") as writer:
    writer.write(np.array([[[ i*j for j in range(16)] for i in range(16)]]))

rioxarray.open_rasterio(filename, chunks=True)

Problem description

When opening a file with rioxarray.open_rasterio passing chunks=True, rasterio.DatasetReader can have "complex_int16" dtype, that is incompatible with dask.core.normalize_chunk dtype parameter (line 682 in rioxarray._io) and cause TypeError: data type 'complex_int16' not understood.

rasterio manage complex_int16 when reading data with rasterio.dtypes._getnpdtype and rasterio.dtypes._is_complex_int to translate internal complex_int16 to np.complex64 .

Expected Output

rioxarray.open_rasterio should work and provide a xarray.DataArray with np.complex64 dtype in the case of processing a rasterio file with complex_int16 dtype.

Environment Information

  • Tested with rioxarray >= 0.10.2
  • rasterio ==1.2.10
  • Python 3.9.9
  • Linux-5.13.0-51-generic-x86_64-with-glibc2.31

Installation method

  • pip install rioxarray==${TARGET_VERSION}

cdubos-fr avatar Jun 30 '22 12:06 cdubos-fr

This is likely the line causing troubles: https://github.com/corteva/rioxarray/blob/570150fbb0d359c5567a08a71adb5efc908164f0/rioxarray/_io.py#L683

I believe this can be fixed by wrapping with this function: https://github.com/corteva/rioxarray/blob/570150fbb0d359c5567a08a71adb5efc908164f0/rioxarray/_io.py#L339-L347

A PR with the fix is welcome.

snowman2 avatar Jun 30 '22 18:06 snowman2