rasterio.errors.RasterioIOError: Read or write failed

[suatcoskun]

I try to run it with docker version of dolphin/0.32.0. I used OPERA CLSC .h5 files. But when I try to run dolphin I took this error message

I added my config file. What do you think about this situation.

Best.

dolphin_config.zip

[scottstanie]

Thanks for submitting this.

I think I saw the same error recently and was struggling to get a minimum reproduceable example together. I'm pretty sure this is a GDAL bug in the geotiff writer relating to LZW compression (the default for dolphin), which pops up when you call it from Python with multi-threading... Even though we aren't actually trying to write in parallel, something about this setup was causing problems like this.

Can you try changing the config file to have

  gtiff_creation_options:
    - COMPRESS=deflate

right now it's this.

  gtiff_creation_options:
    - COMPRESS=lzw

If it works for you, i'll change the defaults in dolphin and look into letting GDAL know about this.

[suatcoskun]

I saw some interesting things when I restart to process but this time I re-downloaded the SLCS files and I did not took any warning. Maybe some download issue caused this problem. If I faced this problem again I applied your suggestion my config file.

I have one more question but it is not about this issue. How can I applied water mask my stack or which parameter trigger this option?

[scottstanie]

You can use the mask_file option: https://github.com/isce-framework/dolphin/blob/main/src/dolphin/workflows/config/_common.py#L438-L445

Right now the mask is used during unwrapping to ignore bad areas. Is that what you were looking for? Or did you mean to just zero out areas in the timeseries/*.tif results based on a water mask?

[suatcoskun]

Yes, I'm asking how to clip areas in time series/*.tif results based on the water mask.

[scottstanie]

ok gotcha. I'm assuming that the watermask has 1 with land pixels and 0 with water.

I might just use something like a loop with dolphin functions:

from dolphin import io
from pathlib import Path

# make sure it's 0/1, then turn into bool
water_mask = io.load_gdal("water_mask", masked=True).filled(0).astype(bool)
# now False is water, True is land.

for in_fname in Path("timeseries").glob("*.tif"):
    data = io.load_gdal(in_fname)
    data[~water_mask] = np.nan  # or however you want to hide water
    io.write_arr(
        arr=data,
        output_name=Path("timeseries") / f"masked_{in_fname.name}",
        like_filename=in_fname,
        nodata=np.nan,
    )

or gdal_calc (untested):

for f in timeseries/2*tif; do
    gdal_calc.py \
        -A "$f" \
        -B water_mask.tif \
        --outfile="masked_$(basename $f)" \
        --calc="A * (B == 1)" \
        --NoDataValue=0 \
        --type=Float32 \
        --co="COMPRESS=LZW"
done

[scottstanie]

Closing for now as I haven't run into this in multiple iterations of trying to reproduce. If anyone comes across this, we'll reopen it.