Unable to read GeoTIFFs created with -ot Float32 -co NBITS=16

[zdila]

FeaturePreservingSmoothing is unable to read GeoTIFFs created with -ot Float32 -co NBITS=16:

$ gdalwarp -srcnodata -overwrite -of GTiff -ot Float32 -co NBITS=16 -co TILED=YES -co BIGTIFF=YES -co COMPRESS=LZW -co NUM_THREADS=ALL_CPUS -wo NUM_THREADS=ALL_CPUS -multi -r cubicspline -order 3 -tr 19.109257071294062687 19.109257071294062687 -tap -t_srs "+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs +over" N47E020.HGT N47E020.tif

$ whitebox_tools -r=FeaturePreservingSmoothing -v --wd="." --dem=N47E020.tif -o=test.tif --filter=15 --norm_diff=20.0 --num_iter=4
*****************************************
* Welcome to FeaturePreservingSmoothing *
*****************************************
Reading data...
thread 'main' panicked at 'The raster was not read correctly', src/main.rs:72:21
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Without -co NBITS=16 it works.

[jblindsay]

@zdila Thank you for submitting this error and the details necessary to diagnose it. Can i ask, how is it possible to have a 32-bit float with 16-bits per sample? To me, that doesn't sound like a possible data format.

[zdila]

From https://gdal.org/drivers/raster/gtiff.html:

NBITS=n: Create a file with less than 8 bits per sample by passing a value from 1 to 7. The apparent pixel type should be Byte. Values of n=9…15 (UInt16 type) and n=17…31 (UInt32 type) are also accepted. From GDAL 2.2, n=16 is accepted for Float32 type to generate half-precision floating point values.

So maybe it is then 16-bit float.

[jblindsay]

16-bit floats are incredibly rare. The only programming language that I am aware of that supports them is zig. Rust, the language that WBT is developed using doesn't have a native 16-bit float. I'd need to look into how to program to use it. Nonetheless, it seems quite contradictory to have a data format of Float32 and a bits-per-sample of 16. The GeoTIFF format does not have a corresponding flag for a Float16 that I am aware of either.

[zdila]

I am ok with that and this issue is not a blocker for me. I only reported it so that you and/or other users are aware of it :-).

[jblindsay]

So it turns out that there is a library for rust to add support for working with 16-bit floats. But honestly, the precision of a 16-bit float is so poor that I'm not sure you'd want to use it to store elevations (I believe you were importing an SRTM DEM). For example, you wouldn't be able to store the small decrement value needed to fix flats after depression removal. You'd be better off multiply the DEM elevations by 10 or 100 (depending on elevation range) and converting to a 16-bit int (a short) data format instead.

[jblindsay]

Now that I look at it more closely, I see that while GDAL refers to it as Float32, in fact the GeoTIFF tag for SampleFormat is actually just 'Floating point data', and it interprets the type of float (single or double) from the BitsPerSample tag, which is usually either 32 or 64. So that said, it seems that the GeoTIFF format could theoretically handle half-precision floating point data. (As a side note, this is one of the things that drives me nuts about the GeoTIFF...it's so darn flexible that it makes it an almost impossible target for a single developer to create an encoder/decoder...but that's another story!) And as such, WBT should be able to read/write half-precision floats. I'll leave this open for now but I still wouldn't recommend using the half precision format for working with DEM data. Besides, your DEM is really quite small and you likely don't need the extra file savings of converting to a half-precision. When using WBT with it, don't forget to use the new compress_rasters flag to output compressed rasters. Lastly, I didn't realize that people still used the SRTM HGT data format. I had thought that most people used the TIFF SRTM outputs that are available. And so if there are still a lot of HGT use out there, I'd be happy to add an SRTM reader to WBT so that you can read these files in natively without the GDAL go-between. The HGT format is quite simple.

[zdila]

Besides, your DEM is really quite small

It was jus a testcase :-)