ccr icon indicating copy to clipboard operation
ccr copied to clipboard

Move portions of quantization set_local() to can_apply() callback

Open czender opened this issue 5 years ago • 2 comments
trafficstars

HDF docs on can_apply() callback:

This callback must determine whether the combination of the dataset creation property list settings, the datatype, and the dataspace represent a valid combination to which to apply this filter. For example, an invalid combination may involve the filter not operating correctly on certain datatypes, on certain datatype sizes, or on certain sizes of the chunk dataspace. If this filter is enabled through H5Pset_filter as optional and the can apply function returns FALSE, the library will skip the filter in the filter pipeline.

This callback can be the NULL pointer, in which case the library will assume that the filter can be applied to a dataset with any combination of dataset creation property list values, datatypes, and dataspaces.

The can apply callback function must return a positive value for a valid combination, zero for an invalid combination, and a negative value for an error.

If can_apply() prescreens variables for quantization, gives the right return values, and filter is marked optional, the H5Zremove_filter() call in set_local() can be removed.

czender avatar Sep 18 '20 17:09 czender

Is this issue still active? I don't actually understand it...

edwardhartnett avatar May 19 '22 14:05 edwardhartnett

This has to do with the circuitous way in which I implemented support for the _FillValue attribute in the quantization filters. The dearth of HDF5 documentation of can_apply and remove_filter prevented me from figuring out how to simplify the implementation. I'm pretty sure there's a better way, but haven't had a chance to revisit the issue. If we ever stumble across an HDF5 filter that correctly handles _FillValue we can probably copy their approach and then close this issue. But I've never seen another filter that does that, even though every lossy filter should do that.

czender avatar May 27 '22 12:05 czender