zfp icon indicating copy to clipboard operation
zfp copied to clipboard

Precision=64 is not considered fixed-precision mode

Open lindstro opened this issue 3 years ago • 0 comments

After calling zfp_stream_set_precision(stream, 64), zfp_stream_compression_mode(stream) returns zfp_mode_expert rather than the expected zfp_mode_fixed_precision. This is a result of zfp_stream_set_precision(stream, 64) not constraining any of the four expert-mode parameters, leaving them at their default values, which is a state currently considered expert mode. This behavior has an unintended effect on back-ends like CUDA that currently support fixed precision but not expert mode.

lindstro avatar Jun 10 '22 19:06 lindstro