Rework bitrate packing from [0.0, 1.0] to [-0.5, 0.5]
See this PR for original inspiration: https://github.com/nfrechette/acl/pull/348 by @ddeadguyy.
//////////////////////////////////////////////////////////////////////////
//
// Integers between 0 and 2^24 are 100% accurate as floats. Leverage this with a maximum quantization of 24 bits.
//
// Floating point */ with 2^x is precision-friendly. It shifts the exponent without touching the mantissa. This drives our quantization.
//
// Normalizing to 0.0f..1.0F is less accurate than normalizing to -0.5F..0.5F. The latter range can handle 1/(2^25), which is the error term of 24 bit quantization.
//
// If our goal was to minimize error within the range, we'd maximize error at the endpoints, so we could stop here. However, ACL expects precise endpoints, so we modify the
// scale accordingly. Note that division is more accurate than multiply-by-reciprocal when the divisor isn't a power of 2, so we monitor discretization error closely.
//
// Always floor after scaling, and before shifting from -halfQ..halfQ to 0..fullQ. Otherwise, IEEE float addition will round the result before you get a chance to floor it.
//////////////////////////////////////////////////////////////////////////
Keeping the normalization range between [0.0, 1.0] works well with the range reduction with (range min, range extent). Reconstruction becomes:
uint32 value_i = 1000;
uint32 max_value = 65535; // 16 bit quantization
float value_0_1 = float(value_i) * max_value;
float remapped = (value_0_1 * range_extent) + range_min;
If we normalize instead between [-0.5, 0.5], we have to introduce an extra addition:
uint32 value_i = 1000;
uint32 max_value = 65535; // 16 bit quantization
float value_0_1 = float(value_i) * max_value + 0.5f; // need to offset here
float remapped = (value_0_1 * range_extent) + range_min;
Considering that the unpacking code path is very hot, introducing a new addition and constant load needs to be worth it.
If we have a measurable improvement in quality/memory footprint as a result of this change, I'd be happy to move forward with it. I agree that it may be optimal, but if there isn't a clear improvement, I would rather keep the decompression code path as lean as possible.
We'll want to measure the impact on decompression as well with the new 2.0 Google Benchmark harness.
This branch will be used for the work: fix/improve-quantization-range
https://github.com/nfrechette/acl/issues/352 didn't quite work out, so I'm coming back to this. I don't expect decompression to change at all(I plan to remove the 24 bit support I added in 1.3.5, not worth it IMO). Compression will be tweaked, though.