Undefined Behaviour in volk_32f_invsqrt_32f

[argilo]

UBSAN shows the following Undefined Behaviour in volk_32f_invsqrt_32f:

/home/argilo/git/volk/kernels/volk/volk_32f_invsqrt_32f.h:71:22: runtime error: signed integer overflow: 1597463007 - -569061536 cannot be represented in type 'int'

This is the problematic line:

https://github.com/gnuradio/volk/blob/e853e9bb5693c7800840b736e77db52d713d553e/kernels/volk/volk_32f_invsqrt_32f.h#L71

[jdemel]

This is: Wikipedia fast inverse square root It is sometimes called John Carmack reverse because of its use in Q3A.

Fast inverse square root, sometimes referred to as Fast InvSqrt() or by the hexadecimal constant 0x5F3759DF

It is an approximation. It works. It is a hack.

Since this is actually a floating point operation, I suggest to ignore this specific error.

[argilo]

I suspect the signed integer overflow occurs because the input is negative. So this could just be a case where the test input (uniformly distributed floats in the range -1 .. +1) doesn't make sense.

[jdemel]

As long as the output signature is 32f, roots of negative values do not exist.

The C reference for sqrt says:

1. +-0 should return the input value
2. Values smaller +-0 are NaN. Since NaN == NaN is false, such input should make the test fail.