dirichlet-numerics icon indicating copy to clipboard operation
dirichlet-numerics copied to clipboard

Possible bug in ConvertToFloat, ConvertToDouble

Open GDennis opened this issue 5 years ago • 2 comments

I've just had a quick look at the code and saw a possible error:

    public static float ConvertToFloat(ref UInt128 a)
    {
        if (a.s1 == 0)
            return a.s0;
        return a.s1 * (float)ulong.MaxValue + a.s0; // looks wrong, should be:
        // return a.s1 * (float)Math.Pow(2,64) + a.s0;
    }

    public static double ConvertToDouble(ref UInt128 a)
    {
        if (a.s1 == 0)
            return a.s0;
        return a.s1 * (double)ulong.MaxValue + a.s0; // same here:
        // return a.s1 * Math.Pow(2,64) + a.s0;
    }

And better extract Math.Pow to a constant.

GDennis avatar Jul 24 '19 10:07 GDennis

Subtle, but you are correct! I will take a look at this. Thanks

ricksladkey avatar Sep 28 '20 15:09 ricksladkey

It appears a bug but in fact it's not. Run the following code and you'll see that these numbers are the same because of the precision limitations of float and double.

            var a = (double)ulong.MaxValue;
            var b = (double)ulong.MaxValue + 1.0;
            var c = Math.Pow(2, 64);
            var eqab = a == b;
            var eqbc = b == c;
            var eqca = c == a;
            Console.WriteLine($"{a}, {b}, {eqab}, {eqbc}, {eqca}");

GarfieldJiang avatar Mar 21 '21 09:03 GarfieldJiang

Approved, ulong.MaxValue as a float or double rounds up to ulong.MaxValue+1 which is pow(2,64).

GDennis avatar Jan 03 '24 05:01 GDennis