Rounding of large numbers leading to incorrect results

[kvap]

Enter 18446744073709549568 and you get the correct bit representation: 0 10000111110 1111111111111111111111111111111111111111111111111111. But then it shows n = 18446744073709550000 and the input field also get updated to that value.

[bartaz]

Hey @kvap, thanks for trying it out!

It's been a while since I made it and worked with JS numbers in such detail, but as far as I remember this is expected behaviour.

The 32 bit format is limited to how large the numbers it can acurately represent are. In JS the largest integer that can be represented safely is Number.MAX_SAFE_INTEGER which is 9007199254740991 anything larger than that will be rounded to the closest value that can be represented.

So in case of number like 18446744073709549568, the closest possible value in 32-bit floating point is 18446744073709550000. Even in browser JS console you can see that when you try to use such number, it shows the rounded value:

So that's why, when you enter such number in my visualisation tool, it shows the 32 bit representation of the closest possible number, and updates the input to whatever given number is.

Hope it helps.

I believe that modern JS engines (node, modern browsers) may support other number formats, like BigInt that can be used to represent larger integers.