msgpack-javascript icon indicating copy to clipboard operation
msgpack-javascript copied to clipboard

Decoding of 32 bit floats is not precise

Open ThetaSinner opened this issue 2 years ago • 3 comments
trafficstars

I am trying to understand why I get unexpected values decoding 32-bit floating point values coming from my back-end.

This is true

decode(encode(59.48)) === 59.48

But looking at the intermediate value, the encoding is producing a 64-bit floating point representation [203, 64, 77, 189, 112, 163, 215, 10, 61].

So far, all is good. The problem starts when I post my raw data to my back-end and that does the serialization. That has been told the type up-front and encodes the input as a 32-bit floating point number -> [202, 66, 109, 235, 133]. That should be fine, and when decoding on the back-end, I get the exact value back.

However, when decoding using this library like decode([202, 66, 109, 235, 133]) what I actually get is 59.47999954223633.

I believe this is a bug because the library can safely round-trip floating point numbers without a loss of precision so it seems like there is an issue in the handling of the 32-bit msgpack representation of this number.

ThetaSinner avatar May 19 '23 11:05 ThetaSinner

I think this library behaves correctly, you see the precision loss of the float64 ⇒ float32 conversion.

> 59.48.toFixed(50)
"59.47999999999999687361196265555918216705322265625000"
> Math.fround(59.48).toFixed(50)
"59.47999954223632812500000000000000000000000000000000"

BerndAmend avatar Jun 02 '23 06:06 BerndAmend

I see what you're saying with the conversion but I really need encoding/decoding to round trip correctly so that would suggest to me that the toFixed isn't appropriate. I would prefer to have the exact value unpacked into a JavaScript number, i.e. 64-bit value. The library already encodes that way anyway because it's not working off a schema. At least in that case users of multiple msgpack libraries (in my case, this one and rmp-serde for Rust) would run into clashes between their backend and frontend. So this library encodes as 64-bit, the backend is strictly expecting 32-bit, that will fail, then the user is naturally pushed over to using 64-bit floats in their backend schema. That doesn't restrict users who aren't using a JavaScript frontend and the problem is made much more obvious to those who are

ThetaSinner avatar Jun 06 '23 16:06 ThetaSinner

This is a typical IEEE 754 rounding error. In short, there's no way to precisely represent the float number 59.48 in 32-bit binary form. The closest floating point number that can be represented is 59.47999954223633.

Aetherus avatar May 17 '24 10:05 Aetherus