mobile icon indicating copy to clipboard operation
mobile copied to clipboard

Strange expression evaluation with real numbers

Open jozef-budac opened this issue 2 years ago • 3 comments

When we have an expression "real1" + "real2" in the form where real1 and real2 are fields with real numbers and real1 = -5.1 and real2 = 1.2 the result should be -3.9 but it is -3.899999999999995

For positive numbers, it works correctly and also when real1 = -5.2 or -5.3

image

jozef-budac avatar Feb 22 '23 09:02 jozef-budac

This is not a bug, IMHO. This is how floating point arithmetic works, see for example https://en.wikipedia.org/wiki/Floating-point_arithmetic

alexbruy avatar Feb 22 '23 10:02 alexbruy

Even though it is correct, it would be good to keep peculiarities of the floating point arithmetic hidden from users :-) probably we could just use fewer decimal numbers when formatting numbers, I don't think anyone needs that much precision...

wonder-sk avatar Feb 22 '23 10:02 wonder-sk

Is this happening also in the number widgets? What happens for decimal fields with set precision? Do we respect precision?

tomasMizera avatar Mar 15 '23 10:03 tomasMizera