TiddlyWikiFormula
TiddlyWikiFormula copied to clipboard
float math issues in 1.2.5
Try this formulas without setting <<formulaPrecision>>
and <<formulaFixed>>
:
-
0.99999
=1
-
0.999999
=0.1
(wrong result at 6+ digits after point, but using<<formulaPrecision>>
and<<formulaFixed>>
can fix it) -
0.999999/1
=0.1
-
trunc(1,1)
=0.9
-
trunc(1,5)
=0.99999
-
trunc(1,6)
=0.10
(with parameter 7 and more it returns same result 0.10) -
trunc(-1,1)
=-0.9000000000000001
-
trunc(-1,2)
=-0.9900000000000005
(others in similar manner) -
round(1,19)
=0.10
-
round(1,37)
=0.10
(here may be more such "wrong numbers") -
round(-1,1)
=-1.0000000000000002
-
round(-1,2)
=-1.0000000000000004
(with parameter 3 and more last numbers changed via floating point precision errors) -
round(-1,14)
=-0.9999999999999958
-
round(-1,15)
=-1.0000000000000002
(16, 17, 18 param value return -1 as it must) -
round(-1,19)
=-0.9999999999999999
(but it already out of bounds)