[Bug]: The `precisionRounding: 14` causes the `=200.05 - 200` equation to print `0.0500000000000114` instead of `0.05`
Description
As in the title, if we use precisionRounding: 14 for the 200.05 - 200 calculation we get 0.0500000000000114 instead of 0.05 value. However, it all works well if we setup the precisionRounding to 10.
Video or screenshots
No response
Demo
https://codesandbox.io/s/black-lake-wdy2d3?file=/src/hyperformulaConfig.js:123-144
HyperFormula version
2.3.1
Your framework
none
Your environment
Chrome 116 / macOS Ventura
It is caused by the (im)precision of the JavaScript floating-point arithmetics. In most programming languages, calculations that involve fractional numbers are never 100% accurate, e.g. the famous case with 0.1 + 0.2 ≠ 0.3.
HyperFormula uses the standard JavaScript implementation of arithmetic operations, which says that 200.05 - 200 = 0.05000000000001137. To make this result more human-friendly, HyperFormula applies rounding according to precisionRounding parameter:
precisionRounding |
Rounded result |
|---|---|
| 10 | 0.05 |
| 11 | 0.05 |
| 12 | 0.05000000000001 |
| 13 | 0.050000000000011 |
| 14 | 0.0500000000000114 |
| 15 | 0.05000000000001137 |
Currently, our documentation says:
We recommend setting
precisionRoundingto a value between10and14.
with 14 being the default value.
What can we do?
- Change the default value of
precisionRoundingto10or11(breaking change) - Change the recommended values of
precisionRounding - Explain the rounding issue better in our documentation:
- Add
Setting precisionRounding too high will expose the floating-point calculation errors (for example, with precisionRounding set to 15, 0.1 + 0.2 results in 0.3000000000000001).to the description ofprecisionRounding - Describe the imprecision of the floating-point arithmetics in our Specifications and limits guide.
After discussing it with @evanSe we decided to apply the documentation changes (points 2 and 3) in the scope of this task. Changing the default value of the config parameter (point 1) will be scheduled for the next major release.
Changed my opinion on this, changing precisionRounding default value to something less than 14 causes more issues than it solves.
Instead of changing the default value, we will expand the reference to describe the js arithmetic issues better and recommend precisionRounding: 10 for most day-to-day use cases.