calculator
calculator copied to clipboard
Fix #222: Wrong result for e.g. √(2.25) - 1.5 #222
Fixes #222, #309, #574
Description of the changes:
- add a helper function _snaprat to check if a result RAT should be zero with given precision. Apply _snaprat to addrat/subrat/lograt.
- The root causes of issues #222, #309, #574 are the same. For an irrational number result, the approximation introduces small errors beyond precision required, which is expected. The issue was Ratpak tried to take "precision" digits from the calculation result regardless where the result came from, and it exposed digits from approximation errors. Take
sqrt(2.25) - 1.5for example.sqrt(2.25)might be approximated to1.499...998...depending on the precision. When this number subtracted1.5, the Ratpak got0.000...001....Ratpak took up to precision number of digits and converts to scientific format, which resulted-1.2566515751494548126925332972115e-47. This PR checks the magnitude difference between the result and the input operands. Since the result is too close to47magnitude smaller than original subtraction operands1.499...998...and1.5, it would be snapped to0. - The same fix applies to
logratas well. - One further optimization could be a new sqrt function specifically checking perfect square first before using Taylor series approximation.
How changes were validated:
- Add 1 test case in
CalculatorManagerTestStandardto verify addrat. - Add 2 test cases in
CalculatorManagerTestScientificto verify addrat and lograt - Manual test the cases in the linked issues.
- Manual test log case:
log(sqrt(2.25)/1.5)