calculator icon indicating copy to clipboard operation
calculator copied to clipboard

Fix #222: Wrong result for e.g. √(2.25) - 1.5 #222

Open guihuaz opened this issue 3 weeks ago • 1 comments

Fixes #222, #309, #574

Description of the changes:

  • add a helper function _snaprat to check if a result RAT should be zero with given precision. Apply _snaprat to addrat/subrat/lograt.
  • The root causes of issues #222, #309, #574 are the same. For an irrational number result, the approximation introduces small errors beyond precision required, which is expected. The issue was Ratpak tried to take "precision" digits from the calculation result regardless where the result came from, and it exposed digits from approximation errors. Take sqrt(2.25) - 1.5 for example. sqrt(2.25) might be approximated to 1.499...998... depending on the precision. When this number subtracted 1.5, the Ratpak got 0.000...001.... Ratpak took up to precision number of digits and converts to scientific format, which resulted -1.2566515751494548126925332972115e-47. This PR checks the magnitude difference between the result and the input operands. Since the result is too close to 47 magnitude smaller than original subtraction operands 1.499...998... and 1.5, it would be snapped to 0.
  • The same fix applies to lograt as well.
  • One further optimization could be a new sqrt function specifically checking perfect square first before using Taylor series approximation.

How changes were validated:

  • Add 1 test case in CalculatorManagerTestStandard to verify addrat.
  • Add 2 test cases in CalculatorManagerTestScientific to verify addrat and lograt
  • Manual test the cases in the linked issues.
  • Manual test log case: log(sqrt(2.25)/1.5)

guihuaz avatar Dec 07 '25 21:12 guihuaz