cpc icon indicating copy to clipboard operation
cpc copied to clipboard

Trig functions result in floating point error

Open EsotericDryad opened this issue 2 years ago • 3 comments

Description: using "pi" with trig functions results in close but wrong floating point results

Steps to reproduce: cpc 'sin(pi)' or cpc 'sin(pi/2)'

Expected result: 0 and 1 respectively

Actual result: 5.012535993076719585277299108368256E-35 and 1.000000000000000000000000000000001

EsotericDryad avatar Aug 21 '23 05:08 EsotericDryad

Not really sure how to fix that, any ideas?

probablykasper avatar Aug 21 '23 05:08 probablykasper

To prevent this from going stale I'll update, but I don't really have a simple and satisfying solution yet...

A few options are

  • pass the buck: submit a feature request to the decimal library to include features like constant::PI and sin(). Both are included for floats in the standard library, as well as other decimal libraries, so it's not too outlandish to request them in a numerical data-type library
  • use another decimal library. For example rust_decimal offers constants and trig functions... but this option probably requires a total rewrite
  • use f128 (or standard library floats) for trig and convert to d128 if possible? The drawback being the loss of precision as magnitude increases
  • use lookups for well-defined angles (i.e. multiples of pi/3 and pi/4)

EsotericDryad avatar Aug 24 '23 01:08 EsotericDryad

I probably won't work on this, but would welcome a PR. fend is a great alternative to try out, trigonometry seems to work there

One thing about other decimal libraries is some have a much smaller range of possible numbers

probablykasper avatar Sep 20 '23 22:09 probablykasper