purescript-prelude icon indicating copy to clipboard operation
purescript-prelude copied to clipboard

`ordNumber` is not compatible with IEEE 754 floats

Open anttih opened this issue 2 years ago • 2 comments

purs compiles the following:

(1.0 < nan) == false

to

1.0 < nan === false

but if we instead go through the Ord API we can observe different behavior that is not compatible with IEEE 754:

compare 1.0 nan == GT

In addition to just being wrong, the more practical issue is that any alternative backend that does not inline the use of comparison operators to operators that implement IEEE 754 correctly won't pass the prelude test suite. For example backend-es does not pass the prelude test suite.

One simple fix that was floated around in Discord is to just remove the tests using NaN from the test suite and leave the behavior undefined.

anttih avatar Jul 26 '23 07:07 anttih

This is one of the situations where Poset would have been nice to have, since comparing NaN with something should be NC.

There are issues with Eq too though, since NaN comparisons aren't reflexive.

garyb avatar Jul 26 '23 12:07 garyb

I think the only solution for status quo is to consider NaN behavior undefined for the purposes of Eq and Ord, and we just shouldn't test it.

natefaubion avatar Jul 26 '23 18:07 natefaubion