algorithmica icon indicating copy to clipboard operation
algorithmica copied to clipboard

description of operations on integer-valued doubles is slightly inaccurate

Open mmp opened this issue 3 years ago • 0 comments

In https://en.algorithmica.org/hpc/arithmetic/float/:

due to the way floating-point arithmetic works, all integer numbers between -2^{53} and 2^{53} and results of computations involving them can be stored exactly,

That's not true for multiplication or division, or if there is overflow, e.g. adding 2^{53}-1 + 2^{53}. (Filing as a bug rather than a PR since I'm not sure how to suggest rewording it.)

mmp avatar Jul 01 '22 15:07 mmp