hardposit-chisel3 icon indicating copy to clipboard operation
hardposit-chisel3 copied to clipboard

1 ULP error in 64 bit square root

Open arunkmv opened this issue 4 years ago • 0 comments

Random tests fail with 1 ULP error for certain 64 bit square root conditions. Need to check whether test generator or DivSqrt module is responsible and rectify it.

Utilization of long double for test generation could be causing the error: pexpected = posit<nbits, es>(std::sqrt((long double)pa));

arunkmv avatar May 26 '20 11:05 arunkmv