libm icon indicating copy to clipboard operation
libm copied to clipboard

FMA implementation subtly incorrect

Open gnzlbg opened this issue 6 years ago • 2 comments

It appears that unsigned integer subtraction was mapped to Sub::sub(a, b) in Rust (a - b), instead of a.wrapping_sub(b). In C, unsigned integer subtraction wraps, but in rust, unless wrapping_sub is used, it panics on overflow in debug builds, and has UB in release builds.

gnzlbg avatar Jul 06 '19 14:07 gnzlbg

It is not UB in release builds, it is wrapping in release builds.

Lokathor avatar Jul 28 '19 03:07 Lokathor

Indeed. So something else must be off as well, because it was producing segfaults for me in release.

gnzlbg avatar Jul 28 '19 04:07 gnzlbg