toys icon indicating copy to clipboard operation
toys copied to clipboard

floor() is actually trunc()

Open MrUnbelievable92 opened this issue 1 year ago • 0 comments

As the title says, the 'double custom_floor(const double x)' function actually truncates negative values, not apply the mathematical floor function, which would for instance result in -15 given -14.5 as an input. This is also true for SSE4, where floor is defined as rounding towards negative infinity. The 'custom_floor' function rounds toward zero.

Also, it doesn't handle infinity and NaN as input, which only results in 'MOVQ', 'TEST' 'CMOVAE' as ouput instructions resulting in 1 additional cycle in latency, but I guess that's a design choice.

MrUnbelievable92 avatar Dec 04 '22 11:12 MrUnbelievable92