fixed_point icon indicating copy to clipboard operation
fixed_point copied to clipboard

fixed_point compare uses too great a range

Open johnmcfarlane opened this issue 8 years ago • 0 comments

In

auto a = fixed_point<int8, 32>(0x100000000LL);
auto b = fixed_point<int8, 0>(1);
ASSERT_GT(a, b);

expression overflows. But only 16 (maybe 15) bits of range are required to scale operands such that comparison can be performed. See issue 123 for a discussion.

johnmcfarlane avatar Jul 01 '17 04:07 johnmcfarlane