dex-lang icon indicating copy to clipboard operation
dex-lang copied to clipboard

Do unsigned remainder on unsigned types

Open axch opened this issue 2 years ago • 1 comments

Manifestation:

big = 2147483647
rem (big + 1) big
> 4294967295

Expected the answer to be 1.

(Note that big + 1 doesn't overflow the underlying Word32 -- it's just the rem operation messing up).

The cause is https://github.com/google-research/dex-lang/blob/c996e988d07f4bec0495136c9d4ca51bcb0c431c/src/lib/ImpToLLVM.hs#L614, which unconditionally emits a signed remainder instruction, regardless of whether the Dex type is meant to be signed or unsigned.

axch avatar Feb 05 '23 14:02 axch

By inspection, I expect IDiv to have the same problem.

axch avatar Feb 05 '23 14:02 axch