libtomcrypt
libtomcrypt copied to clipboard
Cast STORE64H/STORE64L argument to ulong64
Avoids undefined behaviour with right shift greater than 32 bits. (c99 6.5.7 "If the value of the right operand is negative or is greater than or equal to the width of the promoted left operand, the behavior is undefined."
I haven't reproduced it here but it was hit using STORE64H outside of libtomcrypt on BCM4706 MIPS32r2 with GCC 4.2.4 https://github.com/mkj/dropbear/pull/99
The patch doesn't seem to affect code generation with -DLTC_NO_ASM on x64.
IMO this cast belongs in the calling code and not in the STORE macro.
Those macro's clearly say that they store a 64bit Variable and if you pass a 32bit in there you have to cast it.
@karel-m your opinion?
I do not have strong opinion on this. But I slightly lean towards leaving the cast to the calling code as by casting in these macros we may hide some potential issues on the calling side (which now throws at least some warning).
Closing this, since @karel-m and me basically have the same opinion on this.
Please re-open if you have further questions/comments.