AES-NI fails on ‎Westmere

[henrycg]

The AES-NI code in aesni_helpers.c uses the AVX instruction vpslldq but Westmere machines (and possibly others) support AES-NI but do not support the AVX instructions. Because of this, the AES implementation in rust-crypto throws an SIGILL exception on these machines.

[yd021976]

Hello, yes, i logged an issue #391 relative to this bug. I'm trying to use PSLLDQ instead, but actually no luck...

[yd021976]

Hello again, got it working by using PSLLDQ instead of VPSLLDQ .

you can replace this block in aesni_helpers.c, line 62 :

1: \
            pshufd $0xff, %%xmm2, %%xmm2; \
            vpslldq $0x04, %%xmm1, %%xmm3; \
            pxor %%xmm3, %%xmm1; \
            vpslldq $0x4, %%xmm1, %%xmm3; \
            pxor %%xmm3, %%xmm1; \
            vpslldq $0x04, %%xmm1, %%xmm3; \
            pxor %%xmm3, %%xmm1; \
            pxor %%xmm2, %%xmm1; \
            movdqu %%xmm1, (%0); \
            add $0x10, %0; \
            ret; \

By this :

1: \
            pshufd $0xff, %%xmm2, %%xmm2; \
            movdqa %%xmm1, %%xmm3; \
            pslldq $0x04, %%xmm3; \
            pxor %%xmm3, %%xmm1; \
            pslldq $0x04, %%xmm3; \
            pxor %%xmm3, %%xmm1; \
            pslldq $0x04, %%xmm3; \
            pxor %%xmm3, %%xmm1; \
            pxor %%xmm2, %%xmm1; \
            movdqu %%xmm1, (%0); \
            add $0x10, %0; \
            ret; \

[celevra]

thank you @yd021976, this helped me