aircrack-ng icon indicating copy to clipboard operation
aircrack-ng copied to clipboard

Failed to cross compile with CLANG

Open maroviher opened this issue 6 years ago • 18 comments

I am trying to cross compile for android AARCH64.

./configure CC=aarch64-linux-android-clang CXX=aarch64-linux-android-clang++ \
LDFLAGS=-L/tmp/openssl-android/lib LD=aarch64-linux-android-ld \
--host=aarch64-linux --with-openssl=/tmp/openssl-android --enable-libnl

make:

  CC       libaircrack_crypto_arm_neon_la-simd-intrinsics.lo
simd-intrinsics.c:273:2: error: argument should be a value from 0 to 31
        MD5_STEP(MD5_F, a, b, c, d, 0, 0xd76aa478, 7)
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
simd-intrinsics.c:125:10: note: expanded from macro 'MD5_STEP'
                a[i] = vroti_epi32(a[i], (s));                                         \
                       ^~~~~~~~~~~~~~~~~~~~~~
./pseudo_intrinsics.h:97:20: note: expanded from macro 'vroti_epi32'
                   : vsriq_n_u32(vshlq_n_u32(x, 32 + (i)), x, -(i)))
                     ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/toolchains/android_6/lib64/clang/6.0.2/include/arm_neon.h:22971:24: note: expanded from macro 'vshlq_n_u32'
  __ret = (uint32x4_t) __builtin_neon_vshlq_n_v((int8x16_t)__s0, __p1, 50); \
                       ^
/opt/toolchains/android_6/lib64/clang/6.0.2/include/arm_neon.h:24656:21: note: expanded from macro 'vsriq_n_u32'
  uint32x4_t __s0 = __p0; \
                    ^~~~

Any ideas?

maroviher avatar Sep 17 '18 20:09 maroviher

Please post theconfig.log and ./configure output. Also, were there any CFLAGS used?

Thanks, -Joe

jbenden avatar Sep 20 '18 17:09 jbenden

No CFLAGS were set. Here are 2 files, as you metioned:

configure.output.txt config.log

maroviher avatar Sep 21 '18 13:09 maroviher

Thanks!

An important observation is the compiler accepts the NEON flag, but issues a warning on it:

clang60++: warning: argument unused during compilation: '-mfpu=neon' [-Wunused-command-line-argument]

Ideally, it would have failed this test.

I'll have to gain access to a suitable machine and see what I can do.

Best regards, -Joe

jbenden avatar Sep 21 '18 16:09 jbenden

I have committed a change that forces Clang to not report success for this test. However, I was using a very, very old version of Clang. Unfortunately, it went on to report other errors.

Therefore, this patch may or may not completely fix the reported issue.

FWIW: On the other hand, it is known that GCC compiles this all just fine.

jbenden avatar Sep 21 '18 17:09 jbenden

But GCC is deprecated in Android NDK: https://github.com/android-ndk/ndk/issues/26

maroviher avatar Sep 21 '18 17:09 maroviher

First see if the change makes a difference, if not we'll figure out the next step.

jbenden avatar Sep 21 '18 17:09 jbenden

FWIW: here is a hack to run foreign architectures on x86. https://rwmj.wordpress.com/2013/12/22/how-to-run-aarch64-binaries-on-an-x86-64-host-using-qemu-userspace-emulation/

Basically, it registers QEMU as the handler when it sees a matching binary file. One caveat is that it will have to be statically linked (since naturally you won't be able to load the foreign libs at runtime)

ChrisLundquist avatar Sep 22 '18 21:09 ChrisLundquist

Hi! I'm also looking into compiling aircrack-ng 1.3 for android arm and aarch64 (for the terminal emulator termux, with clang/android ndk r18) and I get the same error for both architectures. Aircrack-ng 1.2-rc4 compiled without problem so this might be related to the new build system (in some way).

Building on device works on arm but not aarch64, the aarch64 build fails with the same error as when cross compiling, I would guess that NEON_TRUE='#' leads to the build succeeding while NEON_FALSE='#' leads to the build failing.

Applying the patch in https://github.com/aircrack-ng/aircrack-ng/commit/1e5daa4d7afbd3574ac8ff7518e69b1115374a29 silences the configure warnings but it doesn't seem to give a different result. Here's the logs from configure from aarch64, from arm and from cross-compiling targeting aarch64 (with https://github.com/aircrack-ng/aircrack-ng/commit/1e5daa4d7afbd3574ac8ff7518e69b1115374a29 applied): aarch64_config.log aarch64_configure.log arm_config.log arm_configure.log cross_config.log cross_configure.log

Grimler91 avatar Sep 22 '18 22:09 Grimler91

1.2 probably works because the crypto engine stuff isn't there. hashing got notably faster in 1.3.

These particular lines come from JtR and it looks like this issue has been hit before: https://www.openwall.com/lists/john-dev/2015/08/06/3

https://github.com/magnumripper/JohnTheRipper/issues/3284

ChrisLundquist avatar Sep 23 '18 00:09 ChrisLundquist

I now know why this code does not compile; however, I do not have an idea on how to "quickly" fix it.

The snippet complaining is:

#define vroti_epi32(x, i)                                         \
        (i < 0 ? vsliq_n_u32(vshrq_n_u32(x, 32 - i), x, i)        \
               : vsriq_n_u32(vshlq_n_u32(x, 32 + i), x, -i))

These NEON instructions are a variant that require constant values for certain parameters; due to them being directly encoded in the output opcode. While this is not the real problem, the problem is Clang is "compiling" both side of the test expression. In turn, the failing side fails to compile because it's math on the constant parameter DOES exceed the permitted value. However, every other compiler simply takes the working expression side and discards the other.

Therefore, there is no easy fix currently, and this honestly is a variation in Clang - which means it might very well be a compiler bug. While having them fix this would be certainly useful, it does not fix the immediate needs of those wishing to use the Android NDK.

Thoughts, suggestions, and ideas are welcomed, -Joe

jbenden avatar Sep 23 '18 18:09 jbenden

#include <arm_neon.h>

#define vroti_epi32(x, i)                                         \
        (i < 0 ? vsliq_n_u32(vshrq_n_u32(x, 32 - i), x, i)        \
               : vsriq_n_u32(vshlq_n_u32(x, 32 + i), x, -i))

int main()
{
	uint32x4_t x = vdupq_n_u32(42);
	uint32x4_t value = vroti_epi32(x, 12);

	return 0;
}

jbenden@gcc113:~$ 
jbenden@gcc113:~$ clang-3.6 -o example example.c
example.c:10:21: error: argument should be a value from 0 to 31
        uint32x4_t value = vroti_epi32(x, 12);
                           ^
example.c:5:30: note: expanded from macro 'vroti_epi32'
               : vsriq_n_u32(vshlq_n_u32(x, 32 + i), x, -i))
                             ^
/usr/lib/llvm-3.6/bin/../lib/clang/3.6.0/include/arm_neon.h:23019:24: note: expanded from macro 'vshlq_n_u32'
  __ret = (uint32x4_t) __builtin_neon_vshlq_n_v((int8x16_t)__s0, __p1, 50); \
                       ^
/usr/lib/llvm-3.6/bin/../lib/clang/3.6.0/include/arm_neon.h:24704:21: note: expanded from macro 'vsriq_n_u32'
  uint32x4_t __s0 = __p0; \
                    ^
example.c:10:21: error: argument should be a value from 1 to 32
        uint32x4_t value = vroti_epi32(x, 12);
                           ^
example.c:5:18: note: expanded from macro 'vroti_epi32'
               : vsriq_n_u32(vshlq_n_u32(x, 32 + i), x, -i))
                 ^
/usr/lib/llvm-3.6/bin/../lib/clang/3.6.0/include/arm_neon.h:24707:24: note: expanded from macro 'vsriq_n_u32'
  __ret = (uint32x4_t) __builtin_neon_vsriq_n_v((int8x16_t)__s0, (int8x16_t)__s1, __p2, 50); \
                       ^
2 errors generated.
jbenden@gcc113:~$ cat example.c

jbenden avatar Sep 23 '18 18:09 jbenden

One idea for a fix is to create each of these intrinsic instructions as inline static functions; whereby, the inner logic of each function is a giant switch statement for all numbers from 0-31 and 1-32.

In doing so, a similar, but different macro using these new functions would not complain.

Yucky!

jbenden avatar Sep 23 '18 18:09 jbenden

Some testing shows this idea sort of works, but requires a lot of "work-around" functions. It also severely impacts overall performance.

I recommend using the generic + static aircrack-ng build, for now. ./configure --enable-static --disable-shared

Thanks, -Joe

jbenden avatar Sep 23 '18 18:09 jbenden

https://bugs.llvm.org/show_bug.cgi?id=39087

jbenden avatar Sep 26 '18 17:09 jbenden

@jbenden the issue on bugs.llvm.org is still open and someone is commenting on it

ghost avatar Nov 13 '19 12:11 ghost

@etmatrix There are CLANG bugs that prevent this, right now...

jbenden avatar Nov 13 '19 15:11 jbenden

I tried to compile version 1.5.2 on debian 7 wheezy, I got the same issue. Debian 7 use gcc 4.6. So is not a clang bug but a missing feature. Feature introduced in an unknow version of gcc.

make[2]: Entering directory `/root/aircrack-ng/src/aircrack-crypto'
  CC     libaircrack_crypto_la-memory.lo
  CC     libaircrack_crypto_la-sha1-git.lo
  CC     libaircrack_crypto_la-wpapsk.lo
  CC     libaircrack_crypto_la-crypto_engine.lo
  CCLD   libaircrack-crypto.la
  CC     libaircrack_crypto_arm_neon_la-memory.lo
  CC     libaircrack_crypto_arm_neon_la-sha1-git.lo
  CC     libaircrack_crypto_arm_neon_la-simd-intrinsics.lo
/tmp/ccjENqci.s: Assembler messages:
/tmp/ccjENqci.s:22886: Error: co-processor offset out of range
/tmp/ccjENqci.s:22898: Error: co-processor offset out of range
make[2]: *** [libaircrack_crypto_arm_neon_la-simd-intrinsics.lo] Error 1
make[2]: Leaving directory `/root/aircrack-ng/src/aircrack-crypto'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/root/aircrack-ng/src'
make: *** [all-recursive] Error 1

My scope is running aircrack-ng on an old nokia n900 and smartphone with android 8, and the only way is debian chroot.

ghost avatar Nov 24 '19 10:11 ghost

For AppleSilicon look at John the Ripper. Specifically: https://github.com/openwall/john/commit/c9825e688d1fb9fdd8942ceb0a6b4457b0f9f9b4

areese avatar May 07 '21 22:05 areese