test-wpapsk-cmac regression on arm64/armhf/armel

[samueloph]

I've recently packaged a git snapshot of aircrack-ng for Debian (at the current HEAD https://github.com/aircrack-ng/aircrack-ng/commit/4bf83f1a7631d6721c17a35597d2f71b35ff30bf), and stumbled upon failures when building for arm64/armhf/armel.

The check target fails when running the test test-wpapsk-cmac.

I've ran git bisect on the tree and found the commit that introduces the regression: https://github.com/aircrack-ng/aircrack-ng/commit/5e8ed78e8abdb0baf5b78d42e6be4c752b512a35

Looking at the current open issues/PRs, I can see one that might be related to this issue, but I haven't tested it: https://github.com/aircrack-ng/aircrack-ng/issues/2553 https://github.com/aircrack-ng/aircrack-ng/pull/2554

Contents of test-suite.log (only the important bits)

FAIL: test-wpapsk-cmac
======================

1..2
not ok 1 - test_crypto_engine_generic
# difference at offset 0 0x97 0x2c
# difference at offset 1 0xd8 0x76
# difference at offset 2 0x90 0xdc
# difference at offset 3 0xb4 0x59
# difference at offset 4 0xfc 0x2c
# difference at offset 5 0x65 0x3b
# difference at offset 6 0xf6 0x67
# difference at offset 7 0x5d 0x1b
# difference at offset 8 0x10 0xac
# difference at offset 9 0xbe 0x23
# difference at offset 10 0x4c 0x0f
# difference at offset 11 0x11 0x6c
# difference at offset 12 0x92 0x9e
# difference at offset 13 0x13 0x38
# difference at offset 14 0x78 0xa0
# difference at offset 15 0x27 0x62
# ...
# 31 bytes of 0xaaab033cf920 and 0xaaaadf5d3f68 differ
# test/unit/test-wpapsk-cmac.c:174: error: Failure!
not ok 2 - test_crypto_engine_arm_neon
# difference at offset 0 0x97 0x2c
# difference at offset 1 0xd8 0x76
# difference at offset 2 0x90 0xdc
# difference at offset 3 0xb4 0x59
# difference at offset 4 0xfc 0x2c
# difference at offset 5 0x65 0x3b
# difference at offset 6 0xf6 0x67
# difference at offset 7 0x5d 0x1b
# difference at offset 8 0x10 0xac
# difference at offset 9 0xbe 0x23
# difference at offset 10 0x4c 0x0f
# difference at offset 11 0x11 0x6c
# difference at offset 12 0x92 0x9e
# difference at offset 13 0x13 0x38
# difference at offset 14 0x78 0xa0
# difference at offset 15 0x27 0x62
# ...
# 31 bytes of 0xaaab033d29e0 and 0xaaaadf5d3f68 differ
# test/unit/test-wpapsk-cmac.c:174: error: Failure!
# not ok - tests
FAIL test-wpapsk-cmac (exit status: 2)

Debian build logs

https://buildd.debian.org/status/package.php?p=aircrack-ng https://buildd.debian.org/status/fetch.php?pkg=aircrack-ng&arch=arm64&ver=1%3A1.7%2Bgit20230807.4bf83f1a-1&stamp=1691438883&raw=0 https://buildd.debian.org/status/fetch.php?pkg=aircrack-ng&arch=armel&ver=1%3A1.7%2Bgit20230807.4bf83f1a-1&stamp=1691439285&raw=0 https://buildd.debian.org/status/fetch.php?pkg=aircrack-ng&arch=armhf&ver=1%3A1.7%2Bgit20230807.4bf83f1a-1&stamp=1691439374&raw=0

Script used to bisect

#!/bin/bash

untestable() {
    git reset --hard
    exit 125
}

fail() {
    git reset --hard
    exit 1
}

autoreconf -fiv || untestable
./configure \
    --with-gcrypt \
    --with-experimental || untestable

make -j4 || untestable

# Apply https://github.com/aircrack-ng/aircrack-ng/pull/2563 to
# add missing import.
sed -i '1s/^/#include <inttypes.h>\n/' src/airodump-ng/airodump-ng.c || untestable

# Modify makefile to only run test-wpapsk-cmac on make check.
sed -i -e "s|#am__append_1 = -liphlpapi -lsetupapi|check_PROGRAMS =  \$(am__EXEEXT_5)\nTESTS = \$(am__EXEEXT_5)\n#am__append_1 = -liphlpapi -lsetupapi|g" Makefile || untestable

# Run tests a few times to not get tricked by an issue that
# might only show up on some runs (due to parallelism).
make -j4 check || fail
touch test/unit/test-wpapsk-cmac.c
make -j4 check || fail
touch test/unit/test-wpapsk-cmac.c
make -j4 check || fail
touch test/unit/test-wpapsk-cmac.c
make -j4 check || fail
touch test/unit/test-wpapsk-cmac.c
make -j4 check || fail
touch test/unit/test-wpapsk-cmac.c

# If we got here, success.
git reset --hard
exit 0

[samueloph]

I tested the patch at https://github.com/aircrack-ng/aircrack-ng/pull/2554 and the issue persists, so this is a different issue.

[samueloph]

aircrack-ng will be dropped from Debian testing tomorrow due to this issue (in case anyone finds this bug when looking for the package).

Bug on the Debian side: https://bugs.debian.org/1051201

[bdrung]

The failure can also be seen on Ubuntu: https://bugs.launchpad.net/ubuntu/+source/aircrack-ng/+bug/2039967

[Mister-X-]

How come it doesn't happen when compiling for docker for these CPU architectures?

[samueloph]

@Mister-X- The issue doesn't trigger on every build, I did 5 builds in a row when bisecting the regression. Maybe the reason it didn't fail was because of luck? Or it could also be a lower parallelization factor? I couldn't find the logs for the docker arm builds to check.