Corrade's test suite fails under AddressSanitizer

[cmey]

Hi! We use Corrade heavily in our project, and while debugging one of our own issue, I noticed Corrade's own test suite fails when enabling AddressSanitizer (ASan).

Steps to repro:

cmake -DBUILD_TESTS=ON -DCMAKE_CXX_FLAGS="-fsanitize=address" ../
cmake --build .
ctest

I see 2 failures:

The following tests FAILED:
	  4 - ContainersArrayViewTest (Failed)
	 23 - ContainersStaticArrayViewTest (Failed)

  FAIL [32] access() at /home/cmey/Code/cpp/wonderland_repos/corrade/src/Corrade/Containers/Test/ArrayViewTest.cpp:814
        Expression data == OneToSeven failed.

  FAIL [12] access() at /home/cmey/Code/cpp/wonderland_repos/corrade/src/Corrade/Containers/Test/StaticArrayViewTest.cpp:426
        Expression data == OneToSeven failed.

on Corrade branch master git # ba75f71, no local change.

This is with clang++:

Ubuntu clang version 14.0.0-1ubuntu1
Target: x86_64-pc-linux-gnu
Thread model: posix

~~Same test failure also with g++ version: g++ (Ubuntu 11.2.0-19ubuntu1) 11.2.0~~ (can't repro on gcc, but still can on clang-14).

Note: the tests pass without AddressSanitizer.

I might take a look later but wanted to report first in case you know a quick fix before I spend too much time on this ;-)

[cmey]

The 2 addresses are indeed the same value:

data:       0x559f60951da0
OneToSeven: 0x559f60951da0

but surprisingly the equality test returns 0. :exploding_head:

[mosra]

Hi, ah, Wonderland, now everything clicks together :)

I don't have Clang 14 on my Arch yet, but I tried with both Clang 13 and GCC 11.2, Debug or Release build, and can't reproduce. Neither it was an issue on the CI ASan build.

I can't help myself but think this is some really serious toolchain issue... and if this is broken, then what else is affected as well? :scream: Is this some new Ubuntu (22.04)? Is it possible it would add some additional default CMAKE_CXX_FLAGS that somehow trigger this? In my past experience it was once doing so, causing problems elsewhere and I had to manually patch those flags away.

Random other idea -- if you do this, does that make it work?

diff --git a/src/Corrade/Containers/Test/ArrayViewTest.cpp b/src/Corrade/Containers/Test/ArrayViewTest.cpp
index 09ce277b..a3d4d910 100644
--- a/src/Corrade/Containers/Test/ArrayViewTest.cpp
+++ b/src/Corrade/Containers/Test/ArrayViewTest.cpp
@@ -811,7 +811,7 @@ void ArrayViewTest::access() {
     constexpr ConstArrayView cb = OneToSeven;
 
     constexpr const int* data = cb.data();
-    CORRADE_VERIFY(data == OneToSeven);
+    CORRADE_VERIFY(data == &OneToSeven[0]);
 
     constexpr std::size_t size = cb.size();
     CORRADE_COMPARE(size, 7);
diff --git a/src/Corrade/Containers/Test/StaticArrayViewTest.cpp b/src/Corrade/Containers/Test/StaticArrayViewTest.cpp
index 0f8eb822..a977ac41 100644
--- a/src/Corrade/Containers/Test/StaticArrayViewTest.cpp
+++ b/src/Corrade/Containers/Test/StaticArrayViewTest.cpp
@@ -423,7 +423,7 @@ void StaticArrayViewTest::access() {
     constexpr ConstStaticArrayView<7> cb = OneToSeven;
 
     constexpr const int* data = cb.data();
-    CORRADE_VERIFY(data == OneToSeven);
+    CORRADE_VERIFY(data == &OneToSeven[0]);
 
     constexpr std::size_t size = cb.size();
     CORRADE_COMPARE(size, 7);

[cmey]

Thank you for the quick response!

Is this some new Ubuntu (22.04)?

Yes, I'm on Pop!_OS 22.04 (a derivative of Ubuntu 22.04).

Is it possible it would add some additional default CMAKE_CXX_FLAGS that somehow trigger this?

CMake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON says the compilation command is exactly:

{                                                                                                                          
  "directory": "/home/cmey/Code/cpp/wonderland_repos/corrade/build-Debug-x86_64/src/Corrade/Containers/Test",              
  "command": "/usr/bin/clang++ -DCORRADE_GRACEFUL_ASSERT -I/home/cmey/Code/cpp/wonderland_repos/corrade/src -I/home/cmey/Code/cpp/wonderland_repos/corrade/build-Debug-x86_64/src -fsanitize=address -std=c++11 -Wall -Wextra -Wold-style-cast -Winit-self -Werror=return-type -Wmissing-declarations -Wpedantic -fvisibility=hidden -fvisibility-inlines-hidden -Wmissing-prototypes -Wno-shorten-64-to-32 -Wunused-member-function -Wno-unneeded-member-function -o CMakeFiles/ContainersArrayViewTest.dir/ArrayViewTest.cpp.o -c /home/cmey/Code/cpp/wonderland_repos/corrade/src/Corrade/Containers/Test/ArrayViewTest.cpp",
  "file": "/home/cmey/Code/cpp/wonderland_repos/corrade/src/Corrade/Containers/Test/ArrayViewTest.cpp"
}

if you do this, does that make it work?

Unfortunately not:

Expression data == &OneToSeven[0] failed.

[mosra]

I don't know what to do, honestly... I could compile Clang 14 here and try with that, but since you say GCC 11.2 fails as well (while it works for me), I don't even see a way how to conditionally disable this test. And if #136 makes even the compilation fail, that's an unworkable state.

Bold suggestion -- is it an option for you to try with some older, more stable compiler / toolchain, until this one becomes less buggy? :sweat_smile:

[cmey]

My turn to apologize, I cannot reproduce the issue on GCC, I've updated the description. I think I didn't have a robust way to change compiler, but now I do.

Here are some more data on different versions of compilers:

gcc-11 (The CXX compiler identification is GNU 11.2.0): rm -rf build && mkdir build && cd build && CC=gcc-11 CXX=g++-11 cmake -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTS=ON -DCMAKE_CXX_FLAGS="-fsanitize=address" ../ && cmake --build . --parallel && ctest PASSES

gcc-10 (The CXX compiler identification is GNU 10.3.0): rm -rf build && mkdir build && cd build && CC=gcc-10 CXX=g++-10 cmake -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTS=ON -DCMAKE_CXX_FLAGS="-fsanitize=address" ../ && cmake --build . --parallel && ctest PASSES

clang-14 (The CXX compiler identification is Clang 14.0.0): rm -rf build && mkdir build && cd build && CC=clang-14 CXX=clang++-14 cmake -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTS=ON -DCMAKE_CXX_FLAGS="-fsanitize=address" ../ && cmake --build . --parallel && ctest

The following tests FAILED:
	  4 - ContainersArrayViewTest (Failed)
	 23 - ContainersStaticArrayViewTest (Failed)

ctest --rerun-failed --output-on-failure

  FAIL [32] access() at /home/cmey/Code/cpp/wonderland_repos/corrade/src/Corrade/Containers/Test/ArrayViewTest.cpp:814
        Expression data == OneToSeven failed.

  FAIL [12] access() at /home/cmey/Code/cpp/wonderland_repos/corrade/src/Corrade/Containers/Test/StaticArrayViewTest.cpp:426
        Expression data == OneToSeven failed.

clang-13 (The CXX compiler identification is Clang 13.0.1): rm -rf build && mkdir build && cd build && CC=clang-13 CXX=clang++-13 cmake -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTS=ON -DCMAKE_CXX_FLAGS="-fsanitize=address" ../ && cmake --build . --parallel && ctest

The following tests FAILED:
	 89 - UtilityPathTest (Failed)
	110 - UtilityDirectoryTest (Failed)

ctest --rerun-failed --output-on-failure

  FAIL [044] libraryLocation() at /home/cmey/Code/cpp/wonderland_repos/corrade/src/Corrade/Utility/Test/PathTest.cpp:1316
        Expression libraryLocation != Path::executableLocation() failed.

  FAIL [037] libraryLocation() at /home/cmey/Code/cpp/wonderland_repos/corrade/src/Corrade/Utility/Test/DirectoryTest.cpp:1029
        Expression libraryLocation != Directory::executableLocation() failed.

clang-12 (The CXX compiler identification is Clang 12.0.1): rm -rf build && mkdir build && cd build && CC=clang-12 CXX=clang++-12 cmake -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTS=ON -DCMAKE_CXX_FLAGS="-fsanitize=address" ../ && cmake --build . --parallel && ctest

The following tests FAILED:
	 89 - UtilityPathTest (Failed)
	110 - UtilityDirectoryTest (Failed)

ctest --rerun-failed --output-on-failure

  FAIL [044] libraryLocation() at /home/cmey/Code/cpp/wonderland_repos/corrade/src/Corrade/Utility/Test/PathTest.cpp:1316
        Expression libraryLocation != Path::executableLocation() failed.

  FAIL [037] libraryLocation() at /home/cmey/Code/cpp/wonderland_repos/corrade/src/Corrade/Utility/Test/DirectoryTest.cpp:1029
        Expression libraryLocation != Directory::executableLocation() failed.

clang-11 (The CXX compiler identification is Clang 11.1.0): rm -rf build && mkdir build && cd build && CC=clang-11 CXX=clang++-11 cmake -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTS=ON -DCMAKE_CXX_FLAGS="-fsanitize=address" ../ && cmake --build . --parallel && ctest Fails to even build:

[ 18%] Built target corrade-rc
[ 19%] Compiling data resource file /home/cmey/Code/cpp/wonderland_repos/corrade/build/src/Corrade/PluginManager/Test/init-fini/resource_InitFiniStatic.cpp
==592315==ERROR: AddressSanitizer failed to allocate 0x0 (0) bytes of SetAlternateSignalStack (error code: 22)
...
==593223==AddressSanitizer CHECK failed: /build/llvm-toolchain-11-mnvtwk/llvm-toolchain-11-11.1.0/compiler-rt/lib/sanitizer_common/sanitizer_common.cpp:54 "((0 && "unable to mmap")) != (0)" (0x0, 0x0)
    <empty stack>

So it appears that this problem only shows up with the Clang compiler, on all the last few major versions of it. It is "funny" that 2 different tests fail depending on the version of clang.

[cmey]

The problem is reproducible in a clean Docker environment:

docker run -it archlinux

pacman -Sy clang cmake git make
git clone https://github.com/mosra/corrade.git
cd corrade
rm -rf build && mkdir build && cd build && CC=clang CXX=clang++ cmake -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTS=ON -DCMAKE_CXX_FLAGS="-fsanitize=address" ../ && cmake --build . --parallel && ctest

# The CXX compiler identification is Clang 13.0.1

The following tests FAILED:
	 89 - UtilityPathTest (Failed)
	110 - UtilityDirectoryTest (Failed)

[mosra]

Nice, thanks, that reduces it quite a bit.

• It seems that Clang 14 is the odd one out. I looked through both https://github.com/google/sanitizers/issues?q=is%3Aissue and https://github.com/llvm/llvm-project/issues?q=label%3Acompiler-rt%3Aasan+ but couldn't find anything related. Once I get Clang 14 in Arch, I'll look closer, until then .. is it possible for you to just ignore the problem? :stuck_out_tongue_winking_eye:
• The Clang 11 error seems to be the same as https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100114 (old sanitizer library and new glibc). It fails during build because the tests cause corrade-rc to be built during the build process. (With BUILD_TESTS=OFF, you'd get this error when running corrade-rc after.) Could be that the Clang 11 version in Ubuntu doesn't yet have the patch that fixed it for GCC.
• Clang 12 and 13 have some issue with libraryLocation(). Apparently I already have an XFAIL for certain glibc versions there, so I suppose this is a similar (strange) case: https://github.com/mosra/corrade/blob/c5102b37cb1844637b942c8ff67baf54c4d7a418/src/Corrade/Utility/Test/PathTest.cpp#L1309-L1317 I can't reproduce locally on Arch, but I also have an uptime of 63 days, so it's very likely that once I finally get to updating and rebooting this machine, I'll get the same :D

FYI, for robust compiler changes with CMake it should be enough to pass CMAKE_CXX_COMPILER instead of setting CXX. CMake looks into the CXX variable only during the very first run, and then it uses to detect various features, implicit include paths and whatnot -- meaning, changing a compiler isn't as easy as a lot of things depend on it, thus it doesn't even check CXX again after. But explicitly forcing a different CMAKE_CXX_COMPILER will cause CMake to drop everything and start over, which practically is the same as rm -r build.

[mosra]

I started hitting the libraryLocation() test failure today across all compilers after updating the CI from 16.04 to 18.04. Investigated some more and fixed that in 411d6f45f189a96728759b6cabda683fd4fed556.

What's left here is the Clang 14 error.

[mosra]

I have to admit I forgot this issue existed, heh, sorry.

But yesterday I got Clang 15 and thus was able to test with both 14 and 15. Fortunately it's fixed again in version 15, and version 14 behaves in a very weird way, failing for data() but not for begin() which returns the exact same thing. Anyway, a XFAIL for Clang 14 + ASan is added in 47773538003a957db571bbcc3f762cd533192c6f, and so the tests should pass now, on all Clang versions.