googletest icon indicating copy to clipboard operation
googletest copied to clipboard

[Bug]: Flaky infinite `AddressSanitizer:DEADLYSIGNAL` loop when using gtest in github CI

Open Strilanc opened this issue 1 year ago • 0 comments

Describe the issue

Recently I started noticing my github CI builds hanging. They were getting into an infinite loop of printing out AddressSanitizer:DEADLYSIGNAL. Initially I assumed this was my code. I didn't know where it was so I started gradually deleting the code base while preserving the failure, until I had no code and no tests left (here is the PR where I did this). All that's left is building gtest, and running it with no tests. And I still see around 10% to 20% of builds hanging with AddressSanitizer:DEADLYSIGNAL loops:

image

So it appears the issue is something to do with the very initial work that gtest's main method is doing, before executing any tests.

Steps to reproduce the problem

I don't know how to reproduce this on my machine. But I can trigger it in github actions.

  1. Create an empty github repository with a branch named main.

  2. Create a pull request to add this .github/workflows/ci.yml file:

name: ci
on:
  pull_request:
    branches:
      - main
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        attempt: [0, 1, 2, 3, 4, 5, 6, 7, 8]
    steps:
    - uses: actions/checkout@v1
    - run: |
        cd ..
        git clone https://github.com/google/googletest.git -b v1.14.0
        mkdir googletest/build && cd googletest/build
        cmake .. -DBUILD_GMOCK=OFF
        make
        sudo make install
    - run: g++ -fsanitize=address -o stim_test /usr/local/lib/libgtest_main.a /usr/local/lib/libgtest.a
    - run: ./stim_test
  1. Check whether any of the 9 instances of the test action end up hanging. If none hang, re-run the checks. This is a flaky failure so you may need to try this a few times.

What version of GoogleTest are you using?

I've seen this failure both in 1.12.1 and on 1.14.0

Since this failure appears in such an older version, and it only recently started happening, it was likely triggered by a change in github side or in g++ or in address sanitization tooling rather than on the gtest side.

What operating system and version are you using?

A github actions environment using ubuntu-latest.

What compiler and version are you using?

Based on g++ -v it's 11.4.0:

Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x8[6](https://github.com/quantumlib/Stim/actions/runs/8303532921/job/22727820559#step:4:7)_64-linux-gnu/11/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa
OFFLOAD_TARGET_DEFAULT=1
Target: x[8](https://github.com/quantumlib/Stim/actions/runs/8303532921/job/22727820559#step:4:9)6_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 11.4.0-1ubuntu1~22.04' --with-bugurl=file:///usr/share/doc/gcc-11/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-11 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --enable-cet --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-11-XeT[9](https://github.com/quantumlib/Stim/actions/runs/8303532921/job/22727820559#step:4:10)lY/gcc-[11](https://github.com/quantumlib/Stim/actions/runs/8303532921/job/22727820559#step:4:12)-11.4.0/debian/tmp-nvptx/usr,amdgcn
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04)

What build system are you using?

A single g++ command.

g++ -fsanitize=address -o stim_test /usr/local/lib/libgtest_main.a /usr/local/lib/libgtest.a

Additional context

No response

Strilanc avatar Mar 15 '24 23:03 Strilanc