fix arithmetic on a pointer to an incomplete type

[juchem]

When including bcc/BPF.h there's a compilation error regarding arithmetic on a pointer to an incomplete type 'ebpf::USDT'.

The fix is straightforward: swap the order of the class declarations in the bcc/BPF.h file, after which things compile successfully.

More details are described in the issue #4160

[davemarchevsky]

could you rebase on top of latest master? Some test fixes were recently shipped, should result in tests having better signal for this PR after rebase

[yonghong-song]

Your change sound okay. But I want to understand which flag caused compilation failure. I tried with bcc examples/cpp/HelloWorld.cc with the following scripts (derived from CMakefile file flags.make

[$ ~/work/bcc/build/examples/cpp/CMakeFiles/HelloWorld.dir] cat run.sh
# CMAKE generated file: DO NOT EDIT!
# Generated by "Unix Makefiles" Generator, CMake Version 3.20

# compile CXX with /home/yhs/work/gcc2/gcc12-1/install/bin/g++
CXX_DEFINES="-DHAVE_LIBDEBUGINFOD"
CXX_INCLUDES="-I/usr/src/kernels/5.12.0-0_fbk4_clang_4620_gc9859290c7cd/include/generated/uapi -I/home/yhs/work/llvm-project/llvm/build/install/include/../tools/clang/include -I/home/yhs/work/bcc/build/src/cc -I/home/yhs/work/bcc/src/cc -I/home/yhs/work/bcc/src/cc/api -I/home/yhs/work/bcc/src/cc/libbpf/include/uapi"
CXX_FLAGS="-Wall  -fPIC -O3 -DNDEBUG -std=c++2b"
/home/yhs/work/gcc2/gcc12-1/install/bin/g++ ${CXX_DEFINES} ${CXX_INCLUDES} \
        -I/home/yhs/work/bcc/build/src/cc/ \
        -I/home/yhs/work/bcc/src/cc \
        -I/home/yhs/work/bcc/src/cc/api \
        $CXX_FLAGS /home/yhs/work/bcc/examples/cpp/HelloWorld.cc -c
[$ ~/work/bcc/build/examples/cpp/CMakeFiles/HelloWorld.dir] ./run.sh
[$ ~/work/bcc/build/examples/cpp/CMakeFiles/HelloWorld.dir]

In HelloWorld.cc, we do have BPF class constructor:

...
int main() {
  ebpf::BPF bpf;
...

Could you help find out which flags caused compilation error? Does '-std=c++2a' or '-std=c++17' also cause issue?

[juchem]

@yonghong-song of course, I'll run some tests here. @davemarchevsky yes. Will do that.

[juchem]

@yonghong-song it looks like it's the C++ standard used. I ran a simple script to compile an empty main with different C++ standards on clang 14 and 15. It worked with C++ up to 17 (script at the bottom):

$ ./bcc.sh
************** SUCCESS:  /usr/bin/c++ [c++14]
************** SUCCESS:  /usr/bin/clang++-15 [c++14]
************** SUCCESS:  /usr/bin/c++ [c++17]
************** SUCCESS:  /usr/bin/clang++-15 [c++17]
In file included from /tmp/bcc-XgoZX.cpp:1:
In file included from /usr/local/include/bcc/BPF.h:25:
In file included from /usr/local/include/bcc/BPFTable.h:27:
In file included from /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/vector:64:
/usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_vector.h:367:35: error: arithmetic on a pointer to an incomplete type 'ebpf::USDT'
        _M_impl._M_end_of_storage - _M_impl._M_start);
        ~~~~~~~~~~~~~~~~~~~~~~~~~ ^
/usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_vector.h:526:7: note: in instantiation of member function 'std::_Vector_base<ebpf::USDT, std::allocator<ebpf::USDT>>::~_Vector_base' requested here
      vector() = default;
      ^
/usr/local/include/bcc/BPF.h:59:50: note: in defaulted default constructor for 'std::vector<ebpf::USDT>' first required here
                   const std::vector<USDT>& usdt = {});
                                                 ^
/usr/local/include/bcc/BPF.h:43:7: note: forward declaration of 'ebpf::USDT'
class USDT;
      ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
2 errors generated.

************** FAILED:  /usr/bin/c++ [c++20]
In file included from /tmp/bcc-XgoZX.cpp:1:
In file included from /usr/local/include/bcc/BPF.h:25:
In file included from /usr/local/include/bcc/BPFTable.h:27:
In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/vector:64:
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_vector.h:367:35: error: arithmetic on a pointer to an incomplete type 'ebpf::USDT'
        _M_impl._M_end_of_storage - _M_impl._M_start);
        ~~~~~~~~~~~~~~~~~~~~~~~~~ ^
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_vector.h:526:7: note: in instantiation of member function 'std::_Vector_base<ebpf::USDT, std::allocator<ebpf::USDT>>::~_Vector_base' requested here
      vector() = default;
      ^
/usr/local/include/bcc/BPF.h:59:50: note: in defaulted default constructor for 'std::vector<ebpf::USDT>' first required here
                   const std::vector<USDT>& usdt = {});
                                                 ^
/usr/local/include/bcc/BPF.h:43:7: note: forward declaration of 'ebpf::USDT'
class USDT;
      ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
2 errors generated.

************** FAILED:  /usr/bin/clang++-15 [c++20]
In file included from /tmp/bcc-XgoZX.cpp:1:
In file included from /usr/local/include/bcc/BPF.h:25:
In file included from /usr/local/include/bcc/BPFTable.h:27:
In file included from /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/vector:64:
/usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_vector.h:367:35: error: arithmetic on a pointer to an incomplete type 'ebpf::USDT'
        _M_impl._M_end_of_storage - _M_impl._M_start);
        ~~~~~~~~~~~~~~~~~~~~~~~~~ ^
/usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_vector.h:526:7: note: in instantiation of member function 'std::_Vector_base<ebpf::USDT, std::allocator<ebpf::USDT>>::~_Vector_base' requested here
      vector() = default;
      ^
/usr/local/include/bcc/BPF.h:59:50: note: in defaulted default constructor for 'std::vector<ebpf::USDT>' first required here
                   const std::vector<USDT>& usdt = {});
                                                 ^
/usr/local/include/bcc/BPF.h:43:7: note: forward declaration of 'ebpf::USDT'
class USDT;
      ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
2 errors generated.

************** FAILED:  /usr/bin/c++ [c++2a]
In file included from /tmp/bcc-XgoZX.cpp:1:
In file included from /usr/local/include/bcc/BPF.h:25:
In file included from /usr/local/include/bcc/BPFTable.h:27:
In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/vector:64:
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_vector.h:367:35: error: arithmetic on a pointer to an incomplete type 'ebpf::USDT'
        _M_impl._M_end_of_storage - _M_impl._M_start);
        ~~~~~~~~~~~~~~~~~~~~~~~~~ ^
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_vector.h:526:7: note: in instantiation of member function 'std::_Vector_base<ebpf::USDT, std::allocator<ebpf::USDT>>::~_Vector_base' requested here
      vector() = default;
      ^
/usr/local/include/bcc/BPF.h:59:50: note: in defaulted default constructor for 'std::vector<ebpf::USDT>' first required here
                   const std::vector<USDT>& usdt = {});
                                                 ^
/usr/local/include/bcc/BPF.h:43:7: note: forward declaration of 'ebpf::USDT'
class USDT;
      ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
2 errors generated.

************** FAILED:  /usr/bin/clang++-15 [c++2a]
In file included from /tmp/bcc-XgoZX.cpp:1:
In file included from /usr/local/include/bcc/BPF.h:25:
In file included from /usr/local/include/bcc/BPFTable.h:27:
In file included from /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/vector:64:
/usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_vector.h:367:35: error: arithmetic on a pointer to an incomplete type 'ebpf::USDT'
        _M_impl._M_end_of_storage - _M_impl._M_start);
        ~~~~~~~~~~~~~~~~~~~~~~~~~ ^
/usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_vector.h:526:7: note: in instantiation of member function 'std::_Vector_base<ebpf::USDT, std::allocator<ebpf::USDT>>::~_Vector_base' requested here
      vector() = default;
      ^
/usr/local/include/bcc/BPF.h:59:50: note: in defaulted default constructor for 'std::vector<ebpf::USDT>' first required here
                   const std::vector<USDT>& usdt = {});
                                                 ^
/usr/local/include/bcc/BPF.h:43:7: note: forward declaration of 'ebpf::USDT'
class USDT;
      ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
2 errors generated.

************** FAILED:  /usr/bin/c++ [c++2b]
In file included from /tmp/bcc-XgoZX.cpp:1:
In file included from /usr/local/include/bcc/BPF.h:25:
In file included from /usr/local/include/bcc/BPFTable.h:27:
In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/vector:64:
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_vector.h:367:35: error: arithmetic on a pointer to an incomplete type 'ebpf::USDT'
        _M_impl._M_end_of_storage - _M_impl._M_start);
        ~~~~~~~~~~~~~~~~~~~~~~~~~ ^
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_vector.h:526:7: note: in instantiation of member function 'std::_Vector_base<ebpf::USDT, std::allocator<ebpf::USDT>>::~_Vector_base' requested here
      vector() = default;
      ^
/usr/local/include/bcc/BPF.h:59:50: note: in defaulted default constructor for 'std::vector<ebpf::USDT>' first required here
                   const std::vector<USDT>& usdt = {});
                                                 ^
/usr/local/include/bcc/BPF.h:43:7: note: forward declaration of 'ebpf::USDT'
class USDT;
      ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
2 errors generated.

************** FAILED:  /usr/bin/clang++-15 [c++2b]

driver script:

#!/bin/bash

source="$(mktemp -t bcc-XXXXX.cpp)"
cat <<EOF > "${source}"
#include <bcc/BPF.h>
int main() { return 0; }
EOF

for standard in c++14 c++17 c++20 c++2a c++2b; do
  for compiler in /usr/bin/c++ /usr/bin/clang++-15; do
    "${compiler}" \
      -D__STRICT_ANSI__ \
      -std="${standard}" \
      -static \
      -Wall \
      -Werror \
      -Wno-bitwise-instead-of-logical \
      -Wno-comment \
      -Wno-gnu-string-literal-operator-template \
      -Wno-overlength-strings \
      -Wno-pointer-bool-conversion \
      -Wshadow-field-in-constructor \
      -Wshadow-field-in-constructor-modified \
      -Wshadow-ivar \
      -Wshadow-uncaptured-local \
      -Wundef \
      -x c++ \
      -save-temps=obj \
      -ggdb3 \
      -O1 \
      -Wno-unused-const-variable \
      -Wno-unused-local-typedef \
      -Wno-unused-private-field \
      -ferror-limit=1 \
      -o "${source}.o" \
      -c "${source}" \
      && echo -e "\n************** SUCCESS:  ${compiler} [${standard}]" \
      || echo -e "\n************** FAILED:  ${compiler} [${standard}]"
  done
done

[yonghong-song]

@juchem thanks for the script and explanation. For your script, my locally self build gcc 12.1 still cannot reproduce the issue. Maybe there are some config options I didn't use. Any way, you change looks good to me.

[juchem]

@yonghong-song I'm sorry for having failed to mention. /usr/bin/c++ points to self-built Clang 14.0.6. I've also tried Debian provided Clang 15. Both reproduced the error. I haven't tried GCC. Both use Debian provided libstdc++-12.

[yonghong-song]

I just tried with clang14 and latest development clang16, both works fine without failure even for c++2b. I guess my build might have different cmake options from debian.