clang segmentation fault when testing without OSS-Fuzz integration

[grandnew]

Hi, I tried to test without OSS-Fuzz integration and built simple-example-1 in fuzz-introspector/tests/simple-example-1/work using FuzzIntrospector after building the custom clang:

FUZZ_INTROSPECTOR=1 ../../../build/llvm-build/bin/clang -fsanitize=fuzzer -flto -g ../fuzzer.c -o fuzzer

But clang get error and here is the detailed output:

$ FUZZ_INTROSPECTOR=1 ../../../build/llvm-build/bin/clang -fsanitize=fuzzer -flto -g ../fuzzer.c -o fuzzer -v
clang version 15.0.0 (https://github.com/llvm/llvm-project/ a5258e5b2728e6337462247ba11dcf7af051ae9b)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /root/fuzz-introspector/tests/simple-example-1/work/../../../build/llvm-build/bin
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/8
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Candidate multilib: x32;@mx32
Selected multilib: .;@m64
 "/root/fuzz-introspector/build/llvm-build/bin/clang-15" -cc1 -triple x86_64-unknown-linux-gnu -emit-llvm-bc -flto=full -flto-unit -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name fuzzer.c -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -mllvm -treat-scalable-fixed-error-as-warning -debug-info-kind=constructor -dwarf-version=5 -debugger-tuning=gdb -v -fcoverage-compilation-dir=/root/fuzz-introspector/tests/simple-example-1/work -resource-dir /root/fuzz-introspector/build/llvm-build/lib/clang/15.0.0 -internal-isystem /root/fuzz-introspector/build/llvm-build/lib/clang/15.0.0/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdebug-compilation-dir=/root/fuzz-introspector/tests/simple-example-1/work -ferror-limit 19 -fsanitize-coverage-type=1 -fsanitize-coverage-type=3 -fsanitize-coverage-indirect-calls -fsanitize-coverage-trace-cmp -fsanitize-coverage-inline-8bit-counters -fsanitize-coverage-pc-table -fsanitize-coverage-stack-depth -fsanitize=fuzzer,fuzzer-no-link -fno-builtin-bcmp -fno-builtin-memcmp -fno-builtin-strncmp -fno-builtin-strcmp -fno-builtin-strncasecmp -fno-builtin-strcasecmp -fno-builtin-strstr -fno-builtin-strcasestr -fno-builtin-memmem -fgnuc-version=4.2.1 -fcolor-diagnostics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/fuzzer-890de1.o -x c ../fuzzer.c
clang -cc1 version 15.0.0 based upon LLVM 15.0.0git default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include"
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /root/fuzz-introspector/build/llvm-build/lib/clang/15.0.0/include
 /usr/local/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.
 "/usr/bin/ld" -pie -z relro --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o fuzzer /lib/x86_64-linux-gnu/Scrt1.o /lib/x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/9/crtbeginS.o -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/usr/lib/gcc/x86_64-linux-gnu/9/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib64 -L/lib -L/usr/lib -plugin /root/fuzz-introspector/build/llvm-build/bin/../lib/LLVMgold.so -plugin-opt=mcpu=x86-64 --whole-archive /root/fuzz-introspector/build/llvm-build/lib/clang/15.0.0/lib/x86_64-unknown-linux-gnu/libclang_rt.fuzzer.a --no-whole-archive --whole-archive /root/fuzz-introspector/build/llvm-build/lib/clang/15.0.0/lib/x86_64-unknown-linux-gnu/libclang_rt.fuzzer_interceptors.a --no-whole-archive -lstdc++ --whole-archive /root/fuzz-introspector/build/llvm-build/lib/clang/15.0.0/lib/x86_64-unknown-linux-gnu/libclang_rt.ubsan_standalone.a --no-whole-archive --dynamic-list=/root/fuzz-introspector/build/llvm-build/lib/clang/15.0.0/lib/x86_64-unknown-linux-gnu/libclang_rt.ubsan_standalone.a.syms /tmp/fuzzer-890de1.o --no-as-needed -lpthread -lrt -lm -ldl -lresolv -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-linux-gnu/9/crtendS.o /lib/x86_64-linux-gnu/crtn.o
We are now in the FuzzIntrospector module pass
[Log level 1] : 17:49:53 : Fuzz introspector is running
[Log level 1] : 17:49:53 : Running introspector on ld-temp.o
[Log level 1] : 17:49:53 : This is a fuzzer, performing analysis
[Log level 2] : 17:49:53 : Using default configuration
[Log level 1] : 17:49:53 : Logging next yaml tile to fuzzerLogFile-0-9naMqPai1E.data.yaml
[Log level 1] : 17:49:53 : Wrapping all functions
clang-15: error: unable to execute command: Segmentation fault (core dumped)
clang-15: error: linker command failed due to signal (use -v to see invocation)

Is this error caused by the custom compilation of clang? Thanks!

[DavidKorczynski]

this shouldn't be because of the custom clang -- will take a look at this today after rebuilding my local set up

[grandnew]

@DavidKorczynski Thank you!

[DavidKorczynski]

Am having troubles reproducing this. Although, I used build_all.sh script to build fuzz-introspector, which uses LLVM-14. For your LLVM compilation, did you clone from the latest LLVM?

[grandnew]

Yeah, I cloned the latest LLVM as recommended as follows but failed. https://github.com/ossf/fuzz-introspector/blob/22822c81cf2cfb0f6bd12e06ac8c14266a333ba4/README.md?plain=1#L136-L153 I ran build_all.sh again and succeed. Thanks!

[grandnew]

Btw, the output HTML report gives the detailed information about each fuzzer. How can I get the coverage diff of each pair of fuzzers? For example, which code lines can be only triggered by one of these two fuzzers?

[DavidKorczynski]

How can I get the coverage diff of each pair of fuzzers? For example, which code lines can be only triggered by one of these two fuzzers?

You can't at this moment in time. You can only see what code is statically reachable by a given set of fuzzers (using the All Functions Table at the top of the report). However, this is a neat feature that fits well with this issue:https://github.com/ossf/fuzz-introspector/issues/209 and should be fairly straightforward to implement -- I will see if I can get this done in the near future, unless it's something you'd like to implement? A place where it would be very suitable to implement is in this analysis plugin: https://github.com/ossf/fuzz-introspector/blob/main/post-processing/analyses/fuzz_runtime_coverage_analysis.py