LAF/split compares works poorly with AFL_LLVM_INSTRUMENT=PCGUARD

[vegard]

IMPORTANT

1. You have verified that the issue to be present in the current dev branch. ✅
2. Please supply the command line options and relevant environment variables, e.g., a copy-paste of the contents of out/default/fuzzer_setup. ✅

Thank you for making AFL++ better!

Describe the bug When building a simple program with AFL_LLVM_LAF_ALL=1 afl-clang-fast, the fuzzer doesn't find any new paths. Switching to AFL_LLVM_INSTRUMENT=CLASSIC makes it find new paths almost immediately (and also finds the crashing input fairly quickly).

To Reproduce Steps to reproduce the behavior:

1. Test program:

#include <stdlib.h>
#include <unistd.h>

int main(int argc, char *argv[])
{
        unsigned int x = 0;
        read(STDIN_FILENO, &x, sizeof(x));
        if (x == 0x12345678)
                abort();

        return 0;
}

2. Build with AFL_LLVM_LAF_ALL=1 afl-clang-fast test.c (optimization level does not seem to matter).

$ AFL_LLVM_LAF_ALL=1 AFLplusplus/afl-clang-fast test.c 
afl-cc++4.06a by Michal Zalewski, Laszlo Szekeres, Marc Heuse - mode: LLVM-PCGUARD
Running split-switches-pass by [email protected]
Running compare-transform-pass by [email protected], extended by [email protected]
Split-compare-newpass by [email protected], extended by [email protected] (splitting icmp to 8 bit)
Split-floatingpoint-compare-pass: 0 FP comparisons splitted
1 comparisons found
SanitizerCoveragePCGUARD++4.06a
[+] Instrumented 3 locations with no collisions (non-hardened mode) of which are 0 handled and 0 unhandled selects.

3. echo x > in/x
4. afl-fuzz -i in -o out ./a.out
5. afl-fuzz will not find any new paths in 5 minutes, corpus count remains at 1.

If I build with AFL_LLVM_INSTRUMENT=CLASSIC then new paths are found almost immediately, and the crash (abort()) is found within around 2 minutes:

$ AFL_LLVM_INSTRUMENT=CLASSIC AFL_LLVM_LAF_ALL=1 AFLplusplus/afl-clang-fast test.c 
afl-cc++4.06a by Michal Zalewski, Laszlo Szekeres, Marc Heuse - mode: LLVM-CLASSIC
Running split-switches-pass by [email protected]
Running compare-transform-pass by [email protected], extended by [email protected]
Split-compare-newpass by [email protected], extended by [email protected] (splitting icmp to 8 bit)
Split-floatingpoint-compare-pass: 0 FP comparisons splitted
1 comparisons found
afl-llvm-pass++4.06a by <[email protected]> and <[email protected]>
afl-llvm-pass++4.06a using non-thread safe instrumentation
[+] Instrumented 8 locations (non-hardened mode, ratio 100%).

Expected behavior I expect the default instrumentation mode to fairly quickly find the same paths that are found with the CLASSIC instrumentation.

Screen output/Screenshots If applicable, add copy-paste of the screen output or screenshot that shows the issue. Please ensure the output is in English and not in Chinese, Russian, German, etc.

Additional context

Revision:

$ git rev-parse HEAD
5163a49350ed17149cb3c52bc79bd87e86402510

[+] llvm_mode detected llvm 10+, enabling neverZero implementation and c++14
[+] llvm_mode detected llvm 11+, enabling afl-lto LTO implementation
GNUmakefile.llvm:228: ld.lld not found, cannot enable LTO mode
[+] shmat seems to be working.
[*] Checking for working 'llvm-config'...
[*] Checking for working '/usr/lib/llvm-13/bin/clang'...
[*] Checking for matching versions of '/usr/lib/llvm-13/bin/clang' and 'llvm-config'
[*] We have llvm-config version 13.0.0 with a clang version 13.0.0, good.
[*] Checking for './afl-showmap'...
[+] All set and ready to build.

Build Summary:
[+] afl-fuzz and supporting tools successfully built
[+] LLVM basic mode successfully built
[+] LLVM mode successfully built
[-] LLVM LTO mode could not be build, it is optional, if you want it, please install LLVM 11-14. More information at instrumentation/README.lto.md on how to build it
[-] gcc_mode could not be built, it is optional, install gcc-VERSION-plugin-dev to enable this

out/default/fuzzer_setup:

# environment variables:
# command line:
'AFLplusplus/afl-fuzz' '-i' 'in' '-o' 'out' './a.out'

$ AFLplusplus/afl-clang-fast --version
afl-cc++4.06a by Michal Zalewski, Laszlo Szekeres, Marc Heuse - mode: LLVM-PCGUARD
Ubuntu clang version 13.0.0-++20210327080829+e5f2898bc751-1~exp1~20210327192522.3607
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm-13/bin

[vanhauser-thc]

$ AFL_LLVM_LAF_ALL=1 AFLplusplus/afl-clang-fast test.c 
afl-cc++4.06a by Michal Zalewski, Laszlo Szekeres, Marc Heuse - mode: LLVM-PCGUARD
...
[+] Instrumented 3 locations with no collisions (non-hardened mode) of which are 0 handled and 0 unhandled selects.

you can see it is going wrong because it sees only 3 locations.

$ AFL_LLVM_INSTRUMENT=CLASSIC AFL_LLVM_LAF_ALL=1 AFLplusplus/afl-clang-fast test.c 
afl-cc++4.06a by Michal Zalewski, Laszlo Szekeres, Marc Heuse - mode: LLVM-CLASSIC
...
[+] Instrumented 8 locations (non-hardened mode, ratio 100%).

this looks good instead.

For me it is working though:

$ AFL_LLVM_LAF_ALL=1 afl-clang-fast -o test test.c
afl-cc++4.06a by Michal Zalewski, Laszlo Szekeres, Marc Heuse - mode: LLVM-PCGUARD
Running split-switches-pass by [email protected]
Running compare-transform-pass by [email protected], extended by [email protected]
Split-compare-newpass by [email protected], extended by [email protected] (splitting icmp to 8 bit)
Split-floatingpoint-compare-pass: 0 FP comparisons splitted
1 comparisons found
SanitizerCoveragePCGUARD++4.06a
[+] Instrumented 8 locations with no collisions (non-hardened mode) of which are 0 handled and 0 unhandled selects.

why this is different for you - I do not know. you are running a current afl++ so that is not it. llvm/clang is known to be difficult, so maybe it is that. can you try changing the version? e.g. trying llvm 14?

and maybe can you please give the output of your two compile commands but adding the -v switch?

[vegard]

PCGUARD.txt CLASSIC.txt

The only significant difference I can see is

--fpass-plugin=AFLplusplus/SanitizerCoveragePCGUARD.so
+-fpass-plugin=AFLplusplus/afl-llvm-pass.so

This is admittedly an old system, as you can see from using gcc 9. Not sure if I can use LLVM14 on this.

[vanhauser-thc]

sure llvm 14 can installed anyhere, see https://apt.llvm.org/

[vegard]

I believe you :-) It's more that I'm wary of messing with this system. I can try to reproduce in a snapshot/container/whatever, but it will likely take me some days to get to it. Thanks for looking at it so far.