Finding a kernel that qsym works on

[jonathanmetzman]

I'm trying to find a kernel qsym works on so I can change the image used by our experiments for a one-off experiment that will run qsym properly. I thought that centos7 (kernel: 3.10.0-1062.18.1.el7.x86_64) would work but it doesn't seem to, the qsym queue is still empty. Maybe there's an issue with running qsym in a docker image? WDYT @wideglide ?

[wideglide]

Centos7 is what I have used and has always been stable. I'll pull the docker image and test.

[wideglide]

Where is the Dockerfile?
Are there updated builds? This is what I found (gcr.io/fuzzbench/runners/qsym/freetype2-2017)

[jonathanmetzman]

Yeah I tried gcr.io/fuzzbench/runners/qsym/freetype2-2017

There's a bunch of dockerfiles, but here are the qsym dockerfiles used for that image

If you're using google cloud, I used this command to create the instance: gcloud compute instances create instance-d --image-family=centos-7 --image-project=gce-uefi-images --zone=us-central1-c --scopes=cloud-platform --no-address --boot-disk-size=30GB --custom-memory=12GB --custom-cpu=2

[jonathanmetzman]

Note that the image is ubuntu, but the host is centos-7 so I assumed pin which failed because of the kernel should now work.

[wideglide]

On a local server it seems to be working, but with freetype2 it is not immediately finding solutions.

DEBUG:qsym.afl:Run qsym: input=/out/corpus/afl-slave/queue/id:000769,src:000013,op:havoc,rep:128,+cov
DEBUG:qsym.Executor:Executing timeout -k 5 90 /usr/local/lib/python2.7/dist-packages/qsym/../../../../third_party/pin-2.14-71313-gcc.4.4.7-linux/pin.sh -ifeellucky -t /usr/local/lib/python2.7/dist-packages/qsym/pintool/obj-intel64/libqsym.so -logfile /tmp/tmpZ_7mMx/qsym-out-1/pin.log -i /out/corpus/qsym/.cur_input -s 1 -o /tmp/tmpZ_7mMx/qsym-out-1 -l 1 -b /out/corpus/qsym/bitmap -- /out/uninstrumented/fuzz-target
DEBUG:qsym.afl:Total=16 s, Emulation=16 s, Solver=0 s, Return=0
DEBUG:qsym.afl:Generate 6 testcases
DEBUG:qsym.afl:0 testcases are new
DEBUG:qsym.afl:Run qsym: input=/out/corpus/afl-slave/queue/id:000773,src:000013,op:havoc,rep:64,+cov
DEBUG:qsym.Executor:Executing timeout -k 5 90 /usr/local/lib/python2.7/dist-packages/qsym/../../../../third_party/pin-2.14-71313-gcc.4.4.7-linux/pin.sh -ifeellucky -t /usr/local/lib/python2.7/dist-packages/qsym/pintool/obj-intel64/libqsym.so -logfile /tmp/tmpZ_7mMx/qsym-out-2/pin.log -i /out/corpus/qsym/.cur_input -s 1 -o /tmp/tmpZ_7mMx/qsym-out-2 -l 1 -b /out/corpus/qsym/bitmap -- /out/uninstrumented/fuzz-target
DEBUG:qsym.afl:Total=17 s, Emulation=17 s, Solver=0 s, Return=0
DEBUG:qsym.afl:Generate 18 testcases
DEBUG:qsym.afl:0 testcases are new
DEBUG:qsym.afl:Run qsym: input=/out/corpus/afl-slave/queue/id:000768,src:000013,op:havoc,rep:64,+cov
DEBUG:qsym.Executor:Executing timeout -k 5 90 /usr/local/lib/python2.7/dist-packages/qsym/../../../../third_party/pin-2.14-71313-gcc.4.4.7-linux/pin.sh -ifeellucky -t /usr/local/lib/python2.7/dist-packages/qsym/pintool/obj-intel64/libqsym.so -logfile /tmp/tmpZ_7mMx/qsym-out-3/pin.log -i /out/corpus/qsym/.cur_input -s 1 -o /tmp/tmpZ_7mMx/qsym-out-3 -l 1 -b /out/corpus/qsym/bitmap -- /out/uninstrumented/fuzz-target

I think I have a good idea of what is happening here. There is a miss-match on expectations on how stdin is used. QSYM uses afl-showmap for coverage accounting to determine if the solutions found actually cover new edges. QSYM also just parses the AFL fuzzer.stats file to determine command line arguments. The issue is that the harness will only execute one input when there's an actually argument provided via the command line.

I think QSYM needs a patch so that it doesn't assume stdin because both the pintool and afl-showmap need to be run with a command-line argument of the cur_input.

[wideglide]

Indeed I do believe that is the problem, but QSYM doesn't seem to be very effective on freetype2?

root@c66a12c00499:/out# ls corpus/qsym/queue/
id:000000,src:id:000774
root@c66a12c00499:/out# grep imported corpus/afl-slave/fuzzer_stats
paths_imported    : 1

[jonathanmetzman]

Interesting, can you share the patch please?

[wideglide]

Here are my notes and patch:

• the binaries in the current docker container don't seem to work because they were compiled with ASAN. I think PIN/QSYM works okay, but afl-showmap only records the first generated input as interesting.
• afl-showmap needs the instrumented version, but executed with @@ instead of reusing the way AFL executes the fuzz-target with the integer for the number of persistent loops.
• PIN/QSYM works best with an uninstrumented (non-ASAN) binary, also with the @@ so that the harness executes a single input and exits.
• /proc/sys/kernel/yama/ptrace_scope needs to be 0

The attached patch file is a minimal change to support libFuzzer style harnesses by providing an argument --libfuzzer-harness <afl_intrumented_binary> on the command line. This results a the QSYM command like:

./qsym/bin/run_qsym_afl.py -a afl-slave -o /out/corpus -n qsym --libfuzzer-harness /out/fuzz-target -- /out/uninstrumented/fuzz-target @@

I have this patch plus some stats tracking functionality in a fork here: https://github.com/wideglide/qsym/tree/fuzzbench_compatible

qsym-afl-showmap-patch.txt

[wideglide]

Another interesting note (https://github.com/google/fuzzbench/pull/1165) also has issues related to afl-showmap because SYMCC uses the same integration as QSYM. Really, this integration isn't the greatest as it writes the bitmap out to disk for every input, then reads the bitmap back in with Python, then checks the bitmap in Python for changes. At least SYMCC uses Rust for the integration/ bitmap review.

[jonathanmetzman]

Thanks, I'll try this again in the next day or two.