google
libFuzzer, entropic patch.diff is causing crashes on "No such file or directory" when provided with seed inputs
E.g. run
make run-libfuzzer-matio_matio_fuzzer
You will see a crash
8851: cov: 137 ft: 190 corp: 38 exec/s 110 oom/timeout/crash: 7/0/0 time: 36s job: 8 dft_time: 0
No such file or directory: ��hg; exiting
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/out/fuzzer.py", line 44, in fuzz
extra_flags=['-keep_seed=1', '-cross_over_uniform_dist=1'])
File "/out/fuzzer.py", line 76, in run_fuzzer
subprocess.check_call(command)
File "/usr/local/lib/python3.7/subprocess.py", line 363, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/out/matio_fuzzer', '-print_final_stats=1', '-close_fd_mask=3', '-fork=1', '-ignore_ooms=1', '-ignore_timeouts=1', '-ignore_crashes=1', '-detect_leaks=0', '-keep_seed=1', '-cross_over_uniform_dist=1', '/out/corpus', '/out/seeds']' returned non-zero exit status 1.
ERROR:root:Executed command: "nice -n 5 python3 -u -c import fuzzer; fuzzer.fuzz('/out/seeds', '/out/corpus', '/out/matio_fuzzer')" returned: 1.
ERROR:root:Fuzz process returned nonzero. Extras: {'traceback': 'Traceback (most recent call last):\n File "/src/experiment/runner.py", line 224, in run_fuzzer\n env=fuzzer_environment)\n File "/src/common/new_process.py", line 124, in execute\n raise subprocess.CalledProcessError(retcode, command)\nsubprocess.CalledProcessError: Command \'[\'nice\', \'-n\', \'5\', \'python3\', \'-u\', \'-c\', "import fuzzer; fuzzer.fuzz(\'/out/seeds\', \'/out/corpus\', \'/out/matio_fuzzer\')"]\' returned non-zero exit status 1.\n'}
This is blocking the work on trying to run an experiment with OSS-Fuzz corpora on all benchmarks to see if they can break the coverage wall.
@mboehme - can you please take a look or suggest fix for that https://github.com/google/fuzzbench/blob/master/fuzzers/libfuzzer/patch.diff
Likely coming from this line: https://github.com/google/fuzzbench/blob/master/fuzzers/libfuzzer/patch.diff#L83
RunOneMergeJob calls GetSizedFilesFromDir, which calls ListFilesInDirRecursive. ListFilesInDirRecursive prints a matching error message when the directory doesn't exist.
It seems Job->CorpusDir is uninitialized.
Not sure, I haven't actually repro'd or debugged. Just was looking at the source code.
need to run make run-libfuzzer-matio_matio_fuzzer for a few minutes. ci only does 1-2 min.
I disabled the patch in PR #806. Not fixed, yet, but it should work now. Will look at it later.
friendly ping! if you can please look into it in the next week or two, would appreciate it.
Sure. Been busy with paper reviews and presentations, recently. Will get to it asap.
Cannot reproduce. I ran libfuzzer manually under make debug-libfuzzer-matio_matio_fuzzer for 10 minutes and one more time under make run-libfuzzer-matio_matio_fuzzer for 10 minutes. I get the following ASAN output, but it seems unrelated (since matio is a bug benchmark)
#16185: cov: 621 ft: 967 corp: 68 exec/s 13 oom/timeout/crash: 17/0/53 time: 567s job: 71 dft_time: 0
read_data_impl.h:58:5: runtime error: -nan is outside the range of representable values of type 'int'
read_data_impl.h:58:5: runtime error: -nan is outside the range of representable values of type 'int'
==436==ERROR: AddressSanitizer: allocator is out of memory trying to allocate 0x2929ee0928 bytes
#16447: cov: 621 ft: 969 corp: 70 exec/s 5 oom/timeout/crash: 17/0/54 time: 613s job: 72 dft_time: 0
read_data_impl.h:58:5: runtime error: -nan is outside the range of representable values of type 'int'
read_data_impl.h:58:5: runtime error: -nan is outside the range of representable values of type 'int'
==444==ERROR: AddressSanitizer: requested allocation size 0x170000018000000 (0x170000018001000 after adjustments for alignment, red zones etc.) exceeds maximum supported size of 0x10000000000 (thread T0)
Matio is trying to allocate a lot of memory. Is LF trying to write to a full ram-disk?
Can you try running it again and attach the fuzzer-log.txt?
PR #836 might have resolved it: https://github.com/google/fuzzbench/pull/836/files#diff-81938984514c7725d3ff9e1b378fdbaa0f19254c74121a1ab74a30d11dfe8828
yes does not reproduce anymore with matio for libfuzzer, entropic. closing for now, will reopen if i see this again.