google
Debug 2025
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-08-28-empty-seeds-1 --fuzzers aflplusplus_empty_seeds libafl_empty_seeds honggfuzz_empty_seeds libfuzzer_empty_seeds afl_empty_seeds
Experiment 2024-08-28-empty-seeds-1 data and results will be available later at:
The experiment data.
The experiment report.
The experiment report(experimental).
Seeing errors from trials:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/src/fuzzers/libafl_empty_seeds/fuzzer.py", line 71, in fuzz
prepare_empty_corpus(input_corpus)
File "/src/fuzzers/libafl_empty_seeds/fuzzer.py", line 42, in prepare_empty_corpus
shutil.rmtree(input_corpus)
NameError: name 'shutil' is not defined
Did you have a chance to test this locally?
https://github.com/google/fuzzbench/pull/2025/commits/5a717af6438167a4537550122eb9aa248d65ba2b https://github.com/google/fuzzbench/pull/2025/commits/b8eecbe5bdf3181be18739884ad33c3cb5627dd1
Can you cherry pick these two?
Can you cherry pick these two?
I don't think I can. They are from your forked repo, not FuzzBench. You might need to create a PR based on this one to push those commits.
These two are pushed to the branch here https://github.com/google/fuzzbench/pull/2025 (the last two commits)
These two are pushed to the branch here #2025 (the last two commits)
As I said, they are from your forked repo, not FuzzBench. Pushing them to a PR to FuzzBench does not make them FB.
Hence I cannot cherry-pick:
fatal: bad revision '5a717af'
It's better if you could create a PR based on this one and add them.
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-08-30-empty-seeds-1 --fuzzers aflplusplus_empty_seeds libafl_empty_seeds honggfuzz_empty_seeds libfuzzer_empty_seeds afl_empty_seeds
i think we triggered it we have coverage.json here but they are all empty https://storage.googleapis.com/fuzzbench-data/index.html?prefix=2024-08-30-empty-seeds-1/coverage/data/ and the report is not generated either did you see something in the log?
OK, I just had some time to look into this.
Conclusion
The error is likely caused by missing .profraw files during experiments:
This error is from merge_profdata_files(). The error message indicates there is no src_files provided in the command, hence unable to merge them into the destination profdata file (/work/measurement-folders/sqlite3_ossfuzz-libfuzzer_empty_seeds/merged.profdata)
As a result, FuzzBench failed on generate_json_summary() and generate_coverage_report() while there are no profdata_file:
BTW, this is not the missing fuzz target binary error we have seen before, as this message was not printed and all binaries exist under /out:
Next
I reckon the main question is "Is this caused by FuzzBench or the new fuzzers"? I can do three things to help confirm this:
- Add more debug logs to confirm that
src_filesis not in vanilla libfuzzer but empty for its new variation. - Disable merging previous experiment result data.
- Run another exp with
libfuzzerand its new variation.
I will do 1&2 now, and if you have more logs to add, please do so too. Let me know when it is ready, I will do 3.
nevermind you already did this. you can run.
what i suspected is that probably .profraw aren't successfully generated. but since you already added logs https://github.com/google/fuzzbench/pull/2040/commits/b926a58edacb80452072bbaead50b8f25abc6c43 let's see if the profraw data are there
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-09-03-empty-seeds-1 --fuzzers libfuzzer_empty_seeds afl_empty_seeds libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-09-03-empty-seeds-1 --fuzzers libfuzzer_empty_seeds afl_empty_seeds libfuzzer
https://storage.googleapis.com/fuzzbench-data/index.html?prefix=2024-09-03-empty-seeds-1/ this is empty. so nothing was run (?)