fuzzbench icon indicating copy to clipboard operation
fuzzbench copied to clipboard
verified publisher icon google

A test experiment to compare selected fuzzers on selected benchmarks.

Open DonggeLiu opened this issue 2 years ago • 4 comments

(Not need to merge: This PR is for experiment purpose only)

Coverage

Fuzzers

  1. hastefuzz
  2. aflplusplus
  3. aflplusplus_d82218 (new version)
  4. aflplusplusplus
  5. aflplusplusplus_d82218 (based on new afl++)
  6. aflrustrust
  7. pastis
  8. libafl_libfuzzer

Benchmarks (~29 in total):

Coverage benchmarks that are compatible with all fuzzers.

Bug

Fuzzers

  1. hatefuzz
  2. aflplusplus
  3. aflplusplus_d82218 (new version)
  4. aflplusplusplus
  5. aflplusplusplus_d82218 (based on new afl++)
  6. aflrustrust
  7. pastis
  8. aflmart_plusplus
  9. afl

Benchmarks (~14 in total):

Bug benchmarks that are compatible with all fuzzers.

DonggeLiu avatar May 08 '23 03:05 DonggeLiu

/gcbrun run_experiment.py --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-05-08-test-new-aflpp --fuzzers hastefuzz aflplusplus aflpp_d82218 aflplusplusplus aflppp_d82218 aflrustrust pastis libafl_libfuzzer --benchmarks arduinojson_json_fuzzer bloaty_fuzz_target botan_tls_server curl_curl_fuzzer_http double-conversion_string_to_double_fuzzer draco_draco_pc_decoder_fuzzer firestore_firestore_serializer_fuzzer fmt_chrono-duration-fuzzer freetype2_ftfuzzer guetzli_guetzli_fuzzer harfbuzz_hb-shape-fuzzer icu_unicode_string_codepage_create_fuzzer jansson_json_load_dump_fuzzer jsoncpp_jsoncpp_fuzzer lcms_cms_transform_fuzzer libaom_av1_dec_fuzzer libcoap_pdu_parse_fuzzer libhevc_hevc_dec_fuzzer libjpeg-turbo_libjpeg_turbo_fuzzer libpcap_fuzz_both libpng_libpng_read_fuzzer libxml2_xml libxslt_xpath mbedtls_fuzz_dtlsclient openh264_decoder_fuzzer openssl_x509 openthread_ot-ip6-send-fuzzer proj4_proj_crs_to_crs_fuzzer re2_fuzzer sqlite3_ossfuzz stb_stbi_read_fuzzer systemd_fuzz-link-parser vorbis_decode_fuzzer -a

DonggeLiu avatar May 08 '23 04:05 DonggeLiu

/gcbrun run_experiment.py --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-05-08-test-new-aflpp-bug --fuzzers hastefuzz aflplusplus aflpp_d82218 aflplusplusplus aflppp_d82218 aflrustrust pastis aflsmart_plusplus afl --benchmarks aspell_aspell_fuzzer_e8eb74 assimp_assimp_fuzzer_4d451f bloaty_fuzz_target_52948c ffmpeg_ffmpeg_demuxer_fuzzer_7adeef file_magic_fuzzer_2d5f85 grok_grk_decompress_fuzzer_9cd001 harfbuzz_hb-shape-fuzzer_17863b lcms_cms_transform_all_fuzzer_97d37d libaom_av1_dec_fuzzer_6e1848 libpcap_fuzz_filter_98b0a2 libxml2_xml_e85b9b mbedtls_fuzz_dtlsclient_7c6b0e php_php-fuzz-parser_0dbedb systemd_fuzz-network-parser_288baf -a

DonggeLiu avatar May 08 '23 06:05 DonggeLiu

Coverage: Experiment data. Experiment report.

Bug: Experiment data. Experiment report.

DonggeLiu avatar May 08 '23 06:05 DonggeLiu

/gcbrun run_experiment.py --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2023-05-09-new-aflpp-bug --fuzzers hastefuzz aflplusplus aflpp_d82218 aflplusplusplus aflppp_d82218 aflrustrust pastis aflsmart_plusplus afl --benchmarks aspell_aspell_fuzzer_e8eb74 assimp_assimp_fuzzer_4d451f bloaty_fuzz_target_52948c ffmpeg_ffmpeg_demuxer_fuzzer_7adeef file_magic_fuzzer_2d5f85 grok_grk_decompress_fuzzer_9cd001 harfbuzz_hb-shape-fuzzer_17863b lcms_cms_transform_all_fuzzer_97d37d libaom_av1_dec_fuzzer_6e1848 libpcap_fuzz_filter_98b0a2 libxml2_xml_e85b9b mbedtls_fuzz_dtlsclient_7c6b0e php_php-fuzz-parser_0dbedb systemd_fuzz-network-parser_288baf -a

DonggeLiu avatar May 09 '23 04:05 DonggeLiu