Adding KFuzz and requesting an experiment

[KFuzzing]

Hi @DonggeLiu,

I've just added a new fuzzer, KFuzz, to FuzzBench. Could you assist me in running an experiment to test it?

Thank you!

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-10-10-kfuzz --fuzzers kfuzz

[google-cla[bot]]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[DonggeLiu]

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-10-11-kfuzz --fuzzers kfuzz

[KFuzzing]

Hi @DonggeLiu,

I found KFuzz encountered some bugs while fuzzing certain parts of the programs and I have successfully fixed them.

Could you help me run the experiment again?

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-10-11-kfuzz-1 --fuzzers kfuzz

[KFuzzing]

Hi @DonggeLiu,

Could you please help me run the experiment? Thank you!

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-10-14-kfuzz --fuzzers kfuzz

[DonggeLiu]

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-10-14-kfuzz --fuzzers kfuzz

[KFuzzing]

Hi @DonggeLiu ,

Thank you for your help! Fuzzbench is a nice work!

Based on the current results, I have found that my tool performs well on some programs but struggles with others. As a result, I redesigned certain scheduling strategies and achieved promising outcomes in local experiments.

1. I have got some useful information from the previous experiment. I think it can be stopped to avoid wasting computing resource.
2. Could you please help me run the experiment for KFuzz using the modified strategy?

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-10-14-kfuzz-1 --fuzzers kfuzz

[DonggeLiu]

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-10-14-kfuzz-1 --fuzzers kfuzz

[KFuzzing]

Hi @DonggeLiu ,

I have updated the strategies, could you please help me run a new experiment to test it?

Thank you!

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-10-18-kfuzz --fuzzers kfuzz

[addisoncrump]

Heads up, I'm not a maintainer of this project :sweat_smile:

[KFuzzing]

Heads up, I'm not a maintainer of this project 😅

I am sorry to bother you ~

[DonggeLiu]

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-10-21-kfuzz --fuzzers kfuzz

[KFuzzing]

Hi @DonggeLiu,

I just added four fuzzers to perform a comparison experiment. could you please help me run a new experiment?

Thank you!

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-10-22-aflpp-cmp --fuzzers aflplusplus_mopt kfuzz_mopt aflplusplus_aflfast kfuzz_aflfast

[DonggeLiu]

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-10-22-aflpp-cmp --fuzzers aflplusplus_mopt kfuzz_mopt aflplusplus_aflfast kfuzz_aflfast

[KFuzzing]

Hi @DonggeLiu,

It seems that 2024-10-22-aflpp-cmp has failed during the gcr.io/fuzzbench/experiment-runner step. I don’t have permission to access the error details. Could you please help me check it?

Thank you!

[DonggeLiu]

It seems that 2024-10-22-aflpp-cmp has failed during the gcr.io/fuzzbench/experiment-runner step. I don’t have permission to access the error details. Could you please help me check it?

This appears to be due to insufficient resource:

Let me re-run the experiment, and if it fails, could you please change this line to us-central1-a? Thanks

https://github.com/google/fuzzbench/blob/4f2d7ccbb1e194de34345201ea5793d6f4e507f1/service/experiment-config.yaml#L9

[DonggeLiu]

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-10-22-aflpp-cmp --fuzzers aflplusplus_mopt kfuzz_mopt aflplusplus_aflfast kfuzz_aflfast

[KFuzzing]

Hi @DonggeLiu,

It seems that the experiment still hasn't run. I've already updated the settings.

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-10-22-aflpp-cmp-1 --fuzzers aflplusplus_mopt kfuzz_mopt aflplusplus_aflfast kfuzz_aflfast

[DonggeLiu]

It seems that the experiment still hasn't run. I've already updated the settings.

Strange, I found the The experiment data is created, but only contains input/. The gcbrun page shows success too.

Cloud log did not show anything about the new exp either.

I will start a new exp just in case.

[DonggeLiu]

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-10-22-aflpp-cmp-1 --fuzzers aflplusplus_mopt kfuzz_mopt aflplusplus_aflfast kfuzz_aflfast

[KFuzzing]

Hi @DonggeLiu,

It’s strange—the experiment ran successfully (2024-10-22-aflpp-cmp-1), but no report was generated. Could you please have a look?

Thank you!

[DonggeLiu]

Yep I saw many errors when generating coverage reports.

This error happened elsewhere before and keeps bothering us occasionally. We don't have the manpower to fix this at the moment, unfortunately.

I don't think it is caused by your fuzzers given this PR had many successful results. But if you think reverting some of your commits can mitigate this, I am happy to run more experiments to test if FuzzBench can generate reports for you. Sadly I won't be able to debug this error given I will be traveling for 2 weeks from tomorrow.

[KFuzzing]

Sadly I won't be able to debug this error given I will be traveling for 2 weeks from tomorrow.

Thank you for your response! If I need to run experiments in the next two weeks, who should I contact? Lastly, I wish you a pleasant holiday!

[KFuzzing]

Hi @DonggeLiu,

I have updated the strategies, could you please help me run a new experiment to test it?

Thank you!

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-11-18-kv1 --fuzzers aflplusplus_dev kfuzz

[DonggeLiu]

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-11-18-kv1 --fuzzers aflplusplus_dev kfuzz

[KFuzzing]

Hi @DonggeLiu,

I want to run another version of my method. Could you please help me run it?

Thank you!

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-11-19-kv1 --fuzzers kfuzz

[DonggeLiu]

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2024-11-19-kv1 --fuzzers kfuzz