fuzzbench Experiment request: frameshift

Hi, I'm interested in evaluating a fork of LibAFL and AFL++ (frameshift variant).

Would you be able to run the following experiment:

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-05-20-frameshift --fuzzers aflplusplus_frameshift libafl_frameshift

May 20 '25 23:05 hgarrereyn

@DonggeLiu would you be able to help here :)

May 21 '25 18:05 hgarrereyn

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-05-22-frameshift --fuzzers aflplusplus_frameshift libafl_frameshift

May 22 '25 00:05 DonggeLiu

I'm still only seeing an input folder here: https://storage.googleapis.com/fuzzbench-data/index.html?prefix=2025-05-22-frameshift/

Does it normally take this long to start up?

May 22 '25 15:05 hgarrereyn

@DonggeLiu it looks like this failed due to insufficient resources in us-central1-c. I've switched it to use us-central1-a, would you be able to rerun?

May 22 '25 18:05 hgarrereyn

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-05-23-frameshift --fuzzers aflplusplus_frameshift libafl_frameshift

May 23 '25 00:05 DonggeLiu

@DonggeLiu thanks! Seems like its not running though--do I need to edit anything else for a rerun?

May 23 '25 03:05 hgarrereyn

@DonggeLiu thanks! Seems like its not running though--do I need to edit anything else for a rerun?

No, I will rerun it. It's weird that the parser was triggered but thought no exp was requested. Sorry that FB was not stable recently.

May 26 '25 01:05 DonggeLiu

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-05-23-frameshift --fuzzers aflplusplus_frameshift libafl_frameshift

May 26 '25 01:05 DonggeLiu

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-05-26-frameshift --fuzzers aflplusplus_frameshift libafl_frameshift

May 26 '25 01:05 DonggeLiu

Is there an available report for this?

Jun 01 '25 02:06 hgarrereyn

The experiment failed due to duplicated experiment name. Let me re-run.

Jun 02 '25 03:06 DonggeLiu

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-06-02-frameshift --fuzzers aflplusplus_frameshift libafl_frameshift

Jun 02 '25 03:06 DonggeLiu

Hi @DonggeLiu thanks for helping with this, it seems like this experiment didn't start however. I'm not seeing a folder in https://storage.googleapis.com/fuzzbench-data/index.html

Jun 02 '25 17:06 hgarrereyn

This very strange, the CI experiment keeps failing for different reasons. Let me start another one.

Jun 04 '25 02:06 DonggeLiu

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-06-04-frameshift --fuzzers aflplusplus_frameshift libafl_frameshift

Jun 04 '25 02:06 DonggeLiu

Hmm unfortunately still not seeing an experiment folder

Jun 04 '25 12:06 hgarrereyn

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-06-04-frameshift --fuzzers aflplusplus_frameshift libafl_frameshift

Jun 04 '25 12:06 jonathanmetzman

Thanks @jonathanmetzman! Feel free to run with the systemd and php benchmarks disabled, I'm not sure why those are failing but I'm seeing that locally too.

Jun 04 '25 13:06 hgarrereyn

The experiment infra seems to be broken right now and this experiment didn't seem to work

Jun 05 '25 19:06 jonathanmetzman

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-06-11-frameshift --fuzzers aflplusplus_frameshift libafl_frameshift

Jun 10 '25 21:06 DonggeLiu

@DonggeLiu @jonathanmetzman has there been any progress on the experiment infra here?

Jul 04 '25 18:07 hgarrereyn