google
[DO NOT MERGE] Adding code for experiments using pubsub queues
This PR adds code to use pub sub queues on the measurer when running cloud experiments, instead of python in-memory queues.
This PR is still a draft, I still want to do some improvements on top of it, but I appreciate any feedback.
Will try to trigger a cloud experiment before adding reviewers to it.
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name measurer-pub-sub-test-experiment --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pub-sub-test-experiment --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pub-sub-test-experiment --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pub-sub-test-experiment-2 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name metzman-2024 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name metzman-2024-2 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pubsub-measurer-1 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pubsub-measurer-2 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pubsub-measurer-3 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pubsub-measurer-4 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pubsub-measurer-5 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pubsub-measurer-6 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pubsub-measurer-7 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pubsub-measurer-8 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pubsub-measurer-10 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pubsub-measurer-11 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pubsub-measurer-12 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pubsub-measurer-14 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pubsub-measurer-15 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pubsub-measurer-16 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pubsub-measurer-17 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pubsub-measurer-18 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pubsub-measurer-19 --fuzzers afl libfuzzer
/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --benchmarks sqlite3_ossfuzz bloaty_fuzz_target --experiment-name pubsub-measurer-20 --fuzzers afl libfuzzer
This PR is still a WIP.
In the experiments I've tried to run, it seems that we are having a problem in starting the measurer workers processes.
I couldn't debug to know the reason why this is happening.
As today is my last day at Google, am I afraid that I won't be able to debug any further and merge this PR so I am commenting to let you know that this is the current state of it.