fuzzbench SBFT'23 Benchmarks

Will these benchmarks be added to FuzzBench? Seems like they were added and removed at some point.

May 11 '23 17:05 mvanotti

@Alan32Liu @jonathanmetzman thoughts?

May 24 '23 23:05 oliverchang

Will these benchmarks be added to FuzzBench? Seems like they were added and removed at some point.

Sorry I missed this:

These benchmarks are available under the SBFT'23 tag. We removed them from the master branch after the competition because running every experiment with them is too time-consuming.

May 25 '23 05:05 DonggeLiu

Thanks! Would it make sense to keep them in the main branch but not run them or to at least document that for newcomers?

Jun 15 '23 19:06 mvanotti

Thanks! Would it make sense to keep them in the main branch but not run them or to at least document that for newcomers?

May I ask what is the benefit of having them in the master branch? IIUC, that would require extra work in filtering those benchmarks from automatic experiments. Most experiments do not use them anyway.

Jun 16 '23 01:06 DonggeLiu

I reckon it is a good idea to document that. I can post our paper with the link to SBFT'23 tag in the publication section, once ICSE has a formal link to the paper.

Jun 16 '23 01:06 DonggeLiu